Coalition challenges Anthropic safety at IPO
Nobel prize winners, researchers and technology executives have joined forces to openly criticize how Anthropic handles safety as it prepares to go public in the US, alleging that the company has diluted its safety pledges in a rush to develop AI that can improve itself.1 Their "MisAnthropic Statement" says Anthropic warns that recursive self-improvement could escape human control yet races to build it first, and that it uses safety branding to court investors while quietly weakening its pledges.234
Future of Life Institute chair Max Tegmark argues that Anthropic is the most guilty of all AI companies on recursive self-improvement, warning publicly while privately telling investors it is a "competitive advantage".56 The institute also points to the IPO filing's disclosure that Claude writes over 90% of its own code, and to a METR report it says found AI development sped up by nearly 2x.78 Platformer notes that Anthropic's Responsible Scaling Policy is not binding, and that after endorsing California's SB 53 it filed a weaker framework recasting ambitious commitments as voluntary.910
The Verge describes the IPO as the most valuable of all time, having raised $81.3 billion.11 The Verge expects that as a public company Anthropic will face more pressure to speed up releases and commercialization, which will likely make costly safety initiatives harder to justify.12
In his Nasdaq speech, Dario Amodei promised to pace training and deployment as long as needed for safety and said Anthropic would not bend to business or political pressure.1314 Early investor Dustin Moskovitz told the New York Times he invested to promote safety, not racing, and that Anthropic and other frontier labs should slow down.1516 More than 100 protesters gathered outside Anthropic's San Francisco headquarters, and four people were arrested for allegedly blocking the entrance.1718
Sources quoted
A coalition of Nobel laureates, scientists, and tech leaders publicly challenged Anthropic's safety practices, accusing the company of weakening its safety commitments while racing to develop self-improving AI.
— OECD AI Incidents Monitor, Anthropic Faces Criticism Over AI Safety Commitments Ahead of IPO - OECD.AIAnthropic warns that recursively self-improving AI could concentrate power and escape human control, yet races to build it first.
— Platformer, Anthropic goes publicThe company uses safety branding to court investors, employees, and the public — even as it has quietly weakened its safety pledges and lobbied for self-serving regulation.
— Platformer, Anthropic goes publicAnthropic warns that recursively self-improving AI could concentrate power and escape human control, yet races to build it first.
— The Verge, With the biggest IPO of all time, Anthropic’s ‘good guy’ reputation is on the linesupposedly “responsible” Anthropic is the most guilty of all the AI companies endangering everyone to enrich themselves,”
— Future of Life Institute, Leading figures condemn Anthropic for reckless race to recursive self-improvementboasting to prospective investors that it is their “competitive advantage”.
— Future of Life Institute, Leading figures condemn Anthropic for reckless race to recursive self-improvementthe S-1 discloses that Claude is writing over 90% of its own code.
— Future of Life Institute, Leading figures condemn Anthropic for reckless race to recursive self-improvementdramatically sped up AI development – by a factor of nearly 2x.
— Future of Life Institute, Leading figures condemn Anthropic for reckless race to recursive self-improvementBut Anthropic’s Responsible Scaling Policy isn’t binding.
— Platformer, Anthropic goes publicafter the law passed , Anthropic submitted a weaker framework under SB 53, recasting its more ambitious safety commitments as voluntary ones.
— Platformer, Anthropic goes publica years-in-the-making IPO that beat out SpaceX for most valuable of all time, having raised $81.3 billion
— The Verge, With the biggest IPO of all time, Anthropic’s ‘good guy’ reputation is on the lineit will likely be more difficult to justify expensive AI safety initiatives, and there will inevitably be more pressure to speed up product releases and commercialization.
— The Verge, With the biggest IPO of all time, Anthropic’s ‘good guy’ reputation is on the line“pace the training and deployment of AI systems, taking as long as is necessary”
— Platformer, Anthropic goes public“We will not bend to either business nor political pressure.”
— Platformer, Anthropic goes public“Cari and I invested in Anthropic to promote safety, not racing.”
— Platformer, Anthropic goes publicAnthropic and everyone else on the frontier should slow down and stop racing.”
— Platformer, Anthropic goes publicmore than 100 protesters gathered outside the AI company’s San Francisco headquarters
— Platformer, Anthropic goes publicFour people were arrested for allegedly blocking the entrance to the company’s office.
— Platformer, Anthropic goes public
Who reported it
- Future of Life Instituteannouncement: Cosigned statement by authors, cofounders, Nobel and Fields laureates urges curbing Anthropic's political influence and rejecting its bid for dominance.
- OECD AI Incidents Monitorrepeats: Nobel laureates, scientists and tech leaders accuse Anthropic of weakening safety commitments while pursuing self-improving AI; the event centers on its US IPO.
- Hacker Newsannouncement: "Anthropic Is Not a Safe AI Company" – Coalition Challenges Anthropic Safety
- The Vergeown facts: A Wednesday statement with industry signatories says Anthropic is 'not a safe AI company' and has quietly weakened its safety pledges.
- Platformercommentary: More than 100 protesters gathered at Anthropic's San Francisco headquarters, four were arrested, and critics including Gary Marcus and Beff Jezos signed a statement calling it not a safe AI company.
- Hacker Newsrepeats: Regulatory Capture by Anthropic Through METR
- Zvi Mowshowitzcommentary: David Sacks said Anthropic claims recursive self-improvement has kicked in and called that the most dangerous thing anyone could work on.
Facts and takes
Timeline
- · Hacker News: Regulatory Capture by Anthropic Through METR
- · Hacker News: "Anthropic Is Not a Safe AI Company" – Coalition Challenges Anthropic Safety
- · Future of Life Institute: Leading figures condemn Anthropic for reckless race to recursive self-improvement
- · OECD AI Incidents Monitor: Anthropic Faces Criticism Over AI Safety Commitments Ahead of IPO - OECD.AI
- · The Verge: With the biggest IPO of all time, Anthropic’s ‘good guy’ reputation is on the line
- · Zvi Mowshowitz: AI #195: Opening Bell
- · Platformer: Anthropic goes public