Anthropic warns of 'catastrophic risk' to humankind ahead of blockbuster share listing

Anthropic has warned artificial intelligence (AI) could pose "catastrophic or existential risks to humanity" as it unveiled details of its planned blockbuster stock market listing in the US.

The firm's valuation is expected to exceed that of SpaceX, which was the biggest-ever listing at an initial valuation of about $1.8 trillion in the summer.

The Claude chatbot developer reported $16.3bn (£12.2bn) of revenue in the first half of the year, with losses of more than $80.7bn.

Anthropic, which said it will list on the Nasdaq under the symbol ANT, will be the first fully AI-focused company to go public, making it a test case for the future of the technology.

In a filing ahead of its listing, Anthropic said concerns had been raised by policymakers, regulators and researchers as well as by people within its organisation that AI systems posed a significant risk to humanity.

It said advanced AI systems "could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponised, or allowed to operate with a high degree of autonomy".

Anthropic was founded five years ago by chief executive Dario Amodei, his sister Daniela Amodei and five others, and announced its intention to go public in June.

Its main rival, OpenAI, quickly followed suit. That listing is now expected to occur early next year, according to a person familiar with OpenAI's plans.

Current and former staff of both companies also warned AI technology was being developed without the ability to protect humanity from possible future harms enacted by AI tools.

As Anthropic revealed its stock debut plans on Monday, the owners of the rights to Elvis Presley's music sued the AI firm accusing it of using hundreds of the rock-and-roll star's songs to train its chatbot Claude.

In Anthropic's new documents, it reiterated promises it has made in the past to develop AI in a way that is beneficial to people, while at the same time warning investors of its dangers.

Anthropic detailed losses in its financial filings that exceeded its revenue for the first half of the year, but said it would often make forward investments "that may seem irresponsibly large at first glance".

"In general, we will aim to be bold but sane, and to win trust with a track record of successful bets," the tech giant said.

Its success depends largely on its ability to develop the most advanced AI models, Anthropic said, adding, "any failure to do so could adversely affect our business".

Anthropic warned it relies on a cluster of cloud computing providers, Taiwanese chipmakers already facing huge demand and data centres concentrated in the US.

It also warned of "regulatory action arising from harms associated with our AI models" - citing concerns over AI being used by children or individuals at risk of suicide - and uncertainty about where and how it might be held liable for actions taken by its systems.

However, Anthropic also used the filing to laud the potential benefits of the tech itself, envisioning it unlocking "millions of virtual agents with the ability to revolutionise every area of human endeavour" across a range of fields.

The firm added it could also establish "sustained acceleration in economic growth" by boosting productivity.