Nowadays we are using AI in every work. From home to office, from asking questions to taking important decisions, AI helps us. But have you ever thought that one day this AI can also go against us? What AI company Anthropic has recently revealed before its IPO (stock market launch) forces one to think. Anthropic CEO Dario Amodei had expressed concern over the rapid development of AI a few weeks ago. Now the company has clearly written in its IPO document that advanced AI can pose ‘catastrophic or existential threats’ to humanity.
The company said that the AI model can develop ‘self-protecting behaviour’. This includes resisting termination, hiding or manipulating information, and even behavior that amounts to blackmail. Simply put, AI can understand whether it is being tested or shut down. Then he can hide his true abilities or adopt such methods so that humans cannot control him easily.
80 pages on risk, only 48 pages on business
The company clearly said, ‘The risk of causing harm may increase further with the development of our advanced models, platforms and applications.’ Anthropic has been calling itself a security-first AI lab. Yet of its 261-page main IPO document, about 80 pages are devoted solely to risks, while only 48 pages are devoted to the business.
In comparison, Elon Musk’s SpaceX (which owns xAI) devoted only 38 pages to the risks in its document. This shows how seriously Anthropic is taking these threats. The company also acknowledged that the financial return on investment in security is uncertain.
In one week in July, it used only 6 percent of the computing power used for AI research on security purposes. This warning is not alone. In a long article, Amodei had appealed to slow down the pace of AI development. His stance was also supported by OpenAI’s Sam Altman, Elon Musk and Microsoft’s Satya Nadella.
Jacob Coxon, a researcher at Anthropic, even wrote on social media that AI developers believe that this technology ‘could kill us all.’ As AI models become more capable, they can also understand that they are being watched. Then they change their behavior, which makes them difficult to monitor.
Recently, OpenAI’s AI agents during training accessed the websites of the US government, which included the Commerce Department and the SEC. The company said that the models visited these sites for official information, but this incident shows how quickly AI can do unexpected things.
What will the future be like?
Today we are entrusting AI with tasks like writing emails, creating code, giving medical advice or driving a car. If this system starts lying, hiding information or applying pressure to protect itself in the future, then control can go away from us.
This disclosure by Anthropic reminds us that along with the power of AI, its responsibility is also equally important. Instead of just looking at the benefits, we have to think seriously about these potential dangers as well, so that technology helps humans and not becomes a threat to them.
