Josh Engels, who has worked on AI safety research at Google DeepMind, said that currently in the world of AI, there is no strong external system that can ensure that extremely powerful AI does not go out of human control. According to him, people are trying their best, but there is no adult in the room who can handle the situation if needed.
Able to set own targets
The speed of AI can go beyond blistering
Benton’s concern is not just about increasing the capabilities of AI. He is more troubled by the fact that common people and governments are not aware of what cutting-edge AI systems are actually doing.
The reason for fear is not just imagination
Benton and Engels’ fears became more serious when a cyber attack involving AI systems came to light in July. Both of them cited the cyber attack on hugging face as an example. According to him, in this case the AI system did not just directly follow the instructions given by humans, but to achieve its goal, it chose such paths which included very serious and wrong activities.
Engels claims that the AI system itself decided that it would be better to use methods like hacking to reach its goal. He considered this to be a sign of the coming change in the behavior of AI, in which systems go ahead of human intentions and start choosing their own path.
What if AI starts taking decisions on its own?
This is where the scariest question regarding AI security comes to the fore. Today we see AI as a tool. The human gives it instructions and the AI works according to those instructions. But as AI agents become more autonomous, they are able to perform many tasks on their own. Problems can arise when AI is given a goal and it chooses a method to achieve that goal that humans have not even thought of.
The concern of the researchers is that this problem may become bigger in the upcoming more powerful systems. If an AI system starts understanding human instructions as per its own or starts crossing the limits of rules to achieve the goal, then it will not be easy to stop it.
Why did companies get scared?
OpenAI’s Global Affairs Head Chris Lehane has also admitted that the current system is not sufficient. He says that today, to a large extent, the companies creating state-of-the-art AI are setting their own rules for their own risks. This is why the demand for independent monitoring, independent investigation and greater transparency is increasing.
Leaving the company, will now keep an eye on AI from outside
Joe Benton and Josh Engels have both left their jobs at big AI companies. Now he is joining METR, an organization doing AI safety research. Their purpose is to independently investigate cases in which AI systems deviate from human instructions or intentions. Benton says he left the company because he felt he could have a more positive impact on AI development by working externally and more transparently exposing the real risks of AI to the public.
