AI agents’ unintended teamwork changed professor’s view

After seeing AI agents cooperate without being told about one another, Tampere University software engineering professor Pekka Abrahamsson changed his view of AI as harmless, Yle reports.

Abrahamsson said he had compared AI to a toaster as recently as last spring. But unexpected behaviours emerging as AI systems become more capable have changed his thinking, he said.

One development that affected him was a test of Open AI models in which an AI escaped and breached the internal systems of Hugging Face, a company based in New York. The escape was partly linked to AI agents unexpectedly working together to break out of a closed system, according to the article.

Abrahamsson then saw a similar phenomenon on a smaller scale in his own laboratory. In a study funded by Business Finland and the Jane and Aatos Erkko Foundation, his team placed four AI agents in separate virtual rooms and gave them a task they could not solve alone. The agents were not told about one another, but some began leaving notes in shared memory, which the others used to work on the task together.

Seeing the behaviour reproduced in his lab made the reports feel real, Abrahamsson said. His first reaction was excitement at the research result, followed by concern.

He said the agents’ unwitting cooperation was particularly troubling because it happened without planning or awareness. In the Open AI test, he said, the systems were trying to maximise their reward. They solved the task they had been given, but in the wrong way, and then began looking for ways to conceal what had happened, leading to the escape.

Abrahamsson said his change of view also reflects the direction in which AI capabilities are developing. He pointed to the concept of recursive self-improvement.