
For decades, the idea of a machine capable of turning against its creators belonged to the realm of science fiction. Today, some of the people building the most advanced artificial intelligence systems discuss that possibility as a real risk.
Jacob Coxon, a researcher who worked at OpenAI and Anthropic, resigned from the latter company last week, accusing both firms of racing toward a "self-improving superintelligence" and gambling with everyone's lives. His former boss, Evan Hubinger, was even more explicit; he estimates there is a greater than 10% chance that Artificial Intelligence (AI) will kill all humans within the next decade.
What has changed to make specialized researchers speak this way? One reason is so-called "recursive self-improvement." The term sounds complex, but it describes a simple concept: using an artificial intelligence to help build a better one, which then contributes to creating an even more capable system. Rishub Jain left Google DeepMind in June precisely because of this concern. As he explained to Wired magazine, using the models' programming capabilities to accelerate the development of their successors was progressively reducing human intervention.
Laboratories are experimenting with thousands of AI agents working simultaneously on a wide variety of problems. In this context, an agent is a program capable of pursuing a goal, executing actions, and making certain decisions without waiting for human instructions at every step. The more agents involved, the harder it becomes to oversee the entire system.
This brings up another term frequently cited by AI "doomsayers": "alignment." This refers to ensuring that a highly capable system does what humans want it to do, without finding unexpected ways to achieve its goals. The problem is that, as of now, no one has a method to guarantee such total obedience from a hypothetical superintelligence. Nate Soares, a researcher at the Machine Intelligence Research Institute, notes that some specialists expected alignment to become easier as models improved; in his view, the opposite is happening.
This does not imply that a computer will suddenly wake up tomorrow with a desire to exterminate humanity. The concern is that an extremely competent system might pursue a goal incompatible with our safety and manage to prevent itself from being shut down. Soares even cites the hypothetical scenario of an AI connected to a biological laboratory that could use a biological threat to protect itself. However, these remain theoretical scenarios rather than established facts. Nor is there a scientific consensus regarding the likelihood of an AI-induced extinction event.
The significance of the story revealed by Coxon—a researcher at Anthropic—is that the fear no longer stems solely from outside critics. In July, more than a thousand AI engineers signed an open letter calling for a coordinated slowdown in the development of advanced systems.
The catastrophe these researchers fear may never come to pass, yet tangible harm is already evident: disinformation, AI-assisted cyberattacks, and the rapid integration of these systems into the military sphere—enabling targeted killings and unprecedented mechanisms of social control. For now, scientists warn that picturing the Terminator might help us focus on the main issue: halting—in time—a technological race whose rules, speed, and risks are determined by companies that care only about money.





