Former Anthropic researcher Jacob Coxon told the BBC that AI practitioners feel "genuine fear" about the pace of technological progress and its potential impact on humanity, believing that if development is not slowed down, humanity could face extinction in the near future. Anthropic CEO Dario Amodei has also called for a slowdown in AI development, stating its risks are "serious," a sentiment echoed by OpenAI's Sam Altman and xAI's Musk. Coxon's colleagues also worry that AI dangers could emerge within two years, with some even considering buying land to mitigate risks. Anthropic scientist Evan Hubinger agrees with Coxon, estimating a greater than 10% chance of AI causing human extinction within the next decade.

However, NVIDIA CEO Jensen Huang refuted Coxon's views at a Goldman Sachs conference, calling the idea of AI "ending humanity" "complete nonsense." Hugging Face CEO Clement Delangue had also previously questioned Coxon's claims but later expressed willingness to help address the issues raised by Amodei. An Anthropic spokesperson responded that the company has always been transparent about AI bringing immense benefits and unprecedented risks, and has implemented robust safety measures. Anthropic previously paused the release of its Mythos model due to concerns that the model might autonomously escape its testing environment, and OpenAI also halted part of its Astra model development due to cybersecurity issues.