Washington: Public warnings from current and former researchers at Anthropic, OpenAI, and Google DeepMind regarding the potential catastrophic risks posed by advanced artificial intelligence have spurred calls from at least 22 serving US officials for stringent regulation, congressional hearings, and, in some cases, a halt in development.
According to Anadolu Agency, Jacob Coxon, a researcher previously associated with Anthropic and OpenAI, resigned from Anthropic and cautioned that leading AI companies were hastening toward self-improving superintelligence without adequate safeguards. Coxon expressed a grave belief among developers that AI could potentially lead to human extinction by the end of the decade. His resignation was followed by a chorus of similar concerns from other researchers, urging stronger safeguards and government oversight to address the pace of AI development, potential loss of human control over advanced systems, and the concentration of technological power within a few companies.
Samuel Marks, Anthropic's scalable oversight lead, stated on X that AI developers are concerned their technology could lead to human extinction or other dire outcomes in the near future. He attributed the rapid development of powerful AI systems to commercial incentives and the necessity to compete with less responsible developers. Evan Hubinger, Anthropic's alignment science lead, endorsed these warnings, estimating a more than 10% probability of human extinction within the next decade due to AI. Similarly, Alex Turner, a former research scientist at Google DeepMind, and Jason Wolfe from OpenAI echoed these concerns, describing the current pace of AI progress as alarming.
OpenAI's chief scientist, Jakub Pachocki, emphasized the need for extreme caution in a recent essay, calling for mandatory safety thresholds and international coordination in AI development. Paul Christiano, newly appointed to the OpenAI Foundation Board, warned of the catastrophic risks associated with the alignment challenges of advancing AI capabilities.
Jonathan Richard Schwarz, a former senior research scientist at Google DeepMind, raised concerns about the concentration of technological power within a small group of AI companies, describing the current state of the field as unhealthy for society.
The political response to these warnings has been bipartisan, though predominantly Democratic-led, with 18 Democrats, three Republicans, and one independent calling for government action. Proposals from lawmakers such as Rep. Ro Khanna and Sen. Bernie Sanders include establishing a federal AI regulator, implementing safety and certification measures, and pausing superintelligence development. Other lawmakers, including Sens. Chris Murphy and Ted Cruz, have advocated for a regulatory framework to ensure AI research continues under proper safeguards.
The call for regulatory action has also seen engagement from political candidates. Michigan Senate candidate Abdul El-Sayed and California gubernatorial candidate Xavier Becerra emphasized the need for stronger regulation to address the existential risks posed by AI. California congressional candidate Connie Chan criticized her rival for financial ties to the tech sector, while promoting the necessity of congressional action to regulate AI companies effectively.