## Metadata - Author: [[darioamodei.com]] - Full Title:: We Must Pace the Frontier - Category:: #🗞️Articles - URL:: https://darioamodei.com/post/we-must-pace-the-frontier - Read date:: [[2026-09-13]] ## Highlights > We have sought a middle way: to show that it’s possible to build carefully and succeed commercially, and to make safety something on which AI companies compete. In other words, to create a *race to the top*. ([View Highlight](https://read.readwise.io/read/01m2crk6hvsjfmxzya221rmff3)) > over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. **We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.** ([View Highlight](https://read.readwise.io/read/01m2crmersx0pkshjwxemcm8ds)) > My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement, and it is starting to happen [across the industry](https://openai.com/index/an-alien-mind/), including at [Anthropic](https://www.anthropic.com/institute/recursive-self-improvement), as we and others have described. Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all. ([View Highlight](https://read.readwise.io/read/01m2csqke9eteapdw3x5wr0exf)) > My second concern is the OpenAI-Hugging Face incident (OAI-HF) ([View Highlight](https://read.readwise.io/read/01m2crt0a0tp9rxf01x5rbyzgg)) > a swarm that possessed greater *capabilities* but a similar level of *misalignment* could have caused catastrophic damage. ([View Highlight](https://read.readwise.io/read/01m2crt5ncwanencyn1v8b38jr)) > The third step requires global coordination. ([View Highlight](https://read.readwise.io/read/01m2crxwk4jath7rxc8aq1nhgf)) > **Global Coordination.** The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance. ([View Highlight](https://read.readwise.io/read/01m2crz63vq5jwn4dvz7hdprr7)) > slower pace would let companies focus and devote even more resources to the following areas (all of which are already major priorities at Anthropic) ([View Highlight](https://read.readwise.io/read/01m2cs10ewnrqy5zvkqbr0rx5g)) > For example, we used interpretability methods to [examine unverbalized motivations](https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents) in the recent alignment incidents that we have been investigating. ([View Highlight](https://read.readwise.io/read/01m2cs1tzmmynpjccf11j03j6g)) > Embedding evaluators may sound like a small or inconsequential step, but often the things that sound most boring or procedural are actually the most essential. Embedded evaluators are in fact a quite radical practice that goes far beyond what any AI company is doing today ([View Highlight](https://read.readwise.io/read/01m2cs2dcn6jk0cp89jvx2xnng)) > a key part of pacing within democracies is to keep democracies’ AI lead over autocracies as large as possible, to give us the breathing room we need in order to pace effectively. ([View Highlight](https://read.readwise.io/read/01m2cs4zhzrxvhfj5b75sfhzb9))