Read time: 5 minutes | Unlock my Claude Skills Bundle
Hey friend, Alex here.
Last Tuesday, a researcher at Anthropic quit and said the company he’d just left and OpenAI are racing to build something that could kill us all by the end of the decade. Within hours, a colleague who still works there agreed in public and put the odds above 10%. My feed did nothing else for four days.
I don’t share the fear, and I’ll tell you why below. But I also don’t think the people warning are stupid, because the idea underneath the panic is one I use every week. It’s compound interest. Small improvement, stacked on the last improvement, repeated until the curve goes vertical. While the doomers are staring at the curve where machines turn lethal, we can look at a more useful curve instead, one we can actually use at work.
Today:
What actually happened in the extinction debate this week, in order, and where I land
The Compounding Error Check, a prompt that shows how the small unchecked mistakes in your own AI workflows stack up over a year (copy-paste ready)
4 links worth keeping from the noise, two making the doomers’ case and two making mine
📰 WHAT HAPPENED THIS WEEK
It started on Tuesday evening with a resignation. Jacob Coxon, three years of pretraining research at OpenAI and then Anthropic, posted on X that he’d quit, that neither company is acting responsibly, and that they’re racing straight to self-improving superintelligence and gambling with our lives. Further down the thread came the line everyone screenshotted. The people building AI earnestly believe it could kill us all by the end of the decade, and this is not a marketing stunt. Over 170 million views so far.
A few hours later Evan Hubinger, who leads alignment research at Anthropic and still works there, quoted him. Jacob is correct, staff really do believe AI could kill all humans, and his personal odds are above 10% within the decade. He also said the risk from today’s models is low and his worry is future systems.
By Wednesday it was everywhere, from the Wall Street Journal interview to the BBC, TIME, WIRED, and Axios, with Hubinger’s 10% as the second-day headline. This was the week the warnings came from inside the building.
On Saturday, the CEOs moved. Dario Amodei published an essay called “We Must Pace the Frontier,” arguing the industry needs to slow the pace of capability gains. Within hours Elon Musk replied “Dario is right,” Sam Altman said OpenAI agrees and will accept independent evaluators, and Demis Hassabis backed the direction. Four CEOs agreed to slow down and accept outside checks. None of them repeated the 10% number. That’s a slowdown coalition, not an extinction confession.
The pushback came Monday. Jensen Huang, at the All-In Summit, said the existential-threat talk has no scientific basis. Then President Trump phoned into the same event and posted on Truth Social, declaring himself “the Hoax Buster” and calling the idea that AI will take over and destroy the world a hoax.
I’m closer to Huang’s end of the room than the extinction end, though “hoax” isn’t my word, and I don’t think Coxon and Hubinger are lying. The mechanism they fear, recursive self-improvement, is compounding. A model helps build a better model, which helps build a better one, the same arithmetic that turns a 1% weekly improvement into a doubled result in about 70 weeks. I believe the capabilities are moving fast. I just don’t believe a faster model removes the humans standing between it and the world.
The 2026 International AI Safety Report says today’s systems can’t cause humanity to lose control. To get there, a model would need to get much better at long-term planning, hiding what it’s doing, evading oversight, and resisting shutdown. Every one of those is a human checkpoint it would have to beat. We’ve had weapons that could end civilization since 1945, and they’ve sat unused because they live behind stacks of human authorization and nobody gets to fire one alone. The extinction argument has to assume those checkpoints vanish, and I don’t see why they would.
Extinction is also the best marketing this industry has ever had. It says the product is powerful enough to end the world, which makes it worth paying for, and it makes the people building it the only ones qualified to regulate it. I believe Coxon when he says it’s not a stunt. I’m saying nobody in this story is paid to shrink the fear.
The risk I actually believe in is smaller, duller, and happening now. The labs are shipping agents that log into your accounts, spend your money, and send your emails with nobody approving any step of it. Output nobody checked. A decision made by a model at 3am. A company that can’t explain to a customer what its own system did. That risk is fixable, and it sells nothing, which is why you read about it nowhere this week.
And it compounds too. Just in the other direction.
⚡ THE SUPERPOWER: The Compounding Error Check
Say you’ve got an AI step that runs 40 times a week and gets it right 97% of the time. That feels fine. Nobody reviews it because it’s usually fine. But 3% of 40 is more than one wrong run a week, and if the output of one run feeds the next, the mistake doesn’t stay put. By month three you have a stack of small errors nobody looked at, each one built on the last, and the first time you notice is when a customer asks a question you can’t answer.
Compounding doesn’t care which direction it’s pointed. A 2% error rate with no checkpoint isn’t a 2% problem. It’s a 2% problem multiplied by every run you never looked at. Extinction is the loud version of compounding. The version that actually costs you money is quiet.
The prompt below does one thing. It takes a single recurring AI task and shows you what its errors compound into, then names the one place a human check would bend the curve back down.
⚙️ THE WORKFLOW
Open a fresh Claude chat. Free tier works. Paste this and fill in the five brackets with the most honest numbers you have:
I run a recurring task with AI or automation and I want to see how its errors compound.
Task: [what it does, in one sentence]
Frequency: [how many times per week it runs]
Review: [who checks the output and how often. "Nobody" is a valid answer]
Accuracy: [your honest guess at how often a single run is fully correct, as a percentage]
Cost of one miss: [what one wrong run costs in money, time, or trust]
Do this:
1. Project the number of unchecked errors after 30 days, 90 days, and 12 months at my stated accuracy. Show the arithmetic.
2. Show how the picture changes if my true accuracy is 5 points lower than my guess.
3. Tell me which errors in this task compound (one mistake feeds the next run) and which stay flat (each mistake is independent). Treat only the first kind as urgent.
4. Name the single point in this task where one human authorization step would remove the most compounded error, and estimate what that step costs in minutes per week.
5. Give me one signal I can check weekly that would tell me the curve is bending the wrong way.
Keep it under 300 words. No reassurance.Then do three things with the answer.
Read step 3 first. The split between compounding errors and flat errors is the whole point. Most tasks have both, and only one kind deserves a checkpoint.
Put the step-4 checkpoint on your calendar this week. It’s usually a five-minute review at one specific handoff, which is the same thing as a launch officer turning a key.
Run it again with the accuracy number 5 points lower. If the result changes your mind, your original guess was doing a lot of work.
Fair warning on the arithmetic. The model is multiplying your guess, so the year-end number is only as honest as the accuracy figure you typed in, and if you’ve never measured it, the projection is a story about your assumptions rather than about your workflow. Use it to find the checkpoint, not to quote the number in a meeting.
If you want this as a permanent tool rather than a prompt you re-paste, I turned it into a Claude skill. The install guide is at the bottom of this email.
💬 PROMPT OF THE DAY
Read this thread and tell me the one question that, if answered, would make the rest of the thread unnecessary. Then draft the two-sentence message that asks it.
Why it works: long threads usually orbit one unasked question, and naming it collapses the loop instead of adding another reply to it.
Best on: any model.
📚 USEFUL RESOURCES
🔥 Amodei’s “We Must Pace the Frontier” (the essay the CEOs rallied behind) → the strongest version of the slow-down case, from the person with the most to lose by making it (free: here)
🗞️ Axios: how AI could kill us all (plain-English explainer) → the fastest read on p(doom), who says what number, and the two pathways the doomers actually fear (free: https://www.axios.com/2026/09/09/ai-doom-pdoom-kill-all-humans-anthropic)
📋 International AI Safety Report 2026 (the multi-country expert assessment) → the calmest document in this whole debate, and the source for what today’s models can and can’t do (free: https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026)
🔬 MIT Tech Review on recursive self-improvement (Princeton study, August) → AI agents can do the engineering but not the open-ended research, which is the exact step the doom curve depends on (free with registration: https://www.technologyreview.com/2026/08/18/1142188/ai-recursive-self-improvement/)
What should I put through the Error Check next: an email agent that sends without review, or a weekly report nobody reads before it goes to the client? Hit reply, one word is enough.
Know someone who spent the week doomscrolling instead of checking their own automations? Forward them this.
And as always, remember: LLMs don’t think, you do.
⚡ Alex Prompter
P.S. Today’s Error Check is one prompt. My Claude Skills Bundle turns Claude into 20+ specialists for marketing and business. Unlock it in one click: https://linktr.ee/alex_prompter
AI Superpowers, part of the Alex Prompter network. One new AI superpower every Tuesday, under 5 minutes.