Shut Up or Shut Down? Amodei, Altman, Musk & the Great AI Freakout

Something unprecedented happened yesterday. Sam Altman, Elon Musk, and Dario Amodei — rivals who rarely agree on anything — all called for an AI slowdown. It came after more and more doomsday warnings from experts which seem to have finally woken Congress up to AI’s potentially existential threat. Axios even claimed that Speaker Mike Johnson is considering canceling the House recess to force members back to the Capitol to discuss the crisis. This seems to have been the week that the AI zeitgeist has finally shifted. Seven days that have changed the AI world. For That Was The Week publisher Keith Teare, it’s time to shut up or shut down. While he remains a sceptic about the autonomy of AI agents, he recognizes its potential danger. “The minute there is recursive self-learning, there’s no speedometer. It’s either on or off,” he says. “You either control it or you don’t.” In terms of AI policy, he argues, you can’t be half-pregnant. Either leave it alone or flip the off switch. Anything else fudges the issue and allows slippery characters like Sam Altman and Dario Amodei to pursue their own agenda. I don’t disagree. The real world, however, is forever half-pregnant. Investment in AI and the upcoming Anthropic IPO are the driving force of the economy. So AI’s existential threat isn’t just to “humanity.” It’s also to American prosperity, a politically existential threat (thus Trump’s opposition to any regulation). So, yes, this was the week that the AI zeitgeist shifted. The die is, indeed, cast. But it’s only the beginning rather than the end of this remarkable story. Pass the popcorn. Predicting the future might be a fool’s game, but it’s certainly going to get real interesting. Five Takeaways •       Calling BS on the Slowdown. The week’s convergence was genuinely new — a Dario Amodei essay warning that humans could lose control, Altman and Musk jumping aboard, all three major papers leading with it, Congress “waking up,” and an Axios scoop that Speaker Mike Johnson is being urged to cancel the House recess. Keith’s diagnosis is withering: it illustrates “the weakness of AI leadership,” swayed by a six-week employee who resigned, wrote an essay, and went on TV. His deeper claim: there is no such thing as AI as an independent actor, so the entire scare is misconstrued — “AI is not dangerous, period.” What the slowdown talk really represents, he argues, is the economic self-interest of incumbents in containing open-source AI — rules that prevent others from playing. Is anything new, Andrew pressed? “The heat is ramped up a notch. That’s about it.”•       No Speedometer. The episode’s logical core. If you believe recursive self-improvement can’t be controlled by humans, and that the agents it produces will be malicious, then slowing down is incoherent — “the only logical thing is not to slow it down, but to shut it down”: no Anthropic, no IPO, a ban. “The minute there is recursive self-learning, there’s no speedometer. It’s either on or off.” Scott Bessent’s speech — the week’s most prescient, in Keith’s view — made the practical version: recursion can’t be stopped, and 3,500 mostly open-source models already sit in the Hugging Face store. Stephen Witt’s NYT taxonomy (shut it down, air-crash investigations, monitoring, the kill switch — with his closing COVID-epidemiologist analogy) gets respect but fails Keith’s test: none of it solves the problem if the problem is real. In AI policy, as Andrew’s intro has it, you can’t be half-pregnant: either leave it alone or flip the off switch — anything else fudges the issue. His counter-gameplay: for AI to threaten humanity it needs biological or nuclear weapons, and that chain of events is “fantastical.” Andrew’s retort — so was hijacking planes into the World Trade Center. Keith: “9/11 is one airplane.”•       The IPO Game. Why would the leaders sound alarms about their own products? Keith’s fork: they’re naive (buying space by taking heat out of the conversation — which won’t work, since demands to stop will only grow) or manipulative. The tell arrived mid-week: Altman told Fortune that OpenAI won’t IPO this year, given the safety discussion — which Keith reads as calling Dario’s bluff. Will Anthropic pull its $2 trillion listing? “I don’t think he is. I think he’s playing a game… a high-stakes game.” Andrew’s materialist reading: with the American economy increasingly dependent on the AI sector and its impending IPOs, nothing gets shut down short of an AI 9/11 or COVID event — the story is now about the whole future of the American economy. Keith’s post-midterm forecast: the White House does nothing, and unless Democrats take both houses, “nothing changes” — markets settle, IPOs happen, AI continues. And his path to peace: most innovations become acceptable only after they deliver the bacon; people must feel AI’s benefits to discount its fears. Andrew’s verdict: the AI zeitgeist has shifted and the die is, indeed, cast — but this is the beginning rather than the end of the story. Pass the popcorn.•       Open Source Wins — China Included. Against Noah Smith’s America-is-winning scorecard, Keith reframes the race entirely: the real contest is open versus closed source — with the Chinese, ironically, in the open camp and the Americans building walled gardens. History’s lesson: “open source leaks like water into every crevice” — the cloud is almost 100% Linux. The week’s stranger revelation: reports that Kimi was routing every query to Claude — “there was actually no Kimi” — plus the training-dependency chain that keeps Chinese models about a year behind. The money agrees with American dominance either way: the FT’s moonshot-capitalism piece has US deep-tech VC at $246 billion (80% of it California) against China’s $105 billion and the UK’s 26 — with the power law concentrating it all into a handful of companies, and Andreessen Horowitz reporting that post-SpaceX-IPO venture returns now dwarf private equity. As for regulation: California’s new chatbot and teen-social-media laws are, per Keith, light and barely policed.•       The Rest of the Week. Starbucks’ billion-dollar bet on the coffeehouse as antidote to lonely digital lives — Keith’s analyst verdict: Barnes & Noble tried the sofas, and “the biggest problem is the coffee isn’t very good.” How writers actually use AI, via Every’s interviews: Kevin Roose structured The AGI Chronicles himself, then let AI do what he calls three to four years of work in one — exactly, Keith notes, the process behind his own The Human Dividend. The Edmans crossover, from Monday’s interview of the week: do smart people make dumb investments? Keith’s Wall Street history says the best investors are “working class street smart kids” who smell deals — inventors are not gamblers — though Musk is the exception, and as for Dario: “I think Dario would be good at anything,” Andrew offered. “He’s pretty good at politics.” Apple’s iPhone Duo foldable arrives with, per Ben Thompson, still no AI strategy — while Facebook ships Muse, a full agent with a virtual machine that would need 27 million physical computers at Meta scale. And farewell to Bill Draper — among the first venture capitalists, first into Skype before his son Tim, a seeker of contrarian difference “a bit like AI now.” About the Co-Host Keith Teare is the publisher of the That Was The...