John Herrman @jwherrman.bsky.social

10 days ago

Altman says all sorts of things, all the time. But the hard pivot away from AGI/superintelligence rhetoric by Eric Schmidt is genuinely jarring, sort of funny, and I think telling nymag.com/intelligence...

In the last week, two prominent voices in Al attempted to make some adjustments to the AI discourse. Asked if
"investors as a whole are overexcited about Al," OpenAI's Sam Altman answered "yes" to a group of gathered reporters, according to the Verge. Then he used that word.
"When bubbles happen, smart people get overexcited about a kernel of truth," he said. "If you look at most of the bubbles in history, like the tech bubble, there was a real thing. Tech was really important. The internet was a really big deal.
People got overexcited."
It would be a mistake to read this as a major departure from Altman's wide-ranging and often inconsistent public musings about Al: His intention here, after a rocky rollout for his company's latest model, GPT-5, seems to be to situate OpenAI outside of a possible bubble, implying that his company has more in common with dot-com Amazon than with Pets.com or Worldcom. Still, it's significant that Altman broached the subject, and there are signs that a generally bullish Wall Street has taken notice: In a story about so-called "disaster puts," Bloomberg reports that
"options traders are increasingly nervous about a plunge in technology stocks in the coming weeks and are grabbing insurance to protect themselves from a wipeout."

In another notable shift, former Google CEO Eric Schmidt, who has in recent years become an influential and provocatively severe figure in the Al world, co-authored a piece for the New York Times with tech analyst Selina Xu arguing that Silicon Valley "needs to stop obsessing over superhuman Al." Just a few months ago, in another op-ed, Schmidt argued that "even without a consensus about a precise definition, the contours of an AGI future are beginning to take shape," suggesting that "artificial general intelligence" could "usher in a new Renaissance." This week, he and Xu urged their peers to rethink such messaging and suggested that narratives like this actually gave them
"pause":
It is uncertain how soon artificial general intelligence can be achieved. We worry that Silicon Valley has grown so enamored with accomplishing this goal that it's alienating the general public and, worse, bypassing crucial opportunities to use the technology that already exists. In being solely fixated on this objective, our nation risks falling behind China, which is far less concerned with creating A.I. powerful enough to surpass humans and much more focused on using the technology we have now.

John Herrman @jwherrman.bsky.social

10 days ago

I'm insufferably, interminably on-record arguing that AGI discourse is alienating, historically untethered, and misleading (nymag.com/intelligence...). But now of its biggest boosters — and, as @himself.bsky.social points out, an influential figure in AI and foreign policy — says: ok, yeah

As an insider effort to regulate the frequently wild messaging coming from the Al industry, this is surprising rhetoric. It is consistent with (and actually cites) critical work arguing that while recent developments in Al represent a major technology with great disruptive potential, current models might not be on the cusp of rendering the entire economy obsolete and/or wiping out or subjugating humanity once machines have established themselves as a superior species. The argument that stories about AGI are alienating, distracting, and disempowering - no objection here! - is likewise sort of wild to hear coming from Schmidt, who just this year warned that Al models are beginning "recursive self-improvement" and are already
"learning how to plan, and they don't have to listen to us anymore."
But political scientist and Al theorist Henry Farrell makes a case for the deeper significance of such a pivot from Schmidt, who last year co-authored a book with Henry Kissinger about Al, and who Farrell has argued could be
"the most influential American foreign policy thinker of the early 21st century. In short, Schmidt was instrumental in forming the "mind meld" between Silicon Valley and national-security policymakers, which held that self-improving AI, and AGI, represented a fast-approaching
"inflection point" in a race between the U.S. and China, the winner of which would gain a "technological advantage that would secure its long-term dominance." But, he writes, "if the AGI bet is a bad one, then much of the rationale for this consensus falls apart. And that is the conclusion that Eric Schmidt seems to be coming to."

Replies

Tim Feinstein @timnfeinstein.bsky.social
8 days ago

I don’t see how people can talk about LLMs and AGI even in the same paragraph. There’s no intelligence in a LLM at all. It’s just an average of results associated with the prompt words in a training set, nudged so it doesn’t look too deterministic. AGI would need an entirely different architecture.

0