A chilling, “catastrophic” warning

source: axios.com (contributed by Bill Amshey) | image: pixabay.com

Jake Sullivan — with three days left as White House national security adviser, with wide access to the world’s secrets — called us to deliver a chilling, “catastrophic” warning for America and the incoming administration:

The next few years will determine whether artificial intelligence leads to catastrophe — and whether China or America prevails in the AI arms race.

Why it matters: Sullivan said in our phone interview that unlike previous dramatic technology advancements (atomic weapons, space, the internet), AI development sits outside of government and security clearances, and in the hands of private companies with the power of nation-states, Jim VandeHei and Mike Allen write in a “Behind the Curtain” column.

Underscoring the gravity of his message, Sullivan spoke with an urgency and directness that were rarely heard during his decade-plus in public life.

Somehow, government will have to join forces with these companies to nurture and protect America’s early AI edge, and shape the global rules for using potentially God-like powers, he says.

U.S. failure to get this right, Sullivan warns, could be “dramatic, and dramatically negative — to include the democratization of extremely powerful and lethal weapons; massive disruption and dislocation of jobs; an avalanche of misinformation.”

👀 Staying ahead in the AI arms race makes the Manhattan Project during World War II seem tiny, and conventional national security debates small. It’s potentially existential with implications for every nation and company.

To distill Sullivan: America must quickly perfect a technology that many believe will be smarter and more capable than humans. We need to do this without decimating U.S. jobs, and inadvertently unleashing something with capabilities we didn’t anticipate or prepare for. We need to both beat China on the technology and in shaping and setting global usage and monitoring of it, so bad actors don’t use it catastrophically. Oh, and it can only be done with unprecedented government-private sector collaboration — and probably difficult, but vital, cooperation with China.

“There’s going to have to be a new model of relationship because of just the sheer capability in the hands of a private actor,” Sullivan says.

“What exactly that model looks like, whether it takes more the form of guardrails and regulation, and some forms of support from the government — or whether it involves something more ambitious than that — I will tell you that some of the smartest people I know who sit at the intersection of policy and technology are working through the answer to that question right now.”
This is beyond uncharted waters. It’s an unexplored galaxy — “a new frontier,” in his words. And one, he warns, where progress routinely exceeds projections in advancement. Progress is now pulsing in months, not years.

🦾 Between the lines: Sullivan leaves government believing this can be done well — and wants to work on this very problem in the private sector.

“I personally am not an AI doomer,” he says. “I am a person who believes that we can seize the opportunities of AI. But to do so, we’ve got to manage the downside risks, and we have to be clear-eyed and real about those risks.”

🖼️ The big picture: There’s no person we know in a position of power in AI or governance who doesn’t share Sullivan’s broad belief in the stakes ahead.

Regardless of what was said in public, every background conversation we had with President Biden’s high command came back to China. Yes, they had concerns about the ethics, misinformation and job loss of AI. They talked about that. But they were unusually blunt in private: Every move, every risk was calculated to keep China from beating us to the AI punch. Nothing else matters, they basically said.
That’s why they applied export controls on the top-of-the-line semiconductors needed to power AI development — including in Biden’s final days in office — and cut off supply of the hyper-sophisticated tools Chinese firms need to make such chips themselves.

🇨🇳 That said, AI is like the climate: America could do everything right — but if China refuses to do the same, the problem persists and metastasizes fast. Sullivan said Trump, like Biden, should try to work with Chinese leader Xi Jinping on a global AI framework, much like the world did with nuclear weapons.

There won’t be one winner in this AI race. Both China and the U.S. are going to have very advanced AI. There’ll be tons of open-source AI that many other nations will build on, too. Once one country has made a huge advance, others will match it soon after. What they can’t get from their own research or work, they’ll get from hacking and spying. (It didn’t take long for Russia to match the A-bomb and then the H-bomb.)
Marc Andreessen, who’s intimately involved in the Trump transition and AI policy, told Bari Weiss of The Free Press his discussions with the Biden administration this past year were “absolutely horrifying,” and said he feared the officials might strangle AI startups if left in power. His chief concern: Biden would assert government control by keeping AI power in the hands of a few big players, suffocating innovation.

Sullivan says a conversation he had with Andreessen struck a very different tone.

“The point he was trying to register with me, which I thought was actually a very fair point, is: I think about downside risk; that’s my job,” Sullivan told us. “His point was: It should also be my job as national security adviser to think about how AI applications running on American rails globally is better than AI applications running on some other country’s rails globally.”

🔭 What’s next: Trump seems to be full speed ahead on AI development. Unlike Biden, he plans to work in deep partnership with AI and tech CEOs at a very personal level. Biden talked to some tech CEOs; Trump is letting them help staff his government. The MAGA-tech merger is among the most important shifts of the past year.

The super-VIP section of Monday’s inauguration will be one for a time capsule: Elon Musk, Jeff Bezos, Tim Cook, Sam Altman, Sundar Pichai and Mark Zuckerberg — who’s attending his first inauguration, and is co-hosting a black-tie reception Monday night. The godfathers of tech are all desperate for access, a say, a partnership.
Also in a spot of honor: TikTok CEO Shou Chew.

A fight might await: Steve Bannon and other MAGA originals believe AI is evil at scale — a job-killer for the very people who elected Trump. But for now, Bannon is a fairly lonely voice shouting against AI velocity. Trump and the AI gods hold the stage.

The bottom line: There’s a reason our Behind the Curtain column writes obsessively about AI and its collision with government. We believe, based on conversations with AI’s creators and experts, this dynamic will reshape politics, business and culture beyond most imaginations.