The current AI strategic landscape: one bear's perspective

1. There are signs that AI Ethicists are increasingly aligning with AI safety concerns. This creates the possibility of a united front between AI ethicists and safetyists.

   1. M. Mitchell, who co-authored the Stochastic Parrots paper, recently wrote a paper on the dangers of autonomous AI. Her paper even mentions the paperclip maximizer hypothetical. She seems increasingly sympathetic to safetyism although she is still worried about the diversion of attention from what she sees as more immediate threats.

   2. Emil Torres is adjacent to the AI Ethics community. Emil has been fiercely critical of the AI safety community. About a year ago they pointed towards something of a consilience between the positions. Regardless of whether or not you think the existential risk is real, the CEOS of the big AI companies do think it is real and are conducting experiments they themselves think might kill all of us. This alone proves their wild irresponsibility and gives a basis to restrict such experiments.

   3. Popular science communicators, such as Gary Marcus and Sabine Hossenfelder who have been quite skeptical of the potential of the current wave of AI have nonetheless made it clear that they consider existential risk a serious concern. Hossenfelder in particular has been quite clear on this.

   4. Civil Society organizations like SAG-AFTRA, an actors union, primarily concerned with specific threats have cooperated with organizations concerned with existential risks- e.g. on the California AI safety bill. As AI more and more directly harms concrete groups with associated lobby organizations, such cooperation will become increasingly common as a matter of course.

2. On the other side, AI safetyists are increasingly concerned about social problems of the sort they traditionally ignored or downplayed in favor of existential risks. There is much more recognition of a broader array of threats beyond the direct elimination of the human species.

   1. A recent paper by Kulveit et al. focused on the idea of humans slowly losing control of society due to a combination of capitalist incentives and the increasing deployment of AI.

   2. AI safetyists are increasingly joining in discussions about technological unemployment. There seems to be greater recognition that even though in principle automating all work would open the possibility of a just post-scarcity society, this will not happen automatically. Apolitical solutions like universal basic income (not even a particularly good solution IMO) are not guaranteed. Even though automating everything creates the possibility of a Pareto improvement, in practice, it might transform most people into an impoverished and powerless surplus population unless political and social safeguards exist. It seems important to put such safeguards in place while we still have the bargaining position to demand them. As Holly Elmore of Pause AI puts it:

      

   3. Some organizations such as Stop AI are explicitly ecumenical- focusing on concerns as broad as energy usage, unemployment, loss of control, etc. [Guido Reichstadter of Stop AI helpfully commented on an early draft of this piece].

   4. The work of e.g. Garrison Lovely is extremely refreshing in bridging both the safety and the ethics traditions with a great deal of careful research and scrutiny of the corporate players.

   5. This paragraph from Scott Alexander expresses the consilience well: I don’t want to gloss this as “socialists finally admit we were right all along”. I think the change has been bi-directional. Back in 2010, when we had no idea what AI would look like, the rationalists and EAs focused on the only risk big enough to see from such a distance: runaway unaligned superintelligence. Now that we know more specifics, “smaller” existential risks have also come into focus, like AI-fueled bioterrorism, AI-fueled great power conflict, and - yes - AI-fueled inequality. At some point, without either side entirely abandoning their position, the very-near-term-risk people and the very-long-term-risk people have started to meet in the middle.

3. Public polling suggests concern about multiple different threats from AI and support for more regulation. E.g.:

   1. More Americans than not are concerned about an AI doomsday scenario (46% v 40%) according to a Yougov survey.

   2. A Forbes Advisor survey indicated that 30% of the public is concerned about losing their jobs to AI. A YouGov poll indicated that nearly half of employed Americans believe AI advances will reduce the number of jobs available in their industry.

   3. However, at the moment this is largely soft and passive support for the movement to control AI, and will need to be mobilized. There is a big difference between expressing concern to a pollster when explicitly asked, and AI safety forming a major component of your daily concerns. People say all sorts of weird stuff to pollsters!

4. But that’s where the good news ends. Governments and corporations are increasingly in favor of AI acceleration. From the corporations, we’ve seen funding cuts for AI safety, opposition to any serious attempt to regulate them etc- though anthropic has been notably less bad than the others.

5. What about the government?

   1. Trump’s views on AI safety are no secret. His first act was to abolish Biden’s executive order on AI safety. He has indicated that America will not sign any AI accords coming out of Paris unless discussion of existential risk is excluded. Vance has echoed with similar sentiments.

   2. If you’re hoping to find opposition to Trump on AI in other elements of the US government you’ll likely be disappointed. The national security establishment- or if you prefer a more dramatic term “the deep state” also appears dedicated to acceleration.

      1. There are a few OpenAI board members who are clearly part of The Establishment with a capital E. These include former Secretary of Treasury Larry Summers and Retired four-star general and former head of the NSA Paul Nakasone. Larry Summers, although he has expressed some concern about safety- essentially conceives of the US as in a race.

      2. I think elements of the elite did see Deepseek wake-up call- a call to international competition.

      3. Think about it in terms of the primary purpose of the national security establishment. The national security establishment is a machine for geopolitical competition. It wants to compete with China and it will be very hard to swerve it off this path

   3. The Paris AI Action Summit- formerly called the AI safety summit- was renamed by its organizers. This was not an accident. The results of the summit suggest that we cannot rely on Europe as any “third force” in relation to AI. The story coming from many people, e.g. Seán Ó hÉigeartaigh of the Cambridge AI: Futures and Responsibility Institute is that France has effectively hijacked the conference series and gutted any talk of existential risk and even substantive discussion of AI bias and ethics. This is certainly reflected in the statement coming from the conference which was, even by the standards of international instruments, a meaningless string of platitudes.

      1. Quite possibly this result was driven by the French AI company Mistral’s closeness to Emmanuel Macron.

      2. It is worth emphasizing again that the documents that came out of this conference were also terrible on AI ethics and bias. ANYTHING that might act as a substantial barrier to profit accumulation was rejected, not merely existential risk.

      3. My view is that 80% of the world can be explained by vulgar Marxism- big money rules everything, especially politics. I’m chalking this one up as an explanatory success.

6. So we have the following match: the power of the people and the experts versus corporations and government- mirroring the struggle over climate change. I’m not surprised, but I am alarmed. I’ve spent a lifetime fighting against the joint power of governments and corporations on the side of experts and people- e.g. in campaigns for greater economic equality and less emissions. The most optimistic spin I can on that matchup is that it is AT BEST slow going. We likely do not have time for slow going.

7. 1. We need to keep trying to grab influence in the companies and in the government. “That’s their territory- our power lies with the people” WILL NOT DO. If we had enough time we could pursue a pure outsider strategy- and I concede it might have some advantages. We do not have enough time.

   2. We need to further unite the community: tech workers, academics, intelligentsia, etc. concerned about AI around into a common line. A common line does not require agreement on everything- but we need to coordinate around the essentials. Exactly what the platform of agreement is needs to be worked out by the movement- I have a few thoughts of my own, but that’s an essay in its own right.

   3. And we need to change the passive support of the public to active support. In pursuing the public, we want to do our best to draw people in from all demographics and all political views.

8. How to avoid polarisation? If this becomes a left v right issue we’re in a lot of trouble, yet almost inevitably it is becoming that. I don’t have a complete solution, but here are some angles to interest right-wingers re: AI safety:

   1. The status of the tech CEOs as cultural elites. The fallings out these CEOs will have with Trump (one can even imagine a narrative emerging that they are the wicked advisors to the good czar (Trump)).

   2. Economic issues- being rightwing doesn’t make losing your job any easier. Consider Luigi Mangione and Bernie Sanders- both motivated by strongly leftwing concerns but with surprisingly high esteem among some elements of the right. The fear of job loss has the power to short-circuit culture war narratives to a degree. It is notable that Vance recently felt the need to deny that AI will cause job loss.

   3. Parental concerns. Although it hasn’t exactly worked in debates over global warming “This thing might kill your kids” is worth emphasizing.

   4. National security narratives etc. See this interesting mockup

      https://www.trumpmanhattanproject.com/

   5. Dan Hendrycks may even be trying out the argument that AI is too woke to be trusted.

9. Overall I think everyone should work on what they believe in and do so honestly. I want a movement that engages everyone, but I have zero interest in pretending to be conservative, and I have zero interest in forcing conservatives who want to get involved in the AI safety movement to pretend to be leftists. Supportive non-coordination without substitution or homogeneity seems ideal.

10. I think any chance of mobilizing the public probably relies on a breaking event- something that tears apart the existing political constellations. Overall, I think job loss due to AI is the most likely such event. The coming of agents and their inevitable misuse might also cause something of a “WTF” breaking event, even before job loss. The critical event need not be AI-related. Major political realignment can come in a lot of forms- economic crisis, scandal, or sometimes from nowhere particularly obvious- such realignments create opportunities for outsiders. It is important to have a strong activist, lobbyist, and intellectual infrastructure to take advantage of a break, whenever and in whatever form it may come. There is no sense in waiting for the break before starting action, and who knows? Action may stir things up.

11. I wish I could say more about the international situation here, vis a vis China, but I lack the expertise. My strong gestalt impression is that attempts at Chip-based containment are not working. I fear the response to this will be a race since a negogiated de-escalation seems unlikely. Still, history is full of suprises.

12. I worry the movement has become very focused on pausing AGI research. Partly this is based on a strong view that the disaster story goes like this- *We build AGI—> It goes foom—> It kills us all unless we have a mathematically rigorous and provable approach to alignment. I think many different scenarios are possible- it’s possible for example that we live close to an alignment by default universe, and it just needs a little nudge. I agree pausing AI should be our preferred outcome, and finding a provable approach to alignment would be optimal, but we should take what we can get and fight to get as much as we can grab at each juncture. Whether that is a:

    1. A pause

    2. A pause in some but not all sorts of research

    3. Regulations that attempt a controlled or more controlled approach to AGI development.

    4. Funding for safety research.

    5. Treaties

    6. International or national monitoring bodies.

Thanks for reading. I spend many hours per week on this blog and make it available for free. Your paid subscription would be greatly appreciated both financially and emotionally, especially right now because I just left my old job only to find out that my start date at my new job has been delayed.