AI Tools for Existential Security — EA Forum

This is a linkpost for https://www.forethought.org/research/ai-tools-for-existential-security

Rapid AI progress is the greatest driver of existential risk in the world today. But — if handled correctly — it could also empower humanity to face these challenges.

Executive summary

1. Some AI applications will be powerful tools for navigating existential risks

Three clusters of applications are especially promising:

• Epistemic applications  to help us anticipate and plan for emerging challenges
  • e.g. high-quality AI assistants could prevent catastrophic decisions by helping us make sense of rapidly evolving situations
• Coordination-enabling applications  to help diverse groups work together towards shared goals
  • e.g. automated negotiation could help labs and nations to find and commit to mutually desirable alternatives to racing
• Risk-targeted applications  to address specific challenges
  • e.g. automating alignment research could make the difference between “It’s functionally impossible to bring alignment up to the requisite standard in time” and “this is just an issue of devoting enough compute to it”

2. We can accelerate these tools instead of waiting for them to emerge

• While broad AI progress will drive the development of many applications, we have some flexibility in the timing of specific applications — and even small speed-ups could be crucial (e.g. by switching the order of risk-generating capabilities and risk-reducing ones)
• We could use avariety of strategies to accelerate beneficial applications:
  • Data pipelines & scaffolding: by curating datasets or scaffolding for key capabilities, or laying the groundwork to automate this, we could enable those capabilities as soon as underlying AI progress supports them
  • Complementary tech & removing other barriers to adoption: by building out the UI or other complementary technology, and ensuring that people are eager to use the applications, we could enable the applications to see use as soon as the underlying capabilities are there, rather than accept delays to adoption
  • Shaping compute allocation: by building support among key decision-makers who might allocate compute, we could ensure that crucial applications are among the earliest to see large amounts of automated research
• Accelerating beneficial applications can often be done unilaterally (in contrast to delaying dangerous capabilities, which may need consensus)

Implications

These opportunities seem undervalued in existential risk work. We think a lot more people should work on this — and the broader “differential AI development” space. Our recommendations:

1. Shift towards accelerating important AI tools
   • e.g. curate datasets for automating alignment research; or build AI forecasting systems
2. Plan for a world with abundant cognition
   • Some new approaches will come online, and some current work may be obsoleted
   • e.g. it could make sense to build tools that process rich information to provide bespoke infectious disease exposure advice in contact tracing apps
3. Get ready to help with automation
   • e.g. build relevant expertise, or work towards institutional buy-in

Some AI applications will help navigate existential risks

Epistemic applications

People are more likely to handle novel challenges well if they can see them coming clearly, and have good ideas about what could be done about them.