A Modern Prometheus

At least since 2018, tech executives like Sundar Pichai have been comparing AI to fire, generally in the context of how monumental the impact will be. But this comparison is actually richer and deeper than just hype; it is touching on one of the crucial aspects of AI: its dual-use nature and ability to harm the wielder if used incorrectly.

Fire is a cornerstone of human civilization, and yet it has burnt down cities time and time again; one infamous example is the Great Fire of London (Wikipedia also has a whole list), and that is not even going into wildfires and the modern fires making headlines today. Fire can also be used on a much smaller, more personal scale to cook, keep warm, generate light, and more, and by that same token it can burn the wielder if used incorrectly. The damage an individual can do can range from themselves up to a much larger scale, and the same is true of AI. Used poorly, AI can impede learning, damage reputations, and cause a myriad of harms; the AI safety community will also point out that AI can empower a bad actor to do damage at a scale they could never reach alone.

And here is the crux of the issue: both AI and fire are (1) too useful to ignore, (2) too dangerous to be used without care, (3) likely to provide the most benefit when proliferated. Much as each home benefits from having its own kitchen rather than sharing a communal one, every organization and individual can yield more benefit from having their own access to AI. And even communal kitchens have their own risks: the Great Fire of London is believed to have started in a bakery. The same is true for AI: even large centralized AI systems pose risks; consolidation does not eliminate risk.

Given historical trends, these kinds of technologies offer great benefit to the economy and tend to be accepted by society, proliferation and all, and the balancing point becomes safeguards, education, and study. As a population we share both the risks and the rewards, and the better our society at an individual level understands the risks and how to use the technology, the better we can manage the risks, set policy at the organizational and societal level, and reap the benefits. Fire became safer not because we used it less, but because we studied it: firefighting, fire safety, building codes, and continual innovation. AI safety will come from education, study, and innovation leading to better policy and governance.

So today, the AI labs around the world are acting as Prometheus, in particular those that open-source their work, and the world grapples with the proliferation of a new kind of fire.