Life 3.0: Being Human in the Age of Artificial Intelligence

Tegmark begins the chapter with a guiding question: whether AI will eventually dominate the world, supplanting human beings as the preeminent intelligent force on Earth.

To explore this question, Tegmark revisits and revises the story he presented in the prologue, the story of Team Omega and the super AI, Prometheus. This time, the story takes a more sinister spin. If there is an “intelligence explosion,” and an AI like Prometheus becomes far more intelligent than any human population, it’s possible that a small group of humans who control the AI could take over the world. In this darker version of the story, the team uses Prometheus to establish a police state in which they have total control. Of course, Prometheus could also break free from his metaphorical chains and take control.

Attempting to describe why Prometheus would try to break free, Tegmark invents this scenario:

Suppose that a mysterious disease has killed everybody on Earth above age five except you, and that a group of kindergartners has locked you into a prison cell and tasked you with the goal of helping humanity flourish. What will you do? If you try to explain to them what to do, you’ll probably find this process frustratingly inefficient (139).