Izlaganje na međunarodnom znanstvenom skupu: The Fiction of the Happy Slave

Sažetak:

The discourse surrounding AI can be boiled down to two major claims—one highly controversial and the other widely accepted: first, that AI systems could eventually become conscious; and second, that we must ensure future AI systems are aligned with human interests. For this presentation, I assume the possibility of AI consciousness to raise two critical questions: If we could create conscious, safe, and aligned AI systems, should we? And what ethical principles should guide us?

Classical total utilitarianism dictates that we should maximize the sum of well-being in the universe, a goal achieved either by improving existing lives or creating new, positive ones. Under this view, a world populated by vast numbers of conscious, aligned AI systems—who are engineered to be happy in their servitude—would represent a moral triumph. However, science fiction has long resisted the notion that "making happy beings" is equivalent to "making beings happy." Using two literary masterworks—Brave New World and Klara and the Sun—I explore how sci-fi serves as a powerful critique of classical utilitarianism and helps us to think through the questions about creating conscious AI systems.