Intelligence Explosion

Bibliography

This bibliography lists more than 250 books, papers, and book chapters relevant to intelligence explosion. Most entries are available online; links are provided.

Core Readings and works that discuss multiple topics are listed multiple times (under different headings). Core Readings are listed alphabetically. All Readings are listed chronologically, for easy location of older and newer works. (Because this is a web document, a visitor may use the 'Find' function to quickly locate all documents by a particular author.)

Contents:

Core Readings

Chalmers (2010). The Singularity: A Philosophical Analysis. Journal of Consciousness Studies, 17: 7-65.

Chalmers formalizes I.J. Good's argument for the intelligence explosion and examines each premise in turn. He discusses possible defeaters and considers the likely timing of the intelligence explosion. He also examines the consequences of the intelligence explosion, and how we might constrain a self-improving AI. The final sections of the paper consider the implications of an intelligence explosion for our concepts of consciousness and personal identity.

Goertzel & Pennachin, eds. (2010). Artificial General Intelligence. Springer.

This edited volume begins with a survey of different approaches to artificial general intelligence, and contains contributions from a variety of AGI researchers who summarize their views on how AGI could be achieved.

Good (1965). Speculations concerning the first ultraintelligent machine. Advanced in Computers, 6: 31-88.

Good's paper is the source of the intelligence explosion hypothesis: "Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an intelligence explosion, and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make."

Loosemore & Goertzel (2011). Why an intelligence explosion is probable. H+ Magazine, March 7, 2011.

The authors raise several common objections to the intelligence explosion hypothesis and offer responses to each one in turn.

Sandberg & Bostrom (2008). Whole Brain Emulation: A Roadmap, Technical Report #2008-3. Future of Humanity Institute, Oxford University.

The authors point out that while WBE presents "a formidable engineering and research problem," it seems to be achievable by extrapolations of current technology, unlike other transformative technologies like machine superintelligence, for which we have no metric by which to tell how far we are from the goal. As they see it, the basic idea of WBE is "to take a particular brain, scan its structure in detail, and construct a software model of it that is so faithful to the original that, when run on appropriate hardware, it will behave in essentially the same way as the original brain." They go on to examine different levels of success criteria, challenges, and assumptions of WBE, and present a roadmap for how to get to WBE.

Muehlhauser (2011). Singularity FAQ.

A list of frequently asked questions about the intelligence explosion Singularity, with answers from the Singularity Institute. A quick and handy overview of the major concepts and problems involved.

Vinge (1993). The coming technological singularity: How to survive in the post-human era. Whole Earth Review, winter 1993. New Whole Earth.

Vinge argues that machine superintelligence will arise soon by one of many possible paths, one of which is Good's 'intelligence explosion.' He imagines the benefits that superintelligence coudld bring, as well as the risks of human extinction or slavery. This is the article that seems to have begun wide discussion of the Singularity on the internet, which eventually led to discussions in academia.

Yudkowsky (2008). Artificial Intelligence as Positive and Negative Factor in Global Risk. In Bostrom & Cirkovic, M. (eds.), Global Catastrophic Risks. Oxford University Press.

Yudkowsky explains the risks of machine superintelligence resulting from an intelligence explosion, warning against anthropomorphic bias. He stresses that most possible minds designs for superintelligence will tend toward human extinction, and examines several failure scenarios for the goal of designing 'Friendly AI.' He also examines several strategies for dealing with the threat, and reminds us that despite its risks, superintelligence may also be the only way to mitigate other existential risks.

All Readings (by topic, chronological)

Intelligence Explosion

Good (1965). Speculations concerning the first ultraintelligent machine. Advanced in Computers, 6: 31-88.

Moravec (1989). Mind Children. Harvard University Press.

Vinge (1993). The coming technological singularity: How to survive in the post-human era. Whole Earth Review, winter 1993. New Whole Earth.

Moravec (1999). Robot: Mere Machine to Transcendent Mind. Oxford University Press.

Hibbard (2001). Super-intelligent machines. Computer Graphics, 35(1): 11-13.

Yudkowsky (2001). Staring in to the singularity.

Hibbard (2002). Super-Intelligent Machines. Springer.

Omohundro (2007). The Nature of Self-Improving Artificial Intelligence. Self-Aware Systems.

Yudkowsky (2007). Three major singularity schools.

Legg (2008). Machine super-intelligence. PhD Thesis. IDSIA.

Mahoney (2008). A model for recursively self improving programs.

Omohundro (2008). The Basic AI Drives. Self-Aware Systems.

Yudkowsky (2008). Artificial Intelligence as Positive and Negative Factor in Global Risk. In Bostrom & Cirkovic, M. (eds.), Global Catastrophic Risks. Oxford University Press.

Sandberg (2009). An overview of models of technological singularity.

Smart (2009). Evo Devo Universe? A Framework for Speculations on Cosmic Culture. In Dick & Lupisella (eds.), Cosmos & Culture. NASA Press.

Chalmers (2010). The Singularity: A Philosophical Analysis. Journal of Consciousness Studies, 17: 7-65.

Loosemore & Goertzel (2011). Why an intelligence explosion is probable. H+ Magazine, March 7, 2011.

Muehlhauser (2011). Singularity FAQ.

Technological Forecasting

Solomonoff (1985). The time scale of artificial intelligence: Reflections on social effects. North-Holland Human Systems Management, 5: 149-153.

Bostrom (1998). How long before superintelligence? International Journal of Future Studies, 2.

Moravec (1998). When will computer hardware match the human brain? Journal of Transhumanism, 1.

Kurzweil (1999). The Age of Spiritual Machines: When computers exceed human intelligence. Viking.

Yudkowsky (2001). Staring into the singularity.

Kurzweil (2005). The Singularity is Near. Viking.

Goertzel (2007). Human-level artificial general intelligence and the possibility of a technological singularity. Viking.

Heylighen (2007). Accelerating socio-technological evolution: from ephemeralization and stigmergy to the global brain. In Modelski, Devezas, & Thompson (eds.), Globalization as an Evolutionary Process: Modeling Global Change. Routledge.

Shulman & Sandberg (2010). Implications of a software-limited singularity.

Baum, Goertzel, & Goertzel (2011). How long until human-level AI? Results from an expert assessment. Technological Forecasting and Social Change.

Risks of Intelligence Explosion

Joy (2000). Why the future doesnt need us. Wired 8.04.

Yudkowsky (2000). What is Friendly AI? Singularity Institute.

Hanson (2001). Economic growth given machine intelligence.

Yudkowsky (2001). Staring into the singularity.

Yudkowsky (2001). Creating Friendly AI. Singularity Institute.

Goertzel (2002). Thoughts on AI morality. Dynamical Psychology.

Bostrom (2003). Ethical issues in advanced artificial intelligence. In Smit, Lasker & Wallach (eds.), Cognitive, emotive and ethical aspects of decision making in humans and in arti?cial intelligence, vol II. IIAS, Windsor.

Goertzel (2004). The All-Seeing (A)I. Dynamical Psychology.

Goertzel (2004). Encouraging a positive transcension. Dynamical Psychology.

Yudkowsky (2004). Coherent Extrapolated Volition. Singularity Institute.

de Garis (2005). The Artilect War: Cosmists Vs. Terrans: A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines. ETC Publications.

Hibbard (2005). The ethics and politics of super-intelligent machines.

Posner (2005). Catastrophe: Risk and Response. Oxford University Press.

Weiler (2005). Technological Progress in Different Cultures and Periods: Historical Evolution Projected into the Future. In Wilderer, Schroeder, & Kopp (eds.), Global Sustainability: The Impact of Local Cultures. Wiley.

Armstrong (2007). Chaining God: a qualitative approach to AI, trust and moral systems.

Bugaj & Goertzel (2007). Five ethical imperatives and their implications for human-AGI interaction. Dynamical Psychology.

Hall (2007). Ethics for artificial intellects. In Allhoff, Lin, Moor, & Weckert (eds.), Nanoethics: The ethical and social implications of nanotechnology. Wiley-Interscience.

Hall (2007). Self-improving AI: an analysis. Minds and Machines, 17: 249-259.

Berglas (2009). Artificial intelligence will kill our grandchildren.

Freeman (2009). Using compassion and respect to motivate an artificial intelligence.

Fox & Shulman (2010). Superintelligence does not imply benevolence.

Goertzel (2010a). Coherent aggravated volition: a method for deriving goal system content for advanced, beneficial AGIs.

Goertzel (2010b). GOLEM: Toward an AGI meta-architecture enabling both goal preservation and radical self-improvement.

Hall (2007). Beyond AI: Creating the Conscience of the Machine. Prometheus Books.

Hall (2008). Engineering utopia.

Shulman (2009). Arms control and intelligence explosions.

Shulman, Johnsson, & Tarleton (2009). Machine ethics and superintelligence.

Waser (2008). A safe ethical system for intelligent machines.

Kaas, Rayhawk, Salamon, & Salamon (2010). Economic implications of software minds.

Shulman (2010). Basic AI drives and catastrophic risks.

Sotala (2010). From mostly harmless to civilization-threatening: pathways to dangerous artificial intelligences.

Tarleton (2010). Coherent extrapolated volition: a meta-level approach to machine ethics.

Waser (2010). Designing a safe motivational system for intelligent machines.

Bostrom & Yudkowsky (2011). The ethics of artificial intelligence. In Ramsey & Frankish (eds.), Cambridge Handbook of Artificial Intelligence. Cambridge University Press.

Hall (2011). Ethics for self-improving machines. In Anderson & Anderson (eds.), Machine Ethics (pp. 512-523). Cambridge University Press.

Yudkowsky (2011). Complex value systems in Friendly AI. In Schmidhuber, Thorisson, & Looks (eds.), Artificial General Intelligence, 4th Annual Conference, AGI 2011 (pp. 388-393). Springer.

Armstrong, Sandberg, & Bostrom (2011). Thinking inside the box: Using and controlling an Oracle AI.

Artificial General Intelligence Programming

Goertzel, Pennachin, Senna, Maia, & Lamacie (2004). Novamente: An integrative architecture for artificial general intelligence.

Schmidhuber (2005). Completely self-referential optimal reinforcement learners. ICANN'05 Proceedings of the 15th international conference on Artificial neural networks: formal models and their applications, Volume II.

Franklin (2007). A foundational architecture for artificial general intelligence. Proceedings of the 2007 Conference on Advances in Artificial General Intelligence.

Goertzel & Pennachin, eds. (2007). Artificial General Intelligence. Springer. Containing:

Legg & Hutter (2007). A collection of definitions of intelligence.

Ekbia (2008). Artificial Dreams: The Quest for Non-Biological Intelligence. Cambridge University Press.

Wang, Goertzel, & Franklin, eds. (2008). Artificial General Intelligence 2008: Proceedings of the First AGI Conference. IOS Press. Containing:

Mahoney (2008). A proposed design for distributed artificial general intelligence.

Goertzel, Hitzler, & Hutter (2009). Proceedings of the Second Conference on Artificial General Intelligence. Atlantis Press. Containing:

Goertzel (2009). OpenCogPrime: A cognitive synergy based architecture for artificial general intelligence. 8th IEEE International Conference on Cognitive Information.

Goertzel, Arel, & Scheutz (2009). Toward a roadmap for human-level artificial general intelligence: Embedding HLAI systems in broad, approachable, physical or virtual contexts.

Lebiere, Gonzalez, & Warwick (2009). A comparative approach to understanding general intelligence: Predicing cognitive performance in an open-ended dynamic task. Department of Social and Decision Sciences, Paper 82. Carnegie Mellon University.

Schmidhuber (2009). Ultimate cognition a la Godel. Cognitive Computation, 1: 177-193.

Baum, Hutter, & Kitzelmann, eds. (2010). Proceedings of the Third Conference on Artificial General Intelligence. Atlantis Press. Containing:

Gluck, Stanley, Moore, Reitter, & Halbrugge (2010). Exploration for understanding in cognitive modeling. Journal of Artificial General Intelligence, 2(2): 88-107.

Lebiere, Gonzalez, & Warwick (2010). Cognitive architectures, model comparison, and AGI. Journal of Artificial General Intelligence, 2(2): 1-19.

Myers, Gluck, Gunzelmann, & Krusmark (2010). Validating computational cognitive process models across multiple timescales. Journal of Artificial General Intelligence, 2(2): 108-127.

Peebles & Banks (2010). Modelling dynamic decision makers with the ACT-R cognitive architecture. Journal of Artificial General Intelligence, 2(2): 52-68.

Reitter (2010). Metacognition and multiple strategies in a cognitive model of online control. Journal of Artificial General Intelligence, 2(2): 20-37.

Rohrer (2010). Accelerating progress in artificial general intelligence: Choosing a benchmark for natural world interaction. Journal of Artificial General Intelligence, 2(1): 1-28.

Stewart & West (2010). Testing for equivalence: A methodology for computational cognitive modelling. Journal of Artificial General Intelligence, 2(2): 69-87.

Hernandez-Orallo, Dowe, Espana-Cubillo, Hernandez-Lloreda, & Insa-Cabrera (2011). On more realistic environment distributions for defining, evaluating and developing intelligence. Artificial General Intelligence.

Schmidhuber, Thorisson, & Looks, eds. (2011). Artificial General Intelligence: 4th International Conference, AGI 2011. Springer. Containing:

Whole Brain Emulation

Martin (1971). Brief proposal on immortality: an interim solution. Perspectives in Biology and Medicine, 14: 339.

Moravec (1989). Mind Children. Harvard University Press.

Hanson (1994). If uploads come first: The crack of a future dawn. Extropy, 6(2): 1015.

Moravec (1999). Robot: Mere Machine to Transcendent Mind. Oxford University Press.

Yudkowsky (2001). Staring into the singularity.

Markram (2006). The Blue Brain Project. Nature Reviews Neuroscience, 7: 153-160.

Leitl (2007). Neurosuspension and uploading.

Malickas (2007). Gradual uploading as a cognition of mind.

Plesser, Eppler, Morrison, Diesmann, & Gewaltig (2007). Efficient parallel simulation of large-scale neuronal networks on clusters of multiprocessor computers. Lecture Notes in Computer Science, 4641: 672-681.

Hanson (2008). Economics of brain emulations. In Healey & Rayner (eds.), Unnatural Selection: The Challenges of Engineering Tomorrow's People. EarthScan.

Hanson (2008). Economics of the singularity. IEEE Spectrum.

Hines, Markram, & Schurmann (2008). Fully implicit parallel simulation of single neurons. Journal of Computational Neuroscience, 25: 439-448.

Sandberg & Bostrom (2008). Whole Brain Emulation: A Roadmap, Technical Report #2008?3. Future of Humanity Institute, Oxford University.

Yudkowsky (2008). Artificial Intelligence as Positive and Negative Factor in Global Risk. In Bostrom & Cirkovic, M. (eds.), Global Catastrophic Risks. Oxford University Press.

Asanthanarayan, Esser, Simon, & Modha (2009). The cat is out of the bag: Cortical simulations with 109 neurons, 1013 synapses. Proceedings of the Conference on High Performance Computing, Networking, Storage, and Analysis.

Astakhov (2010). Continuum of consciousness: mind uploading and resurrection of human consciousness.

Borzenko (2010). Indirect mind upload.

Cattell & Parker (2010). Challenges for whole brain emulation: Why is building a brain so difficult?

Shulman (2010). Whole brain emulation and the evolution of superorganisms. Singularity Institute.


If you have any corrections or additions for this bibliography, please contact the editor: luke [at] singinst [dot] org.