{"id":117,"date":"2011-12-15T13:17:44","date_gmt":"2011-12-15T21:17:44","guid":{"rendered":"http:\/\/facingthesing.wpengine.com\/?p=117"},"modified":"2014-12-01T13:46:52","modified_gmt":"2014-12-01T21:46:52","slug":"not-built-to-think-about-ai","status":"publish","type":"post","link":"https:\/\/intelligenceexplosion.com\/sv\/2011\/not-built-to-think-about-ai\/","title":{"rendered":"Inte byggda f\u00f6r att t\u00e4nka p\u00e5 AI"},"content":{"rendered":"

Tyv\u00e4rr \u00e4r denna artikel enbart tillg\u00e4nglig p\u00e5 English<\/a>, Fran\u00e7ais<\/a>, \u0440\u0443\u0441\u0441\u043a\u0438\u0439<\/a>, Sloven\u010dina<\/a>, \u4e2d\u6587<\/a> och Italiano<\/a>.<\/p>

I<\/span>n the Terminator<\/em> movies, the Skynet<\/a> AI becomes self-aware, kills billions of people, and dispatches killer robots to wipe out the remaining bands of human resistance fighters. That sounds pretty bad, but in NPR<\/em>\u2019s piece<\/a> on the intelligence explosion, eBay programmer Keefe Roedersheimer explained that the creation of an actual<\/em> machine superintelligence would be much worse than that.<\/p>\n

Martin<\/span> Kaste<\/span> (NPR): Much worse than Terminator<\/em>?<\/p>\n

Keefe<\/span> Roedersheimer<\/span>: Much, much worse.<\/p>\n

Kaste<\/span>: . . . That\u2019s a moonscape with people hiding under burnt out buildings being shot by lasers. I mean, what could be worse than that<\/em>?<\/p>\n

Roedersheimer<\/span>: All the people are dead.1<\/sup><\/a><\/p><\/blockquote>\n

Why did he say that? For most goals an AI could have\u2014whether it be proving the Riemann hypothesis<\/a> or maximizing oil production\u2014the simple reason is that<\/a> \u201cthe AI does not love you, nor does it hate you, but you are made of atoms which it can use for something else.\u201d2<\/sup><\/a> And when a superhuman AI notices that we humans are likely to resist<\/em> having our atoms used for \u201csomething else,\u201d and therefore pose a threat to the AI and its goals, it will be motivated to wipe us out as quickly as possible\u2014not in a way that exposes its critical weakness to an intrepid team of heroes who can save the world if only they can set aside their differences . . . No. For most goals an AI could have, wiping out the human threat to its goals, as efficiently as possible, will maximize the AI\u2019s expected utility<\/a>.<\/p>\n

\"\"<\/a><\/p>\n

And let\u2019s face it: there are easier ways to kill us humans than to send out human-shaped<\/em> robots that walk on the ground and seem to enjoy tossing humans into walls instead of just snapping their necks. Much better if the attack on humans is sudden, has simultaneous effects all over the world, is rapidly lethal, and defies available countermeasures. For example: Why not just use all that superintelligence to engineer an airborne, highly contagious and fatal supervirus? A 1918 mutation of the flu<\/a> killed 3% of Earth\u2019s population, and that was before air travel (which spreads diseases across the globe) was common, and without an ounce of intelligence going into the design of the virus. Apply a bit of intelligence, and you get a team from the Netherlands creating a variant of bird flu that \u201ccould kill half of humanity<\/a>.\u201d3<\/sup><\/a> A superintelligence could create something far worse. Or the AI could hide underground or blast itself into space and kill us all with existing<\/em> technology: a few thousand nuclear weapons<\/a>.<\/p>\n

The point is not that either of these particular<\/em> scenarios is likely. I\u2019m just trying to point out that the reality of a situation can be, of course, quite different from what makes for good storytelling. As Oxford philosopher Nick Bostrom put it<\/a>:<\/p>\n

When was the last time you saw a movie about humankind suddenly going extinct (without warning and without being replaced by some other civilization)?4<\/sup><\/a><\/p><\/blockquote>\n

It makes a better story if the fight is plausibly winnable by either side. The<\/em> Lord of the Rings<\/em> wouldn\u2019t have sold as many copies if Frodo had done the sensible thing and dropped the ring into the volcano from the back of a giant eagle<\/a>. And it\u2019s not an interesting story if humans suddenly lose<\/em>, full stop.<\/p>\n

When thinking about AI, we must not generalize from fictional evidence<\/a>. Unfortunately, our brains do that automatically<\/em>.<\/p>\n

A famous 1978 study asked subjects to judge which of two dangers occurred more often. Subjects thought accidents caused about as many deaths as disease, and that homicide was more frequent than suicide.5<\/sup><\/a> Actually, disease causes sixteen times as many deaths as accidents, and suicide is twice as common as homicide. What happened?<\/p>\n

Dozens of studies on the availability heuristic<\/a> suggest that we judge the frequency or probability of events by how easily they come to mind. That\u2019s not too bad a heuristic to have evolved in our ancestral environment when we couldn\u2019t check actual<\/em> frequencies on Wikipedia<\/em> or determine actual<\/em> probabilities with Bayes\u2019 Theorem<\/a>. The brain\u2019s heuristic is quick, cheap, and often right.<\/p>\n

But like so many of our evolved cognitive heuristics, the availability heuristic often gets the wrong results. Accidents are more vivid than diseases, and thus come to mind more easily, causing us to overestimate their frequency relative to diseases. The same goes for homicide and suicide.<\/p>\n

The availability heuristic also explains<\/a> why people think flying is more dangerous than driving when the opposite is true: a plane crash is more vivid and is reported widely when it happens, so it\u2019s more available to one\u2019s memory, and the brain tricks itself into thinking the event\u2019s availability<\/em> indicates its probability<\/em>.<\/p>\n

What does your brain do when it thinks about superhuman AI? Most likely, it checks all the instances of superhuman AI you\u2019ve encountered\u2014which, to make things worse, are all fictional<\/em>\u2014and judges the probability of certain scenarios by how well they match with the scenarios that come to mind most easily (due to your mind encountering them in fiction). In other words<\/a>, \u201cRemembered fictions rush in and do your thinking for you.\u201d<\/p>\n

So if you\u2019ve got intuitions about what superhuman AIs will be like, they\u2019re probably based on fiction<\/em>, without you even realizing it.<\/p>\n

This is why I began Facing the Intelligence Explosion<\/em> by talking about rationality<\/a>. The moment we start thinking about AI is the moment we run straight into a thicket of common human failure modes. Generalizing from fictional evidence is one of them. Here are a few others<\/a>:<\/p>\n