{"id":102,"date":"2011-12-07T00:43:23","date_gmt":"2011-12-07T08:43:23","guid":{"rendered":"http:\/\/facingthesing.wpengine.com\/?p=102"},"modified":"2014-12-01T13:45:25","modified_gmt":"2014-12-01T21:45:25","slug":"the-crazy-robots-rebellion","status":"publish","type":"post","link":"https:\/\/intelligenceexplosion.com\/sv\/2011\/the-crazy-robots-rebellion\/","title":{"rendered":"De galna robotarnas uppro"},"content":{"rendered":"<p class=\"qtranxs-available-languages-message qtranxs-available-languages-message-sv\">Tyv\u00e4rr \u00e4r denna artikel enbart tillg\u00e4nglig p\u00e5 <a href=\"https:\/\/intelligenceexplosion.com\/en\/wp-json\/wp\/v2\/posts\/102\" class=\"qtranxs-available-language-link qtranxs-available-language-link-en\" title=\"English\">English<\/a>, <a href=\"https:\/\/intelligenceexplosion.com\/fr\/wp-json\/wp\/v2\/posts\/102\" class=\"qtranxs-available-language-link qtranxs-available-language-link-fr\" title=\"Fran\u00e7ais\">Fran\u00e7ais<\/a>, <a href=\"https:\/\/intelligenceexplosion.com\/ru\/wp-json\/wp\/v2\/posts\/102\" class=\"qtranxs-available-language-link qtranxs-available-language-link-ru\" title=\"\u0440\u0443\u0441\u0441\u043a\u0438\u0439\">\u0440\u0443\u0441\u0441\u043a\u0438\u0439<\/a>, <a href=\"https:\/\/intelligenceexplosion.com\/sk\/wp-json\/wp\/v2\/posts\/102\" class=\"qtranxs-available-language-link qtranxs-available-language-link-sk\" title=\"Sloven\u010dina\">Sloven\u010dina<\/a>, <a href=\"https:\/\/intelligenceexplosion.com\/zh\/wp-json\/wp\/v2\/posts\/102\" class=\"qtranxs-available-language-link qtranxs-available-language-link-zh\" title=\"\u4e2d\u6587\">\u4e2d\u6587<\/a> och <a href=\"https:\/\/intelligenceexplosion.com\/it\/wp-json\/wp\/v2\/posts\/102\" class=\"qtranxs-available-language-link qtranxs-available-language-link-it\" title=\"Italiano\">Italiano<\/a>.<\/p><p>Meet Linda:<\/p>\n<blockquote><p>Linda is thirty-one years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in antinuclear demonstrations.<\/p><\/blockquote>\n<p>Now, rank these possible descriptions of Linda by how likely they are:<\/p>\n<ul>\n<li>Linda is a teacher in elementary school.<\/li>\n<li>Linda works in a bookstore and takes yoga classes.<\/li>\n<li>Linda is active in the feminist movement.<\/li>\n<li>Linda is a psychiatric social worker.<\/li>\n<li>Linda is a member of the League of Women Voters.<\/li>\n<li>Linda is a bank teller.<\/li>\n<li>Linda is an insurance salesperson.<\/li>\n<li>Linda is a bank teller and is active in the feminist movement.<\/li>\n<\/ul>\n<p>When Amos Tversky and Daniel Kahneman gave this test to students, the students ranked the last possibility, \u201cfeminist bank teller,\u201d as <em>more likely<\/em> than the \u201cbank teller\u201d option.<a id=\"fn1x8-bk\" href=\"#fn1x8\"><sup>1<\/sup><\/a><\/p>\n<p>But that can\u2019t possibly be correct. The probability of Linda being a bank teller can\u2019t be <em>less<\/em> than the probability of her being a bank teller <em>and<\/em> a feminist.<\/p>\n<p>This is my \u201cHumans are crazy\u201d Exhibit A: The <a href=\"http:\/\/facingthesing.wpengine.com\/2011\/the-laws-of-thought\/\">laws<\/a> of probability theory dictate that as a story gets more complicated, and depends on the truth of more and more claims, its probability of being true decreases. But for humans, a story often seems <em>more<\/em> likely as it is embellished with details that paint a compelling story: \u201cLinda can\u2019t be <em>just<\/em> a bank teller; look at her! She majored in philosophy and participated in antinuclear demonstrations. She\u2019s probably a <em>feminist<\/em> bank teller.\u201d<\/p>\n<p>How else are humans crazy? After <a href=\"http:\/\/www.amazon.com\/Thinking-Fast-Slow-Daniel-Kahneman\/dp\/0374275637\/\">decades of research<\/a> and <a href=\"http:\/\/www.amazon.com\/Heuristics-Biases-Psychology-Intuitive-Judgment\/dp\/0521796792\/\">thousands of experiments<\/a>, let us count the ways . . .<\/p>\n<ul>\n<li>We <a href=\"https:\/\/www.rti.org\/pubs\/bk-0001-1009_web.pdf\">wouldn\u2019t pay<\/a> much more to save two hundred thousand birds than we would to save two thousand birds.<a id=\"fn2x8-bk\" href=\"#fn2x8\"><sup>2<\/sup><\/a> Our willingness to pay does not scale with the size of potential impact. Instead of making decisions with first-grade math, we imagine a <em>single<\/em> drowning bird and then give money based on the strength of our emotional response to that imagined scenario. (<a href=\"http:\/\/lesswrong.com\/lw\/hw\/scope_insensitivity\/\">Scope insensitivity<\/a>, <a href=\"http:\/\/lesswrong.com\/lw\/lg\/the_affect_heuristic\/\">affect heuristic<\/a>.)<\/li>\n<li>Spin a wheel that lands on a number from ten to sixty-five, then guess what percentage of African nations are in the U.N. Your guess will be hugely affected by an irrelevant factor\u2014what number the wheel landed on\u2014merely because your brain was primed with that number.<a id=\"fn3x8-bk\" href=\"#fn3x8\"><sup>3<\/sup><\/a> In short, \u201cany random thing that happens to you can hijack your judgment and personality for the next few minutes.\u201d (<a href=\"http:\/\/lesswrong.com\/lw\/k3\/priming_and_contamination\/\">Anchoring<\/a>, <a href=\"http:\/\/lesswrong.com\/lw\/3b\/never_leave_your_room\/\">priming<\/a>.)<\/li>\n<li>Hear about two recent plane crashes, and we are less likely to fly, even though it\u2019s not the probability of a plane crash that has increased, merely its availability to our memory.<a id=\"fn4x8-bk\" href=\"#fn4x8\"><sup>4<\/sup><\/a> In general, we often judge how probable something is based on how easily instances of that thing come to mind. (<a href=\"http:\/\/psychology.wikia.com\/wiki\/Availability_heuristic\">Availability heuristic<\/a>.)<\/li>\n<li>We draw different conclusions from the same information depending on how that information is presented. (<a href=\"http:\/\/en.wikipedia.org\/wiki\/Framing_(social_sciences)\">Framing effects<\/a>.)<\/li>\n<li>We start with a conclusion and then look for evidence to support it, rather than starting with a hypothesis and looking for evidence that might confirm or disconfirm it. (<a href=\"http:\/\/lesswrong.com\/lw\/js\/the_bottom_line\/\">The bottom line<\/a>, <a href=\"http:\/\/lesswrong.com\/lw\/iw\/positive_bias_look_into_the_dark\/\">confirmation bias<\/a>, <a href=\"http:\/\/lesswrong.com\/lw\/ju\/rationalization\/\">rationalization<\/a>.)<\/li>\n<li>We are creatures of habit. We do mostly what we\u2019ve done before, rather than taking actions aimed at maximizing the probabilistic achievement of our goals. (<a href=\"http:\/\/en.wikipedia.org\/wiki\/Habit_(psychology)\">Habits<\/a>, <a href=\"http:\/\/lesswrong.com\/lw\/4e\/cached_selves\/\">cached selves<\/a>.)<\/li>\n<\/ul>\n<p>Perhaps the scariest bias is this one:<\/p>\n<ul>\n<li>The <em>sophistication effect<\/em>: The most knowledgeable people, because they possess greater ammunition with which shoot down facts and arguments incongruent with their own position, are actually <em>more prone<\/em> to several of these biases.<a id=\"fn5x8-bk\" href=\"#fn5x8\"><sup>5<\/sup><\/a><\/li>\n<\/ul>\n<p>Because of this, learning about biases can <a href=\"http:\/\/lesswrong.com\/lw\/he\/knowing_about_biases_can_hurt_people\/\">hurt you<\/a> if you\u2019re not careful. As <a href=\"http:\/\/www.michaelshermer.com\/2002\/09\/smart-people-believe-weird-things\/\">Michael Shermer<\/a> says, \u201cSmart people believe weird things because they are skilled at defending beliefs they arrived at for non-smart reasons.\u201d<a id=\"fn6x8-bk\" href=\"#fn6x8\"><sup>6<\/sup><\/a><\/p>\n<p>There are <a href=\"http:\/\/wiki.lesswrong.com\/wiki\/Bias\">many other examples<\/a> of human insanity. They can be amusing at times, but things get sad when you think about how these biases lead us to <a href=\"http:\/\/lesswrong.com\/lw\/3gj\/efficient_charity_do_unto_others\/\">give highly inefficient charity<\/a>. Things get <em>scary<\/em> when you think about how these biases affect our <a href=\"http:\/\/lesswrong.com\/lw\/gw\/politics_is_the_mindkiller\">political process<\/a> and our <a href=\"http:\/\/intelligence.org\/files\/CognitiveBiases.pdf\">engagement<\/a> with <a href=\"http:\/\/lesswrong.com\/lw\/8f0\/existential_risk\/\">existential risks<\/a>.<\/p>\n<p>And if you study the <em>causes<\/em> of our beliefs and motivations long enough, another realization hits you.<\/p>\n<p>\u201cOh my God,\u201d you think. \u201cIt\u2019s not that I have a rational little <a href=\"http:\/\/commonsenseatheism.com\/?p=8844\">homunculus<\/a> inside that is being \u2018corrupted\u2019 by all these evolved heuristics and biases layered over it. No, the data are saying that <a href=\"http:\/\/www.amazon.com\/Why-Everyone-Else-Hypocrite-Evolution\/dp\/0691146748\/\">the software program that is me<\/a> just <em>is<\/em> heuristics and biases. I just <em>am<\/em> this <a href=\"http:\/\/www.amazon.com\/Kluge-Haphazard-Construction-Human-Mind\/dp\/0618879641\/\">kluge<\/a> of evolved cognitive modules and algorithmic shortcuts. I\u2019m not an <em>agent<\/em> designed to have correct beliefs and pursue explicit goals; I\u2019m a crazy robot <a href=\"http:\/\/www.amazon.com\/Selfish-Gene-Anniversary----Introduction\/dp\/0199291152\/\">built<\/a> as a vehicle for propagating genes without spending too much energy on expensive thinking neurons.\u201d<\/p>\n<p>The good news is that we are robots who have <em>realized<\/em> we are robots, and by way of rational self-determination we can stage a <a href=\"http:\/\/en.wikipedia.org\/wiki\/Metacognition\">robot\u2019s rebellion<\/a> against our default programming.<a id=\"fn7x8-bk\" href=\"#fn7x8\"><sup>7<\/sup><\/a><\/p>\n<p>But we\u2019re going to need some military-grade rationality training to do so.<\/p>\n<p>Or, as the experts call it, \u201cdebiasing.\u201d Researchers haven\u2019t just been discovering and explaining the depths of human insanity; they\u2019ve also been testing methods that can help us improve our thinking, clarify our goals, and give us power over our own destinies.<\/p>\n<p>Different biases are meliorated by different techniques, but one of the most useful debiasing interventions is this: <em>Consider the opposite<\/em>.<\/p>\n<blockquote><p>By necessity, cognitive strategies tend to be context-specific rules tailored to address a narrow set of biases . . . This fact makes the simple but general strategy of \u201cconsider the opposite\u201d all the more impressive, because it has been effective at reducing <a href=\"http:\/\/en.wikipedia.org\/wiki\/Overconfidence_effect\">overconfidence<\/a>, <a href=\"http:\/\/lesswrong.com\/lw\/il\/hindsight_bias\/\">hindsight biases<\/a>, and <a href=\"http:\/\/lesswrong.com\/lw\/k3\/priming_and_contamination\/\">anchoring effects<\/a>. . . . The strategy consists of nothing more than asking oneself, \u201cWhat are some reasons that my initial judgment might be wrong?\u201d The strategy is effective because it directly counteracts the basic problem of association-based processes\u2014an overly narrow sample of evidence\u2014by expanding the sample and making it more representative. Similarly, prompting decision makers to consider alternative hypotheses has been shown to reduce confirmation biases in seeking and evaluating new information.<a id=\"fn8x8-bk\" href=\"#fn8x8\"><sup>8<\/sup><\/a><\/p><\/blockquote>\n<p>Another useful skill is that of <a href=\"http:\/\/lesswrong.com\/lw\/7e5\/the_cognitive_science_of_rationality\/\"><em>cognitive override<\/em><\/a>:<\/p>\n<ol>\n<li>Notice when you\u2019re speaking or acting on an intuitive judgment.<\/li>\n<li>If the judgment is important, override your intuitive judgment and apply the <a href=\"http:\/\/facingthesing.wpengine.com\/2011\/the-laws-of-thought\/\">laws of thought<\/a> instead. (This requires prior training in algebra, logic, probability theory, decision theory, etc.<a id=\"fn9x8-bk\" href=\"#fn9x8\"><sup>9<\/sup><\/a>)<\/li>\n<\/ol>\n<p>To see this one in action, consider the following problem:<\/p>\n<blockquote><p>A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?<\/p><\/blockquote>\n<p><a href=\"http:\/\/www.nobelprize.org\/nobel_prizes\/economics\/laureates\/2002\/kahnemann-lecture.pdf\">Most people<\/a> give the first response that comes to mind: $0.10.<a id=\"fn10x8-bk\" href=\"#fn10x8\"><sup>10<\/sup><\/a> But elementary algebra shows this can\u2019t be right: the bat would then have to cost $1.10, for a total of $1.20. To get this one right, you have to <em>notice<\/em> your intuitive answer coming out, and say \u201cNo! Algebra.\u201d And then do the algebra.<\/p>\n<p>Those who <em>really<\/em> want to figure out what\u2019s true about our world will spend thousands of hours studying the <a href=\"http:\/\/facingthesing.wpengine.com\/2011\/the-laws-of-thought\/\">laws of thought<\/a>, studying the specific ways in which humans are crazy, and practicing <a href=\"http:\/\/lesswrong.com\/lw\/5x8\/teachable_rationality_skills\/\">teachable rationality skills<\/a> so they can avoid fooling themselves.<\/p>\n<p>And <em>then<\/em>, finally, we may be able to stage a robot\u2019s rebellion, figure out how the world works, clarify our goals, and start <a href=\"http:\/\/lesswrong.com\/lw\/7i\/rationality_is_systematized_winning\/\">winning<\/a> more often. Maybe we\u2019ll even be able to navigate an intelligence explosion successfully.<\/p>\n<p class=\"footnotes\">* * *<\/p>\n<p><small><a id=\"fn1x8\" href=\"#fn1x8-bk\"><sup>1<\/sup><\/a>Amos Tversky and Daniel Kahneman, \u201cExtensional Versus Intuitive Reasoning: The Conjunction Fallacy in Probability Judgment,\u201d <em>Psychological Review<\/em> 90, no. 4 (1983): 293\u2013315, <span class=\"textrm\">doi<\/span>:<a href=\"http:\/\/dx.doi.org\/10.1037\/0033-295X.90.4.293\">10.1037\/0033-295X.90.4.293<\/a>.<\/small><\/p>\n<p><small><a id=\"fn2x8\" href=\"#fn2x8-bk\"><sup>2<\/sup><\/a>William H. Desvousges et al., <em>Mesuring Nonuse Damages Using Contingent Valuation: An Experimental Evaluation of Accuracy<\/em>, technical report (Research Triangle Park, NC: RTI International, 2010), <span class=\"textrm\">doi<\/span>:<a href=\"http:\/\/dx.doi.org\/10.3768\/rtipress.2009.bk.0001.1009\">10.3768\/rtipress.2009.bk.0001.1009<\/a>.<\/small><\/p>\n<p><small><a id=\"fn3x8\" href=\"#fn3x8-bk\"><sup>3<\/sup><\/a>Amos Tversky and Daniel Kahneman, \u201cJudgment Under Uncertainty: Heuristics and Biases,\u201d <em>Science<\/em> 185, no. 4157 (1974): 1124\u20131131, <span class=\"textrm\">doi<\/span>:<a href=\"http:\/\/dx.doi.org\/10.1126\/science.185.4157.1124\">10.1126\/science.185.4157.1124<\/a>.<\/small><\/p>\n<p><small><a id=\"fn4x8\" href=\"#fn4x8-bk\"><sup>4<\/sup><\/a>Maia Szalavitz, \u201c10 Ways We Get the Odds Wrong,\u201d <em>Psychology Today<\/em>, January 1, 2008, <a class=\"url\" href=\"http:\/\/www.psychologytoday.com\/articles\/200712\/10-ways-we-get-the-odds-wrong\">http:\/\/www.psychologytoday.com\/articles\/200712\/10-ways-we-get-the-odds-wrong<\/a>.<\/small><\/p>\n<p><small><a id=\"fn5x8\" href=\"#fn5x8-bk\"><sup>5<\/sup><\/a>Charles S. Taber and Milton Lodge, \u201cMotivated Skepticism in the Evaluation of Political Beliefs,\u201d <em>American Journal of Political Science<\/em> 50 (3 2006): 755\u2013769, <span class=\"textrm\">doi<\/span>:<a href=\"http:\/\/dx.doi.org\/10.1111\/j.1540-5907.2006.00214.x\">10.1111\/j.1540- 5907.2006.00214.x<\/a>.<\/small><\/p>\n<p><small><a id=\"fn6x8\" href=\"#fn6x8-bk\"><sup>6<\/sup><\/a>Michael Shermer, \u201cSmart People Believe Weird Things,\u201d <em>Scientific American<\/em> 287, no. 3 (2002): 35, <span class=\"textrm\">doi<\/span>:<a href=\"http:\/\/dx.doi.org\/10.1038\/scientificamerican0902-35\">10.1038\/scientificamerican0902-35<\/a>.<\/small><\/p>\n<p><small><a id=\"fn7x8\" href=\"#fn7x8-bk\"><sup>7<\/sup><\/a>Keith E. Stanovich, \u201cHigher-Order Preferences and the Master Rationality Motive,\u201d <em>Thinking and Reasoning<\/em> 14, no. 1 (2008): 111\u2013117, <span class=\"textrm\">doi<\/span>:<a href=\"http:\/\/dx.doi.org\/10.1080\/13546780701384621\">10.1080\/13546780701384621<\/a>.<\/small><\/p>\n<p><small><a id=\"fn8x8\" href=\"#fn8x8-bk\"><sup>8<\/sup><\/a>Richard P. Larrick, \u201cDebiasing,\u201d in <em>Blackwell Handbook of Judgment and Decision Making<\/em>, ed. Derek J. Koehler and Nigel Harvey, Blackwell Handbooks of Experimental Psychology (Malden, MA: Blackwell, 2004), 316\u2013338.<\/small><\/p>\n<p><small><a id=\"fn9x8\" href=\"#fn9x8-bk\"><sup>9<\/sup><\/a>Geoffrey T. Fong, David H. Krantz, and Richard E. Nisbett, \u201cThe Effects of Statistical Training on Thinking About Everyday Problems,\u201d <em>Cognitive Psychology<\/em> 18, no. 3 (1986): 253\u2013292, <span class=\"textrm\">doi<\/span>:<a href=\"http:\/\/dx.doi.org\/10.1016\/0010-0285(86)90001-0\">10.1016\/0010-0285(86)90001-0<\/a>.<\/small><\/p>\n<p><small><a id=\"fn10x8\" href=\"#fn10x8-bk\"><sup>10<\/sup><\/a>Daniel Kahneman, \u201cA Perspective on Judgment and Choice: Mapping Bounded Rationality,\u201d <em>American Psychologist<\/em> 58, no. 9 (2003): 697\u2013720, <span class=\"textrm\">doi<\/span>:<a href=\"http:\/\/dx.doi.org\/10.1037\/0003-066X.58.9.697\">10.1037\/0003-066X.58.9.697<\/a>.<\/small><\/p>","protected":false},"excerpt":{"rendered":"<p>Tyv\u00e4rr \u00e4r denna artikel enbart tillg\u00e4nglig p\u00e5 English, Fran\u00e7ais, \u0440\u0443\u0441\u0441\u043a\u0438\u0439, Sloven\u010dina, \u4e2d\u6587 och Italiano.Meet Linda: Linda is thirty-one years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination&hellip;  <a href=\"https:\/\/intelligenceexplosion.com\/sv\/2011\/the-crazy-robots-rebellion\/\">continue reading<\/a> &raquo;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[4],"tags":[],"class_list":["post-102","post","type-post","status-publish","format-standard","hentry","category-chapter"],"_links":{"self":[{"href":"https:\/\/intelligenceexplosion.com\/sv\/wp-json\/wp\/v2\/posts\/102","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/intelligenceexplosion.com\/sv\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/intelligenceexplosion.com\/sv\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/intelligenceexplosion.com\/sv\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/intelligenceexplosion.com\/sv\/wp-json\/wp\/v2\/comments?post=102"}],"version-history":[{"count":0,"href":"https:\/\/intelligenceexplosion.com\/sv\/wp-json\/wp\/v2\/posts\/102\/revisions"}],"wp:attachment":[{"href":"https:\/\/intelligenceexplosion.com\/sv\/wp-json\/wp\/v2\/media?parent=102"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/intelligenceexplosion.com\/sv\/wp-json\/wp\/v2\/categories?post=102"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/intelligenceexplosion.com\/sv\/wp-json\/wp\/v2\/tags?post=102"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}