Friday, June 26, 2009

101 Interesting Things, part nineteen: The Monty Hall problem

So a while ago, someone in my space game brought up the Monty Hall problem in corporate e-mail. There was some argumentation, but I remembered that it had just recently been discussed at Philosophy Club (recently in terms of meetings, not days) as an example of our intuitions leading us astray. I thought I'd bring it up here as one of my hundred-and-one interesting things.

The Wikipedia page (linked above) goes into great detail about how it's one of the most misunderstood problems in history. As cognitive psychologist Massimo Piatelli-Palmarini says, "...no other statistical puzzle comes so close to fooling all the people all the time... even Nobel physicists systematically give the wrong answer, and ... insist on it, and they are ready to berate in print those who propose the right answer." This is a prime example of an extremely common failure to reason correctly, and a lesson that I think anyone could stand to learn from. Hell, I got it wrong at first, and was very resistant to having my mind changed until it was shown in excruciating detail just where I went wrong.

So, to the point! For those who did not click the link or don't know about Monty Hall, the problem is: you're on a game show and have advanced to the final round. Before you are three doors, behind one of which is a car, and behind the other two of which are goats. You pick a door, and before opening it, the host stops and opens one of the remaining doors, revealing a goat. He then asks, "Do you want to keep what you've got behind door x? Or risk it all to see what's behind door y?" What should you do to maximize your chance of winning the car? Should you stand pat? Should you switch? Does it even matter?

Most people think it doesn't matter - a goat has been eliminated, meaning that of the two doors left, one has the car, the other has a goat, so it's 50/50. It will not make a difference whether you stand pat or switch, they say. This is incorrect, and in a big way. In truth, standing (as a strategy) will only win you the car 1/3 of the time, whereas switching will win you the car 2/3 of the time. Exploiting this common failure to reason correctly, and exacerbating the problem with some suggestive phrasing, Monty Hall saved his show a lot of money by handing out a lot fewer prizes than they otherwise might have.

The reason it is this way is that when you make your initial choice, you only have a 1/3 chance of selecting the car (the "right" door), and this probability does not change when one of your choices is eliminated. Standing, as a strategy, stakes your bet on picking the right door of three. Since there is a 2/3 chance that you picked the wrong door, there's a 2/3 chance that the car is behind a door you did not pick, and thus Monty actually does you a favor by eliminating a goat - which ought to make your choice all the more obvious.

Another way of looking at it is that there's (statistically) a 1/3 chance behind each door, and when you make your choice, you eliminate that door from the "opening pool," fixing its probability (since Monty will never open the door you pick). Since Monty will also never eliminate the car, but there's a 2/3 chance that it's behind the group of un-picked doors, eliminating one door "consolidates" the odds behind the un-picked and unopened door.

One of our corporation members proposed an apt re-phrasing of the problem which I will outright steal (with a little change). Suppose there are a billion doors (and still only one car), so picking the right one off the bat is only a one-in-a-billion chance. You pick one anyway, and then Monty opens all the remaining doors but one, showing goats behind all of them. The odds are overwhelmingly likely that you picked a wrong door, and the car is behind one of the other nine-hundred ninety-nine million, nine-hundred ninety-nine thousand, nine-hundred ninety-nine doors (the number is more impressive when written out, I promise). When you are given your second chance, it's obvious you should switch, since Monty's done most of the work for you. The same reasoning applies to the three-door scenario, it's just scaled down a bit and thus not so obvious, since only one door is picked by you and only one door is opened by Monty.

As an interesting side note, a less-obvious rephrasing could be that he eliminates one of the remaining doors instead of all but one, which just so happens to pan out as an identical behavior in the three-door scenario. Let's say there are four doors, so you have a 25% chance of picking correctly right off the bat. The remaining 75% is distributed evenly among the other three doors, until one is opened, at which point it's distributed among the two unopened and un-picked doors. Since we have no means by which to distinguish them, they're even, and both get half of the opened door's quarter, putting them at 37.5% each, so you should still switch. A five-door scenario, by these rules, yields a 20% chance for standing, and 26.7% behind each of three doors. As the number of doors rises, the margin of advantage to switching shrinks, but it's still non-zero and higher, thus making it always better to switch. The trend happens to have its peak in the three-door scenario.

I feel like this should be wrapped up somehow, but now all I can think of is other interesting problems in game theory, which is like the coolest thing ever.

1 comment:

Joffan said...

Amazing, a Monty Hall post without a single comment. Allow me to besmirch its maiden purity, mwahaha.

I concocted a similar analogy to your unnamed colleague, except I only went for one hundred doors, not one billion. However I have another version of the game which might help to illuminate the situation for some.

Suppose we have the set-up as originally, you choose your door and Monty steps forward. However, instead of opening a door, he allows you the option to change your pick from your chosen door to both of the other doors. Now it is clear that switching is the better option. Translating this back into the original format, the option is to switch from the original choice to both the other doors, at the cost of a goat (which you are assured, by display, you will have). So you switch.

Personally I believe that most people are so bad at understanding probability etc that the odds never really figure very strongly in the decision. The gut level argument is more like this: "If I switch, and lose the car, I'll feel a lot worse than if I stick with what I have and unluckily don't get the car." So they don't switch.