Thursday, January 6, 2011

Review of Jason Rosenhouse's "The Monty Hall Problem"

I've had the recent pleasure of reading Jason Rosenhouse's "The Monty Hall Problem." Rosenhouse's book is a comprehensive investigation into the eponymous Monty Hall problem, variations of the problem, and the larger implications of the problem.

The original Monty Hall problem named after a game played on an old television game show "Let's Make a Deal" with host Monty Hall. Rosenhouse describes the problem as:

You are shown three identical doors. Behind one of them is a car. The other two conceal goats. You are asked to choose, but not open one of the doors. After doing so, Monty, who knows where the car is, opens one of the two remaining doors. He always opens a door he knows to be incorrect, and randomly chooses which door to open when he has a more than one option (which happens on those occasions where your initial choice conceals the car). After opening an incorrect door, Monty gives you the option of either switching to the other unopened door or sticking with your original choice. You then receive whatever is behind the door you choose. What should you do?

(Presumably you are attempting to maximize your chance of winning one's chance of getting a car). Most people conclude that there's no benefit from switching. The general logic against switching is that after the elimination of a door there are two doors remaining, so each should now have a 1/2 chance of containing the door.

This logic is incorrect. One door has a 2/3rds chance of getting the car if one's general strategy is switching. Many people find this claim extremely counterintuitive. To see quickly the correctness of this claim, note that if one chooses a strategy of to always switching, then one will switch to the correct car-containing door exactly when your original door was not the car door. This will occur 2/3rd of the time.

Many people have great difficulty accepting the correct solution to the Monty Hall problem. This includes not just laypeople, but also professional mathematicians, including most famously Paul Erdos who initially did not accept the answer. The problem, and variants thereof, not only raise interesting questions of probability but also give insight into how humans think about probability.

Rosenhouse's book is very well done. He looks not just at the math, but also the history of the problem, and philosophical and psychological implications of the problem. For example, he discusses studies which show that cross-culturally the vast majority of people when given the problem will not switch. I was unaware until I read this book how much cross-disciplinary work there had been surrounding the Monty Hall problem. Not all of this work has been that impressive, and Rosenhouse correctly points out where much of the philosophical argumentation over the problem simply breaks downs. Along the way, Rosenhouse explains such important concepts as Bayes' Theorem (where he uses the simple discrete case), the different approaches to what probabilities mean (classical, frequentist, and Bayesian) and their philosophical implications. The book could easily be used for supplementary reading for an undergraduate course in probability or reading for an interested highschool student.

By far the most interesting parts of the book were the chapters focusing on the psychological aspects of the problem. Systematic investigation of the common failure of people to correctly analyze the Monty Hall problem has lead to much insight about how humans reason about probability. This analysis strongly suggests that humans use a variety of heuristics which generally work well for many circumstances humans run into but break down in extreme cases. In a short blog post I can’t do justice to the clever, sophisticated experimental set-ups used to test the nature and extent of these heuristics, so I'll simply recommend that people read the book.

For my own part, I'd like to use this as an opportunity to propose two continuous versions of the Monty Hall problem that to my knowledge have not been previously discussed. Consider a circle of circumference 1. A point is randomly picked as the target point on circle (and not revealed to you). You then pick a random interval of length 1/3rd on the circle. Monty knows where the target point is. If you picked an interval that contains the target point, Monty picks a random 1/3rd interval that doesn't overlap your interval and reveals that interval as not containing the target point. If your interval does not contain the target point, Monty instead picks uniformly a 1/3rd interval that doesn't include the target point and doesn't overlap with your interval. At the end of this process, you have with probability one, three possible intervals that might contain the target point, your original interval, or the intervals on either side of Monty's revealed interval. You are given the option to switch to one of these new intervals. Should you switch and if so to which interval?

I'm pretty sure that the answer in this modified form is also to switch, in this case switching to the larger of the two new intervals.

However, the situation becomes a bit trickier if we modify it a bit. Consider the following situation that is identical to the above, but instead of Monty cutting out an interval of length 1/3rd, he picks k intervals of each length 1/(3k) (thus the initial case above is k=1). Monty picks where to place these intervals by each picking one of the valid intervals uniformly and then going on to the other, then revealing the locations of all his intervals at the end. The remaining choices for an interval for you to pick are your original interval or any of the smaller intervals created in between Monty's choices. You get an option to stay or to switch to one of these intervals. It seems clear that even for k=2, sometiimes you should switch and sometimes you should not switch, depending on the locations of Monty's intervals. However, it isn't clear to me when to stay and when to switch. Thoughts are welcome.


Ian said...

Oh, he opened one of the other doors. I missed that part of the problem. Now it makes a lot more sense.

Ian said...

I do recall when I first came across the idea that the probability of the other event happening was (1-p) rather than q, and I was quite unhappy with the idea - mostly because I got the question wrong when I had been sure I was right.

Commonsense says that it shouldn't matter. You've got two options, one is right, one is wrong, and you have a 50-50 chance of being right. But phrase it a little differently and it makes a lot of sense.Don't take away one wrong answer. Let the person pick one door, then tell him/her "you can stick with your pick, or you can trade it for the other two. Phrase it that way and most people would (presumably) choose to switch.

Etienne Vouga said...

I think a lot of the confusion arises from people repeating imprecise or incomplete formulations of the problem, which can completely change the answer. For instance if Monty opens one of the two doors at random instead of knowing their contents and always opening a goat, there is no longer any benefit to switching doors.

Anonymous said...

The obvious guess in the 2nd continuous problem would be to switch to the largest new interval if it is bigger than 1/6, and stick if they are all less than 1/6 (since the "density" among the new intervals is twice that in the original one, so 1/6 in a new interval is equally likely to the 1/3 original interval). But this assumes that the 2/3 chance of it not being in your original interval is uniformly distributed among the unknown intervals, which is not obvious to me.

For example, if Monte picks an interval very near the edge of your original one, then he can't subsequently pick one inside that small gap, which might be an asymmetry that makes the simple uniform argument false. On the other hand, sometimes asymmetrical seeming situations turn out to be symmetric but you need to look carefully to see the symmetry.

I'm can't tell here

Ned Rosen

Miles Rind said...

Nice to see you posting again, Josh.

I once presented the Monty Hall puzzle on a message board to see how the responses would go and to try out different explanations of the solution. After a time, the responses of some posters simply exhausted my patience, as they could not respond without getting abusive and condescending in their defense of the erroneous answer.

One of them was particularly exasperating. At length he granted that, if you were playing the game in an open-ended series of rounds, then the strategy of switching when Monty gives you the chance would give you winning outcomes with a frequency that would approach 2/3; BUT this person insisted nonetheless that if you are playing just ONE round, then your chance of winning is 1/2 whether you switch or stay!

Apparently the backfire effect can occur even in non-empirical questions: the more seemingly cogent an explanation one gives of the correct solution, the more convinced of the incorrect solution some (not all) proponents of that solution grow.

By the way, the way of explaining the solution that seems most intuitively compelling to me (though, of course, no explanation is equally compelling to everyone) is this: First, imagine that there are two television studios. In studio A, the game is played as conventionally specified: (1) there are three doors, (2) the prize is hidden from the contestant behind one of the doors, (3) Monty knows where the prize is, (4) the contestant picks a door, (5) Monty opens a door that, as he knows, has no prize behind it, and (6) Monty then gives the contestant the opportunity to switch to the remaining yet-unchosen door. In studio B, everything goes the same way on points (1) through (5), but the contestant does not get the chance to switch: the opening of the empty door is just for show and makes no difference to the outcome. Now imagine that in studio A, a series of contestants play, every one of whom happens to stick to his original choice: for whatever reason, none of them ever takes the opportunity to switch to the remaining unchosen door.

Now the "50–50" view (the view that the chance of winning is 1/2 regardless of whether you switch doors or stick with your original choice) dictates the conclusion that the contestants in studio A will win at a long-run frequency of 1/2. In studio B, however, it should be evident that contestants will win with a long-run frequency of 1/3. But the contestants in studio A are doing exactly the same thing as the contestants in studio B. In order to believe that the contestants in studio A would win 1/2 the time rather than 1/3 of the time, you must believe that Monty's opening the door retroactively makes the contestant's original guess go from a 1/3 chance of success to a 1/2 chance of success. This is clearly superstitious nonsense. The contestants in studio A, who always stick to their original guess, must win only 1/3 of the time, not 1/2.