Adventures in math: finding a red dot in a book, Marilyn vos Savant is wrong!
Mood: geeky
Posted on 2013-05-05 11:43:00
Tags: math
Words: 567

Let's consider the following problem, which is based on reality and caused a 15 minute argument lately: You're given a book and told it has a 90% chance of having one red dot in it (and a 10% chance of having no red dot). If you read the first half of the book and don't find a red dot, what is the probability the red dot is in the second half of the book?

To join in our argument, your choices are 90% or something less than 90%.

My stance (which I think is correct!) was that the chances go down the more you read. There are two equivalent ways to think about this:

- Assuming the location of the red dot is distributed uniformly throughout the book, before you start reading the book the chances that the red dot is in the first half of the book are 45%, in the second half 45%, and not at all is 10%. Once you know it's not in the first half of the book you can eliminate that first 45% chance, so the chances it's in the second half are (.45)/(.45+.1) = .45/.55, or around 82%.

- You can formalize this by using Bayes' theorem. Let's set up our events:
A = red dot is in book
B = red dot is not in first half of book

So the probability we want is P(A|B). (meaning the probability that A is true given that B is true)
Some values we're going to need:
P(A) = .9
P(B|A) = .5 (this is where the assumption that the distribution of the red dot is uniform comes in)
P(B|not A) = 1 (if the red dot isn't in the book, it's not in the first half!)
P(B) = P(B|A)*P(A) + P(B|not A)*P(not A)
= .5*.9 + 1*.1
= .55

So, by Bayes' theorem, P(A|B) = (P(B|A) * P(A))/(P(B)) = .5*.9/.55=.45/.55, which is the same result as we got above. You can see that Bayes' theorem is really just formalizing the logic we were using in the first case!

You can generalize this - if the red dot isn't in the first x of the book (for x between 0 and 1), then P(B|A) = 1 - x, and we end up with
((1-x)*.9)/((1-x)*.9 + .1)

So for x = 0 this is .9, as we expect (since we haven't read anything) and for x = 1 this is 0 (since we know it's not in the book). And here's the Wolfram Alpha plot of the function!

--

I was reading Parade this morning, and Marilyn Vos Savant's column had a math problem, which made me say goody! Here's the column. Unfortunately her answer is wrong. The problem with counting the combinations this way is that we're overcounting some of them. In her example, 5 is the number that you know is in the combination, and in step 1 we count the numbers 5000-5999. But then in the next step we count 0500-9599, keeping the 5 constant, and this double counts the numbers from 5500-5599. Similarly the third and fourth steps overcount, and the combination 5555 is counted four times!

The correct way to do this problem in the general case is to use the Inclusion-exclusion principle, but an even easier way is to read "at least one 5" as "everything except no 5's". So there are 10^4 total combinations, and 9^4 of them have no 5's, so the number of combinations that have at least one 5 is 10^4-9^4=3439, significantly less than 4000.

11 comments

Comment from alteredhistory:
2013-05-05T14:00:19+00:00
The counterargument to Greg: the expected probability of finding the dot on the first page is your expected probability for the whole book divided by the number of pages. When you do not find a dot on the first page, you know nothing more about the book as a whole, so the probability on the second page is slightly higher -- book probability divided by pages minus one. It is an asymptotic function with a big divide-by-zero when you check the final page.

The flaw in Greg's argument is that the expected probability is not the same as the actual probability. Whether or not the dot is included was decided by principles well beyond your knowledge. IF reading a page made you reevaluate the content of the book, then I would agree with part of Greg's analysis. But Bayes' Theorem requires an input of how confident you are in the original estimate and how confident you are that new information should make you reevaluate confidence

If you check a whole book, then you would adjust up or down your confidence up or down for the next whole book from that publisher. But having checked a single page does not make you reevaluate *this* book.

Comment from ext_1788645:
2013-05-05T17:23:29+00:00
Greg is correct.

Let's say you read 90% of the pages. You're saying there is a 90% chance the red dot is on one of the remaining pages. And, when you read all but the last page, there is a 90% chance it will be on the last page.

The logic flaw is that you do, in fact, learn something about the location of the red dot when you read half the book and don't find it. It's not like the Monty Hall problem, where you really don't learn anything about your choice when the goat is revealed.

Greg is correct that you should assume a 45% chance the dot is in each half of the book. When you find no dot in the first half of the book, there is a .45/.55 probability in the last half of the book.

Comment from alteredhistory:
2013-05-05T19:03:03+00:00
No. I am NOT saying there is a 90% chance of the red dot being there.
I am saying that *I EXPECT* there is a 90% chance of the red dot being there.

Those are two very different thing. I have no expectations when I start about which page the red dot will be on. None. So discovering that the dot is NOT on page 1 does not change my expectation about finding the dot *somewhere*.

Essentially, I am saying that I don't have a 90% expectation for any page *except* the last page that I check.

Comment from gregstoll:
2013-05-05T18:11:28+00:00
I think my chief disagreement here is

When you do not find a dot on the first page, you know nothing more about the book as a whole

Comment from alteredhistory:
2013-05-05T19:27:20+00:00
Your theorem leads to an incorrect belief in the following scenario:

Let's say the author writes 100 books. He chooses 90 of those books to put a red dot in. He does NOT roll a die. Instead, he puts the red dot on the last page of every single book.

If that's the case, then when you read half of every book, you would encounter the red dot on none of them. Under your rule, you would then assume that none of the books included a red dot, with near certainty.

You are collecting information that does not apply.

On the other hand, my knowledge of the author suggests that he included the red dot *somewhere*. That it wasn't in the first half just tells me that he probably didn't use random positioning to choose the red dot's location -- which is more than likely true. So when I continue to believe that 90% of those books do include a red dot, I have the more correct view of the world _regardless of where the author *chooses* to put the dot_.

Comment from alteredhistory:
2013-05-05T19:28:23+00:00
Basically, my interpretation works for any position the author chooses for the dot. Yours only works if the author is randomly selecting pages.

Comment from gregstoll:
2013-05-05T21:12:11+00:00
Yes, and part of my assumption was that the dot is uniformly distributed when it's in the book.

If we have no idea about how the author is putting the dots in the book, no interpretation in the world is going to help. Maybe the author is actively trying to thwart us and always putting it right after where we're going to expect. Maybe there are no red dots at all! But in the real life situation this was based on, I had no reason to believe the red dot was more likely to be at any spot than at any other.

Comment from alteredhistory:
2013-05-05T23:09:59+00:00
We are both wrong. I ran the simulations. The probability fall off is weird.

https://plus.google.com/116077417321878358008/posts/YJrxFXHMCtY

Comment from alteredhistory:
2013-05-06T00:59:02+00:00
And when I run Bayes Theorem, I get similar non-linear result.

newx = (xy) / (xy + z(1-x))

https://plus.google.com/116077417321878358008/posts/Q9bHSEtesxQ

Given a 100 page book...

x = probability beforehand of dot NOT in book = 0.1
y = probability of first page being blank if the book does not have a red dot = 1 (total certainty)
z = probability of first page being blank if the book does have a red dot = 0.99 (because 99 of 100 pages do not have red dot)

newx = 0.100908
The probability decreases, as you expected, but not linearly.

Comment from gregstoll:
2013-05-06T09:38:34+00:00
Indeed, both of these results look very similar to the Wolfram Alpha plot of ((1-x)*.9)/((1-x)*.9 + .1) that I linked to above.

Comment from trice_ranger:
2013-05-07T01:21:11+00:00
Is that if we can't figure out a math problem with the kids' homework, (unlikely with Stephen, I know), I know who I'm calling.

Also, I may have the kids learn calculus or programming or something over the summers at y'all's house, someday... Mwah-ha-ha-ha...

<3

This backup was done by LJBackup.