In a world as crazy as this one,
it ought to be easy to find something that happens solely by chance.
It isn't.

Kevin McKeen [ quoted in Probability ]

Topics

• Suppose I have three cards. One is red on both sides, one is white on both sides, and one is red on one side and white on the other. I shuffle the cards, and then pick one and lay it flat on a table. It is red, i.e. the side showing is red. I now ask you: how likely is it that the side not showing is also red?
  Once you have decided on your answer, write it down, and read on. .
  Consider a puzzle, or an experiment, which has a number of possible outcomes. For example, this is the set of possible outcomes when you throw a die: { 1, 2, 3, 4, 5, 6 }. Suppose now that I throw the die 'many, many times,' and then examine how many times the die came to a stop showing 1 or 2, or 3, etc. The ratio between the number of times a certain number showed up and the total number of trials is a plausible estimate of how likely that number is to show up if you throw the die again. The likelihood that next time I throw the die 5 shows up could thus be said to be (number of times 5 showed up so far) divided by (number of times I have thrown the die).
  Here is a little instructive program, borrowed from Probability. Click the Start button, and see how, as the number of trials grows (one per second), the numbers in the three boxes will build up. They should all be the same, and if you let the program run, they will differ less and less from each other. Notice, however, that only seldom are the three numbers exactly equal.

  1	2	3

  So, perhaps not unexpectedly, the greater the number of trials is, the more accurate the estimate becomes. In the case of the dice, assuming that it is a fair dice, this estimate is the same for all possible outcomes, and is 1/6. In the case of a fair coin (head, tail), the estimate would be 1/2. Therefore we must ask, what do we mean by "many, many times"? Without entering into mathematical technicalities, I will simply mention that the answer is the so-called Law of Large Numbers. "This law justifies the intuitive interpretation of the expected value of a random variable as the 'long-term average when sampling repeatedly'." [ from Wikipedia ] Here is a famous example: Buffon's Needle. Imagine a large piece of paper with a large number of straight, parallel lines drawn on it. Now take a needle and drop it on the paper. Question: how likely is that the needle will cross one of the lines? The answer is 2/π, or approximately 0.6366197. (Notice that this is a neat way to calculate π !). Now, on the webpage referred to above, the author dropped the needle 240 times, and got 0.6363636, pretty close to 0.6366197. If he had dropped it 1000 times, the agreement would have likely be even closer.
  You may want to check The Flip Site, where Mr Flipper "has been flipping coins every five seconds since midnight on January 1, 2004! By the end of this year, he will have flipped his coin more than six million times … Every time he flips a coin, the results are stored in our FlipParade where you can see the last series of current flips or scroll back in time to see the rest. Why do we do this? We have created a large database with a number of ways to see some neat things about random processes."
  Consider now, for simplicity, the case of a fair coin. One could argue that to say that a coin is fair does indeed mean that it is equally likely (1/2) that head or tail will show up when I throw it. This is different from the more empirical approach described above, where I come to the (tentative) conclusion (1/2) only after throwing the coin many, many times. In the first case I have an a priori probability, while in the second case I have a frequency, or an estimate of such probability, a likelihood. Although, in the cases so far considered, the two seem to point to the same number, this is not necessarily the case in general. This is an important consideration. Here is why.
  In science we study new phenomena all the time. We set up an experiment, for example, which, before we do it, can have possibly several outcomes. It is in the nature of such experiments that most often we do not know all the possibilities, nor how likely each one is to actually happen. We do not know the a priori probabilities of the various outcomes. In fact, one of the purposes of the experiment is to establish at least the range of possible outcomes and their frequency.
  But, are we sure that if we run the experiment, say, 1000 times, all the possible outcomes will in fact happen? We can not be sure: for example, one possible outcome may be so unlikely that it might take 100,000 repetitions of the experiment before it shows up (of course, it is also possible for it to occur right away). Hence the need to repeat experiments, as many times as possible.
  Assuming now that we do not know all the possible outcomes, that we are ignorant about their respective probabilities. The most reasonable, initial assumption we can make is to declare our complete ignorance, and adopt what is called the Principle of Indifference: all outcomes are equally probable. In other words, we have no grounds to assume anything else. This is an example of the famous Occam's Razor: "Of two competing theories or explanations, all other things being equal, the simpler one is to be preferred." Or, quoting Newton, "We are to admit no more causes of natural things than such as are both true and sufficient to explain their appearances.." The Wikipedia article points out that "in modern usage, 'true' may best be taken to mean 'well established,' and 'simple' or 'simplicity' is used to mean 'fits in best with available facts and possibilities,with the least needed assumptions'."
  Suppose finally that we know the set of all possible outcomes, that each outcome is independent of the others (they are mutually exclusive), and that we know the probability P(x) of each possibile outcome x. For simplicity, we will assume the following basic properties of probabilities (or probability functions)
  1. The probability P(x) of any event x is a number between 0 (impossible) and 1 (certain)
  2. The probability that any one of the outcomes occurs, is 1 (certainty), or P(x or y or z or …) = 1
  3. The probability of either one or both of two, mutually exclusive events is the sum of the probabilities of each event separately, or P(x and y) = P(x) + P(y)

• You will remember the game of the 2 jars with red and blue marbles we played in Lecture 15. If I had simply told you that I had taken a sample of marbles from one of the two jars, without showing its composition to you, you should have said that the probability that such sample came from, say, jar A, would be 0.5 (since it could equally well come from jar B). However, when I showed you the sample, I clearly gave you a little extra information: the composition of the sample looked suspiciously identical to that of jar A (the same red-to-blue ratio). Shouldn't this extra information lead you to revise your estimated probability? For most of you it did, but only to a rather conservative extent.
  This is where Thomas Bayes (1702 - 1761) comes in. He discovered a very important principle, now called Bayes' Theorem, which allows one to revise one's a priori probabilities in light of new evidence. The probabilities so revised are called a posteriori or posterior probabilities. The new evidence can come under various guises: it could be new, independent evidence, or a new hypothesis, and so on. It is necessary to point out that, while everybody agrees that new evidence should result in a revision of one's probabilities, the particular method for doing so devised by Bayes has given rise to subtle disagreements, which however, we can not consider here. A simple but good introduction to Bayes' Theorem and the controversies arising from it, together with very interesting examples, is the article
  In Praise of Bayes, which first appeared on The Economist.
We will now return to the three-card game. Can you apply some of the above considerations to it? Here is the the answer: the chance that the non-showing side of the card is red is 2 to 1, or 0.666. What is interesting is that most people say 0.5. In other words, most people respond as if they had not been told anything about the cards, except that the back of card in question can be either red or white. In such case the correct answer would indeed be 0.5. Can you explain why the correct answer to the problem, as stated, is 0.666 or 2/3?
The theory of probability and, more generally, statistics, play a central role in our lives, even though we do not realize it. After all, there are very few things which are certain in our world (except of course death and taxes …). Most of our life, personally and collectively, depends on somehow assessing the relative likelihood of various events, on establishing priorities, on evaluating risks, and so on. We must do so in the absence of full information, in the real presence of various degrees of ignorance. A discussion of these topics lies outside the scope of this course, and we must resort to simpler, more idealized situations to glimpse the importance of the probability and statistics. Consider for example the famous Monty Hall or Let's Make a Deal problem. I suggest you visit two webpages where the problem is clearly stated:
The Let's Make a Deal Applet and   The Monty Hall Problem

"This paradox is related to a popular television show in the 1970's. In the show, a contestant was given a choice of three doors of which one contained a prize. The other two doors contained gag gifts like a chicken or a donkey. After the contestant chose an initial door, the host of the show then revealed an empty door among the two unchosen doors, and asked the contestant if he or she would like to switch to the other unchosen door. The question is: should the contestant switch? Do the odds of winning increase by switching to the remaining unopened door?

The intuition of most students tells them that each of the doors, the chosen door and the unchosen door, are equally likely to contain the prize, so that there is a 50-50 chance of winning with either selection. This, however, is not the case. The probability of winning by using the switching technique is 2/3 while the odds of winning by not switching is 1/3. The easiest way to explain this to students is as follows. The probability of picking the wrong door in the initial stage of the game is 2/3. If the contestant picks the wrong door initially, the host must reveal the remaining empty door in the second stage of the game. Thus, if the contestant switches after picking the wrong door initially, the contestant will win the prize. The probability of winning by switching reduces then to the probability of picking the wrong door in the initial stage which is clearly 2/3." [ from The Let's Make a Deal Applet ]

In even simpler terms: when I pick a door initially, the probability that there is a car behind it is 1/3. After the host opens one door, and I see a goat, if I don't switch, the probability that there is car behind the door I picked coninues to be 1/3. But if I switch, then the probability that I pick a car is 1/2, since one of the two doors not yet open hides a goat and the other a car.
As the Wikipedia article points out, two assumptions must be kept in mind, although they "are rarely made explicit":
1. Monty always opens a door
2. There is always a goat behind the door Monty opens
Go back for a moment to In Praise of Bayes, and consider what the author says about the problems of patients in clinical trials. This is no game, yet the issues raised by the Monty Hall game are much the same. You can also visit Cleaner Power Causes Cancer Problem, where the author, after stating that "people seem to have very poor intuition about probability. It can take a lot of training to learn how to calculate probabilities correctly," proceeds to present yet another version of the Monty Hall game and reporting on a paradox that I hope we can discuss in class.
Finally, another, and much more famous problem is the The Prisoners' Dilemma. Once again it is important to realize that the issues raised by this seemingly academic example are in fact the same issues encountered in many daily situations, and not just personal ones.

 
Readings, Resources and Questions

• A good (PowerPoint) presentation on probabilities is D Lin's Uncertainty and Probability Theory.
  A very important work on the history of probability and statistics is Ian Hacking, The Emergence of Probability, Cambridge U Press 1975.
• The September 25 - October 1, 2004 issue of New Scientist features a series of simple, clear articles on randonmness, order, probabilities, chance, etc. Go to the Library.

  "Whether you flip a coin or roll the dice, the outcome is utterly unpredictable. Or so we like to think. We rely on randomness for cryptography, engineering, physics—and to explain the workings of some ecosystems. But is it quite what it seems? In this 10-page investigation, we take a closer look at random events and the part they play in our world, from quantum theory to coincidence. Ian Stewart kicks things off with a provocative question: is randomness anything more than an invention of our superstitious minds?"

• "Innumeracy—the mathematical counterpart of illiteracy—is a disease that has ravaged our technological society. To cambat this terrible disease, John Allen Paulos has conconcted the perfect vaccine: this book, which is in many ways better than an entire high school math education! Our society would be unimaginably different if the average person truly understood the ideas of this marvelous and important little book." So writes Douglad Hofstadter on the back cover of John Allen Paulos, Innumeracy. Mathematical Illiteracy and Its Consequences, Vintage Books, 1990.
  You may also want to explore the Chance! project.

  "'What's the chance of that!?' It is a question that almost all people ask—sometimes after the fact—in trying to make sense of a seemingly improbable event and, at other times, in preparation for action, as an attempt to foresee and plan for all the possibilities that lie ahead. In either case, it is mathematics in general, and probability and statistics in particular, that the public looks to for a final answer to this question. One out of one hundred, 4 to 1 odds, an expected lifetime of 75 years—these are the sorts of answers people want. When used honestly and correctly, numbers can help clarify the essence of a confusing situation by decoupling it from prejudicial assumptions or emotional conclusions. When used incorrectly—or even worse, deceitfully—they can lend a false sense of scientific objectivity to an assertion, misleading those who are not careful enough or knowledgeable enough to look into the reasoning underlying the numerical conclusions."

  On the Chance! website you can also find an excellent, though not elementary, book, Introduction to Probability, by C M Grinstead and J L Snell (2003), which you can download for free [ click on "here" in the first paragraph ].
• A very readable survey of cognitive illusions and fallacies is Massimo Piattelli Palmarini, Inevitable Illusions: How Mistakes of Reason Rule Our Minds, J Wiley & Sons, NY 1994. The original Italian title is even more interesting; literally translated, it reads: "The Illusion of Knowing: What Hides Behind Our Mistakes."
• There is yet another area where dangerous fallacies lurk: logic. These fallacies are called logical fallacies. Here is a solid, yet simple, book that I urge you to read, better sooner than later: S Morris Engel, With Good Reason. An Introduction to Informal Fallacies. Bedford/St Martin's, Boston, New York, 2000.

  "Through the mass media particularly, we are bombarded with appeals to buy this product or that one, to believe this speaker or that one, to take this action or that one [ … ] We are often aware of something illogical in an attempt to persuade us, but we find it hard to fight the attempt because we aren't sure why the argument's logic is faulty or what in particular is wrong with it
  [ … ] Accordingly, logic may be regarded as among the most powerful studies one can undertake, particularly in an age like ours that is so full of impressive claims and counterclaims."