1,10-11. The Monty Hall 3-door puzzle

Monty Hall 3-door Puzzle

You have a chance to win a large prize in a game show hosted by Monty. The prize is behind one of 3 closed doors, and the other two doors are losers, with a goat behind each.

• You are asked to select (but not open) one of the doors.
• Then Monty, opens one of the doors he knows is a losing door, selecting randomly if both remaining doors are losers.
• Then asks you if you want to switch doors.

Which strategy is the best?

• Switch doors.
• Stick to original selection.

title: Not in exam!

You selected a door, let’s say ${\text{door}}_1$. Then Monty opened another door:

As $\frac{1}{3}+\frac{1}{3}=\frac{2}{3}>\frac{1}{3}=\frac{1}{6}+\frac{1}{6}$, hence you have a better chance switching from ${\text{door}}_1$ to the door Monty did not open.

Using Bayes’ Theorem

• You select ${\text{door}}_1$.
• Monty opens another, ${\text{door}}_2$.
• We label the remaining door as ${\text{door}}_3$.

Consider the following:

• $W_1:{\text{door}}_1\text{ is the winning door}$
• $W_2:{\text{door}}_2\text{ is the winning door}$
• $W_3:{\text{door}}_3\text{ is the winning door}$
• $M_2:\text{Monty opened }{\text{door}}_2$

We want to know, $\boxed{p({\bar{W}}_1|M_2)}$:

• If $p({\bar{W}}_1|M_2)>\frac{1}{2}$, we should switch doors.
• If $p({\bar{W}}_1|M_2)<\frac{1}{2}$, we should stick to our door.
• If $p({\bar{W}}_1|M_2)=\frac{1}{2}$, it doesn’t matter.

We need to know:

• $p(W_1)=p(W_2)=p(W_3)=\frac{1}{3}$
• $p({\bar{W}}_1)=\frac{2}{3}$
• $p(M_2|W_1)=\frac{1}{2}$ If your selected door is the winning one, then Monty randomly opens one of the other two doors. We also know that $p(M_2|W_3)=1$ because if ${\text{door}}_3$ is the winning door and you selected the first one, then Monty must open the second. (as he never opens the winning door) $$\begin{array}{rl}p(M_2|W_1)& =\frac{p(M_2\cap {\bar{W}}_1)}{p({\bar{W}}_1)}\\ & =\frac{p(M_2\cap (W_2\cup W_3))}{\frac{2}{3}}\\ & =\frac{p((M_2\cap W_2)\cup (M_2\cap W_3))}{\frac{2}{3}}\\ & =\frac{p(M_2\cap W_3)}{\frac{2}{3}}\\ & =\frac{p(M_2|W_3)\cdot p(W_3)}{\frac{2}{3}}\\ & =\frac{1\cdot \frac{1}{3}}{\frac{2}{3}}\\ & =\frac{1}{2}\end{array}$$
• Hence, $p(M_2|{\bar{W}}_1)=\frac{1}{2}$

We can now apply Bayes’ theorem:

$$\begin{array}{rl}p({\bar{W}}_1|M_2)& =\frac{p(M_2|{\bar{W}}_1)\cdot p({\bar{W}}_1)}{p(M_2|{\bar{W}}_1)\cdot p({\bar{W}}_1)+p(M_2|W_1)\cdot p(W_1)}\\ & =\frac{\frac{1}{2}\cdot \frac{2}{3}}{\frac{1}{2}\cdot \frac{2}{3}+\frac{1}{2}\cdot \frac{1}{3}}\\ & =\frac{\frac{2}{6}}{\frac{3}{6}}\\ & =\frac{2}{3}\end{array}$$

As $p({\bar{W}}_1|M_2)=\frac{2}{3}>\frac{1}{2}$, you have a better chance switching.

Link to original

2. Probability distributions

Probability distributions

Experiment

An experiment is a procedure that yields one of a given set of possible outcomes.

Link to original

Sample space

The sample space, $S/\Omega$, of the experiment is the set of all possible outcomes $$ S/\Omega = \{ s_1, s_2, ..., s_n \} $$

Link to original

Probability distribution

A probability distribution in the sample space $S$ is a function $p$ assigning a number $p(s_i)$ to each possible outcome $s_i$ in $S$ (the probability of $s_i$), such that:

• values are exhaustive (cover everything)
• values are mutually exclusive (no overlap)
• $0\le p(s_i)\le 1$, for all $i=1\text{ to }n$
• $p(s_1)+p(s_2)+...+p(s_n)=1$ All probability values add up to $1$.

$$P(\text{Weather})=\left(\begin{array}{c}0.72\\ 0.1\\ 0.08\\ 0.1\end{array}\right)=⟨0.72,0.1,0.08,0.1⟩$$

This means, for example: $P(\text{Weather}=sunny)=0.72$.

Link to original

Uniform distribution

A uniform distribution is where the probabilities are evenly distributed across the sample space. $$ p(s_i) = \frac{1}{n} \text{ for all } i = 1 \text{ to } n $$

title: Example
Given the uniform distribution on $S$.
If a fair dice is rolled, then there are six possible equally likely outcomes: $1,2,3,4,5,6$.
So $p(1) = p(2) = ... = p(6) = \frac{1}{6}$.

Link to original

Link to original

3. Probability of events

Probability of events

Event

An event is a subset of the [[Sample space|sample space]], or otherwise a set of possible outcomes.

$$P(A)=\sum _{\{\omega \in A\}}P(\omega )$$ Link to original

Probability of an event

The probability of an event $E$ is the sum of the probabilities of the outcomes in $E$.

title: Example 1
If two fair dice are rolled, what is the probability that the sum is $7$?
 
By product rule, there are $6 \cdot 6 = 36$ possible outcomes, each of them equally likely with a probability of $\frac{1}{36}$.
 
There are $6$ successful outcomes: $(1,6), (2,5), (3,4), (4,3), (5,2), (6,1)$.
 
Hence probability is: $6 \cdot \frac{1}{36} = \boxed{\frac{1}{6}}$

title: Example 2
Given that a dice is weighed in such a way that $3$ appaers twice as often as each of the other numbers but the others are equally likely. What is the probability we roll an odd number?
 
We need to find $p(E)$ of the event $E = \{ 1, 3, 5 \}$.
 
We know that:
- $\sum^n_1 p(n) = 1$
- $p(1) = p(2) = p(4) = p(5) = p(6)$
- $p(3) = 2p(1)$
 
Hence $p(1) = p(2) = p(4) = p(5) = p(6) = \frac{1}{7}$ and $p(3) = \frac{2}{7}$.
 
So, $p(E) = p(1) + p(3) + p(5) = \frac{4}{7}$.

title: Example 3
Take for example a lottery, where we can win if we correctly choose a set of $6$ numbers out of the first $59$ positive natural numbers.
 
The total number of ways to choose $6$ numbers out of $59$ is found using [[Order does not matter, repetition not allowed|order does not matter, repetition not allowed]] combinations:
 
$$
	\begin{pmatrix} 59 \\ 6 \end{pmatrix}
	= \frac{59 \cdot 58 \cdot 57 \cdot 56 \cdot 55 \cdot 54}
	{1 \cdot 2 \cdot 3 \cdot 4 \cdot 5 \cdot 6}
	= 45, 057, 474
$$
 
Hence we find that:
 
$$
	p(W) = \frac{1}{45, 057, 474}
$$

Link to original

Link to original

4. Complement and union of events

Complementary events

Complementary event

Let $E$ be an event in a sample space $S$. The probability of event $\bar{E}$, or the complement of $E$ in $S$ is the sum of the probabilities of the outcomes not in $E$: $$ \boxed{ p(\bar E) = 1 - p(E) } $$

title: Example
We flip a fair coin $10$ times, what is the probability that heads comes up at least once?
 
Let $E$ be the event that heads comes up at least once of the $10$ flips.
Let $\bar E$ be the event that each of the $10$ times, tails comes up.
 
$$
	\begin{aligned}
		p(E) &= 1 - p(\bar E) \\
		&= 1 - \frac{1}{2^{10}} \\
		&= 1 - \frac{1}{1024} \\
		&= \frac{1023}{1024}
	\end{aligned}
$$

Link to original

Union of events

Union of events

The probability of the union of events is given by: $$ p(E_1 \cup E_2) = p(E_1) + p(E_2) - p(E_1 \cap E_2) $$

title: Example
What is the probability that a number randomly selected from $X = \{ n \in \mathbb N^+ | n \le 100 \}$ is divisible by either $2$ or $5$ or both (by $10$)?
 
- Let $E_1$ be the event that the selected number in $X$ is divisible by $2$.
- Let $E_2$ be the event that the selected number in $X$ is divisible by $5$.
- Then $E_1 \cup E_2$ is the event that it is divisible by both.
 
We are given that $|X| = 100, |E_1| = 50, |E_2| = 20, |E_1 \cap E_2| = 10$.
 
$$
	\begin{aligned}
		p(E_1 \cup E_2) &= \frac{50}{100} + \frac{20}{100} - \frac{10}{100} \\
		&= \frac{60}{100} \\
		&= \frac{3}{5} \\
		&= 0.6
	\end{aligned}
$$

Link to original

Link to original

5. Conditional probability and independence

Conditional probability

Conditional probability

The conditional probability of event $a$ given another event $b$ occurs is $$ P(a\vert b) = \frac{p(a \land b)}{p(b)} \text{ if } p(b) > 0$$

Link to original

title: Example 1
Flip a coin $3$ times. We already know how to find the probabiltiy of event $E$ that at least $2$ heads come up in a row.
 
$$
	\begin{aligned}
		S \text{ contains } 2^3 &= 8 \text{ outcomes} \\
		p(E) &= \frac{3}{8}
	\end{aligned}
$$
 
But what if the first flip comes up heads and we are asked about the probability of $E$ given $F$ has already occurred?
 
Our sample space is now $HHT, HTH, HTT, HHH$.
For an outcome in $E$ to occur, it must belong to $E \cap F$, so either $HHT$ or $HHH$.
 
So we have $p(F) = \frac{4}{8} = \frac{1}{2}$ and $p(E \cap F) = \frac{2}{8} = \frac{1}{4}$.
 
$$
	\begin{aligned}
		p(E | F) &= \frac{\frac{1}{4}}{\frac{1}{2}} \\
		&= \boxed{\frac{1}{2}}
	\end{aligned}
$$

title: Example 2
Throwing a pair of fair dice, what is the probability at least one dice is a $3$ given the sum of the two dice is $5$?
 
- Let $A$ be the event that at least one dice is a $3$. ($11$ outcomes)
- Let $B$ be the event that the sum of two dice is $5$.
  $B$ consists of the outcomes $(1,4), (2,3), (3,2), (4,1)$
- The sample space consists of $6^2 = 36$ equally probable outcomes.
  $p(B) = \frac{4}{36} = \frac{1}{9}$
- The event $A \cap B$ consists of two outcomes: $(2,3)$ and $(3,2)$.
  Hence $p(A \cap B) = \frac{2}{36} = \frac{1}{18}$
  
$$
	\begin{aligned}
		\therefore p(A|B) &= \frac{p(A \cap B)}{p(B)} \\
		&= \frac{\frac{1}{18}}{\frac{1}{9}} \\
		&= \frac{1}{2}
	\end{aligned}
$$

Independence

Independence

Two events $E$ and $F$ are said to be independent if the occurrence of one of the events gives no information about the probability of the other event. $$ \boxed{p(E | F) = p(E)} \text{ or } \boxed{p(F | E) = p(F)} $$

Using conditional probability, we can obtain that $E$ and $F$ are independent events whenever $p(E\cap F)=$ $p(E) \cdot p(F)$.

Link to original

title: Example 3
We flip a fair coin $3$ times. What is the probability an odd number of tails appear given that the first flip comes up tails?
 
- Let $X$ be the event that an odd number of tails appear.
- Let $Y$ be the event that the first flip comes up tails.
- The sample space consists of $2^3 = 8$ outcomes so:
  $p(Y) = \frac{4}{8} = \frac{1}{2}$
- The event $X \cap Y$ consists of two outcomes: $THH$ and $TTT$.
  $p(X \cap Y) = \frac{2}{8} = \frac{1}{4}$
 
$$
	\begin{aligned}
		p(X | Y) &= \frac{p(X \cap Y)}{p(Y)} \\
		&= \frac{\frac{1}{4}}{\frac{1}{2}} \\
		&= \frac{1}{2}
	\end{aligned}
$$
 
We can also find the unconditional probability of $X$, giving $p(X) = \frac{4}{8} = \frac{1}{2}$.
As $p(X | Y) = p(X)$, then the two events are **independent**.

Link to original