Probability

Written and reviewed by Devendra Dubey, Mathematics faculty at UniDrill (M.Sc. Mathematics, Osmania University · 10 years teaching Mathematics). Based on NCERT Mathematics Class XII, Chapter 13.

A family has two children, and one is a boy. Are both boys?

Most people say one half. The correct answer is one third. List the equally likely families by birth order: BB, BG, GB, GG. The statement "at least one is a boy" removes GG, leaving three equally likely families: BB, BG, GB. Only one of them is BB, so the chance is $\dfrac{1}{3}$.

Nothing about the children changed. Your information changed, and that shrank the list of possibilities. This chapter is the mathematics of that shrinking. Conditional probability measures chances inside the smaller world. The multiplication rule chains such steps together. Independence asks when new information changes nothing at all. Total probability and Bayes' theorem handle the case where a result could have come from several different causes. For the basics (sample space, events, addition rule) see Probability in Class XI.

Conditional probability

The probability of $E$ given that $F$ has already occurred, with $P(F) \neq 0$, is

$$P(E \mid F) = \dfrac{P(E \cap F)}{P(F)}$$

Read the formula as a fraction of a fraction. Once $F$ has happened, $F$ becomes the whole world. Inside it, the part where $E$ also happens is $E \cap F$. The denominator $P(F)$ rescales the world so that it again has total probability 1. That is why $P(F \mid F) = 1$.

Three properties follow because conditional probability behaves like any probability on the smaller sample space.

Two more facts to carry. If $P(F) = 0$ then $P(E \mid F)$ is not defined, since you would divide by zero. And $P(E \mid F)$ is not the same as $P(F \mid E)$, because the two have different denominators.

Worked example: shrinking the sample space. Two dice are thrown. Given that the numbers are different, what is the probability that the sum is 6?

Common wrong step. Dividing by 36, or counting $(3,3)$. The word "given" means the denominator is the condition, not the whole sample space. By the formula: $P(\text{sum } 6 \cap \text{different}) = \dfrac{4}{36}$ and $P(\text{different}) = \dfrac{30}{36}$, and the ratio is again $\dfrac{4}{30}$.

Pause and predict: $P(E) = 0.6$, $P(F) = 0.3$ and $P(E \cap F) = 0.2$. Which is larger, $P(E \mid F)$ or $P(F \mid E)$?

Check your answer: $P(E \mid F) = \dfrac{0.2}{0.3} = \dfrac{2}{3}$ and $P(F \mid E) = \dfrac{0.2}{0.6} = \dfrac{1}{3}$. The numerator is the same. The conditional probability is larger when the condition is the rarer event, because it gives a smaller world to divide by. They are equal only when $P(E) = P(F)$.

The multiplication rule: building chains

Rearranging the definition gives the rule for "and":

$$P(E \cap F) = P(E)\,P(F \mid E) = P(F)\,P(E \mid F)$$

In words: the chance that both happen is the chance that the first happens, times the chance that the second follows given the first. For three events it extends to $P(E \cap F \cap G) = P(E)\,P(F \mid E)\,P(G \mid E \cap F)$, and each further event adds one more factor. In a tree diagram you multiply along the branches to get the probability of one path, and add the paths that lead to the same event.

The key question in every sequence problem is whether the probabilities change after each step. Drawing without replacement changes them. Drawing with replacement does not.

Worked example: two kings. Two cards are drawn one after the other from a well shuffled pack of 52, without replacement. Find the probability that both are kings.

Common wrong step. Using $\dfrac{4}{52}$ again for the second card. That treats the draws as independent and corresponds to putting the first king back.

Pause and predict: three cards are drawn without replacement. What is the probability that all three are aces?

Check your answer: $\dfrac{4}{52} \times \dfrac{3}{51} \times \dfrac{2}{50} = \dfrac{24}{132600} = \dfrac{1}{5525}$. Each factor is smaller than the last, because both the number of aces and the number of cards fall by one.

Independent events

Two events are independent if knowing that one happened does not change the chance of the other. Equivalently, the multiplication rule loses its conditional factor:

$$P(E \cap F) = P(E)\,P(F), \qquad \text{equivalently } P(E \mid F) = P(E)$$

To test independence, compute $P(E)$, $P(F)$ and $P(E \cap F)$ separately and check whether the product equals the intersection. Do not decide by feel.

Complements. If $E$ and $F$ are independent, so are $E$ and $F'$, and so are $E'$ and $F'$. The proof takes one line: $P(E \cap F') = P(E) - P(E \cap F) = P(E) - P(E)P(F) = P(E)\,(1 - P(F)) = P(E)\,P(F')$. This is the reason "at least one" problems for independent events are solved as $1 - P(\text{none})$, with $P(\text{none})$ a product of the complements.

The most-tested confusion in the chapter is between independent and mutually exclusive:

The point of the whole comparison is this: with non-zero probabilities, a pair of events cannot be both. If $P(E), P(F) > 0$ and the events are mutually exclusive, then $P(E \cap F) = 0$, which is not $P(E)P(F) > 0$, so they cannot be independent. Mutually exclusive events are, in a sense, as dependent as events get.

Worked example: testing independence. A die is thrown. $E$ is "the number is a multiple of 3" and $F$ is "the number is even". Are $E$ and $F$ independent?

Common wrong step. Saying the events are dependent because they share the outcome 6. Overlapping does not mean dependent. Independence is about whether the numbers multiply.

Worked example: at least one. Two students attempt a problem independently. Their chances of solving it are $\dfrac{1}{2}$ and $\dfrac{1}{3}$. What is the probability that the problem is solved?

Common wrong step. Adding: $\dfrac{1}{2} + \dfrac{1}{3} = \dfrac{5}{6}$. Addition without subtracting the overlap counts the "both solve it" case twice. The correct addition rule is $\dfrac{1}{2} + \dfrac{1}{3} - \dfrac{1}{6} = \dfrac{2}{3}$.

Pause and predict: can two events, each with non-zero probability, be both mutually exclusive and independent?

Check your answer: No. Mutual exclusivity forces $P(E \cap F) = 0$, while independence needs $P(E \cap F) = P(E)P(F)$, a product of two positive numbers. The two conditions contradict each other.

Beyond the textbook. For the exam: three events $A, B, C$ are mutually independent only if all three pairwise products and the triple product hold, four conditions in all. Pairwise independence alone is not enough. In reality: the standard example uses two fair coins. Let $A$ be "first toss is heads", $B$ be "second toss is heads" and $C$ be "both tosses agree". Each has probability $\dfrac{1}{2}$, and each pair has intersection $\{HH\}$ with probability $\dfrac{1}{4} = \dfrac{1}{2} \times \dfrac{1}{2}$, so the events are pairwise independent. But $A \cap B \cap C = \{HH\}$ has probability $\dfrac{1}{4}$, not $\dfrac{1}{8}$. Knowing two of the events tells you everything about the third.

Partition and total probability

Sometimes an event can arise through several different routes. A partition of the sample space is a family of events $E_1, E_2, \ldots, E_n$ that are pairwise disjoint, exhaustive (together they make up the whole sample space) and each have non-zero probability. Think of cutting a cake into slices that do not overlap and leave no crumbs.

Take any event $A$. It is split by the slices into pieces $A \cap E_1, A \cap E_2, \ldots$, which do not overlap. Adding their probabilities and applying the multiplication rule to each piece gives the theorem of total probability:

$$P(A) = \sum_{j=1}^{n} P(E_j)\,P(A \mid E_j)$$

It is a weighted average: the chance of $A$ under each scenario, weighted by how likely that scenario is.

Worked example: a project on time. The probability of a strike is $0.65$. If there is a strike, a project finishes on time with probability $0.32$. If there is no strike, the probability is $0.80$. What is the probability that the project finishes on time?

Common wrong step. Averaging the two conditional probabilities, $\dfrac{0.32 + 0.80}{2} = 0.56$. The two scenarios are not equally likely, so each must be weighted by its own probability.

Bayes' theorem: reversing the condition

Total probability moves forward: given the cause, find the chance of the effect. Bayes' theorem moves backward: given that the effect happened, find the chance that a particular cause produced it. The proof is two lines. By the multiplication rule $P(E_i \cap A) = P(E_i)P(A \mid E_i) = P(A)P(E_i \mid A)$. Divide by $P(A)$ and replace $P(A)$ by the total probability formula:

$$P(E_i \mid A) = \dfrac{P(E_i)\,P(A \mid E_i)}{\sum_{j=1}^{n} P(E_j)\,P(A \mid E_j)}$$

The $E_i$ are called hypotheses. $P(E_i)$ is the prior probability, what you believed before seeing $A$. $P(A \mid E_i)$ is the likelihood. $P(E_i \mid A)$ is the posterior, what you believe after. The denominator is just $P(A)$. Every Bayes problem fits one four column table.

The posterior of each hypothesis is its last column entry divided by the total of that column.

Worked example: which machine? Machines $A$, $B$, $C$ produce $25\%$, $35\%$ and $40\%$ of the bolts in a factory. Of their output, $5\%$, $4\%$ and $2\%$ respectively are defective. A bolt is chosen at random and found defective. What is the probability that it came from machine $B$?

The posterior for $B$ is $\dfrac{0.0140}{0.0345} = \dfrac{140}{345} = \dfrac{28}{69}$.

Common wrong step. Stopping at the prior $0.35$, or at the numerator $0.0140$. The prior is the answer before you know the bolt is defective. The numerator is not a probability of $B$ given defective until you divide by the total $P(\text{defective})$.

Pause and predict: a box holds three coins: one two headed, one biased with $P(\text{head}) = \dfrac{3}{4}$, and one fair. A coin is chosen at random and tossed; it shows a head. Is the probability that it is the two headed coin more than, less than or equal to $\dfrac{1}{3}$?

Check your answer: More than $\dfrac{1}{3}$. A head is more likely under the two headed coin, so seeing one raises its probability. The numbers: $P(\text{head}) = \dfrac{1}{3}\left(1 + \dfrac{3}{4} + \dfrac{1}{2}\right) = \dfrac{3}{4}$ and $P(\text{two headed} \mid \text{head}) = \dfrac{1/3}{3/4} = \dfrac{4}{9}$.

Beyond the textbook. For the exam: apply Bayes' formula and give the posterior. In reality: the posterior depends heavily on the prior, which is why a positive medical test is often less alarming than it sounds. Suppose $1\%$ of people have a disease, the test catches $99\%$ of cases, and it wrongly flags $5\%$ of healthy people. Out of 10,000 people, 100 are ill and 99 of them test positive. Of the 9,900 healthy people, $5\%$, which is 495, also test positive. So $P(\text{ill} \mid \text{positive}) = \dfrac{99}{99 + 495} = \dfrac{1}{6}$, about $0.167$, even though the test is $99\%$ sensitive. A rare condition produces many false alarms compared with true cases, which is why doctors confirm positives with a second test.

Beyond the textbook. For the exam: prior, likelihood and posterior are three names to match. In reality: the same machinery lets software learn from evidence. A spam filter starts with a prior chance that an email is spam and uses Bayes' theorem to update it as it sees words such as "winner" or "prize". The idea that a belief should be updated by multiplying by the likelihood of the evidence is the foundation of statistical inference.

Random variables in brief

A random variable $X$ is a real valued function defined on the sample space: it attaches a number to every outcome. Toss a fair coin twice, so the sample space is $\{HH, HT, TH, TT\}$, and let $X$ be the number of heads. Then $X(HH) = 2$, $X(HT) = X(TH) = 1$ and $X(TT) = 0$, and the probabilities of its values are

The probabilities add to 1, because every outcome gives exactly one value of $X$. Every later idea about distributions starts from a table like this one.

Which tool when?

Exam traps

Formula sheet

60-second revision

Where this chapter connects

• Before: Probability in Class XI defines events, the addition rule and equally likely outcomes, which every formula here builds on.
• Counting: Permutations and Combinations supplies the counting behind "number of favourable outcomes" in card and ball problems.
• Functions: Relations and Functions explains what it means for a random variable to be a function from the sample space to the real numbers.