Chapter 11
Probability Distributions
Random variables, distributions, expectation and special discrete distributions.
1. Random variables
The outcome of a random experiment is not always a number. Toss a coin and you get or ; draw balls from an urn and you get colours. To do arithmetic — averaging, spreading, computing probabilities — we attach a real number to each outcome. That rule is a random variable.
A random variable is a function that assigns a real number to each outcome of the sample space , in such a way that the inverse image of any interval is an event whose probability is defined.
We write random variables with capitals () and their possible values with small letters (). The set of all values can take is its range.
Random variables come in two flavours. A discrete random variable takes isolated, countable values such as (things you count). A continuous random variable can take any value in an interval of the real line (things you measure).
- Discrete (count): number of heads, number of defective items, score on two dice — you can list the values.
- Continuous (measure): height, weight, waiting time, bulb lifetime — the values fill an interval and cannot be listed one by one.
2. Discrete distributions and the pmf
For a discrete random variable we describe its behaviour by listing every value together with its probability. That list is the probability distribution, and the function giving those probabilities is the probability mass function (pmf).
For a discrete random variable , the pmf is — the probability that equals the value .
A list of numbers is a valid pmf only if it passes two tests:
3. Cumulative distribution function (discrete)
Often we want the probability of being at or below a value rather than exactly equal to it. The cumulative distribution function (cdf) accumulates probability from the left.
The cdf is — the total probability of all values up to and including .
- Discrete: — add the probabilities of all values .
- is a non-decreasing step function; it jumps by at each value .
- and .
- Interval probability by subtraction: .
4. Continuous distributions and the pdf
For a continuous random variable the probability of hitting one exact value is 0 — there are infinitely many possibilities crammed into every interval. Instead we use a probability density function and read probability as area under its curve.
A function is a pdf for a continuous random variable if everywhere and the total area under it is .
Probability over an interval is the area under the curve on that interval:
5. Cumulative distribution function (continuous)
The cdf is defined the same way for a continuous variable, , but now it is an area — the integral of the density from the far left up to .
- Given the pdf, integrate to get the cdf.
- Given the cdf, differentiate to get the pdf: the density is the slope of the cdf.
- is continuous and non-decreasing, with , .
- Interval probability: .
6. Mathematical expectation (mean)
The expectation (mean) is the long-run average value of the random variable — a probability-weighted balance point. It generalises the ordinary average: instead of dividing by , we weight each value by its probability.
More generally, the expected value of any function weights by the probabilities. Taking gives the second moment , which we need for the variance:
7. Variance and standard deviation
The mean tells you where a distribution sits; the variance tells you how widely it spreads around that mean. A small variance means values cluster tightly about the mean; a large variance means they scatter.
A useful shortcut: for constants , (adding a constant shifts but does not spread).
8. Mean and variance by integration (continuous)
For a continuous random variable the recipe is identical, with integrals replacing sums. Find and by integrating and , then use the same variance formula.
9. Bernoulli and binomial distributions
A Bernoulli trial is a single experiment with just two outcomes — success (prob ) or failure (prob ). Scoring for success and for failure gives the Bernoulli distribution, with mean and variance .
Repeat a Bernoulli trial independent times and count the successes: that count follows a binomial distribution, written .
- A fixed number of trials, with finite.
- Each trial has exactly two outcomes: success or failure.
- The success probability is the same on every trial.
- The trials are independent of one another.
If is the number of successes in trials, then for :
Here counts the ways to place successes among trials, is the chance those succeed, and the chance the rest fail. The mean and variance have neat closed forms:
10. Choosing the right model (and other distributions)
The binomial is the central discrete model for Class 12, but it is one of a family. When trials are rare but numerous (large , small , with moderate), the Poisson distribution approximates the binomial. When measurements cluster symmetrically about a mean, the continuous normal distribution — the bell curve — takes over.