Probability Theory · Jiayu Chen

Prerequisite path Preparation & skip guidance · Calculus

Studying Probability Theory

Probability has two entry points here. Grinstead and Snell build intuition through distributions, conditioning, expectation and finite chains using calculus. Lawler compresses the foundations into the language of measurable spaces and integration. The latter is a bridge from graduate real analysis, so its brevity should not be mistaken for an easier first course. Choose the elementary route if conditional probability still feels like a formula detached from a sample space.

What to work on

Compute conditional probabilities and distinguish independence from disjointness.
Use expectations, variances and indicator variables.
State the weak law and central limit theorem with appropriate assumptions.
Find a finite Markov chain's stationary distribution and distinguish stationarity from convergence.

Conditioning renormalizes the sample space

Let

A

be “first toss heads” and

B

be “at least one heads.” Fair independent tosses make the four ordered outcomes equiprobable. Conditioning on

B

removes

TT

and divides each remaining mass by

3/4

. Thus

\mathbb P(A\mid B)=(1/2)/(3/4)=2/3

, while

\mathbb P(B\mid A)=1

. Although the tosses are independent, these events are dependent:

\mathbb P(A\cap B)=1/2\ne3/8=\mathbb P(A)\mathbb P(B)

Enlarge: Conditioning renormalizes the sample space

A starting text

Grinstead and Snell's Introduction to Probability, CHANCE version dated July 4, 2006 — Charles M. Grinstead and J. Laurie Snell; CHANCE version by Peter G. Doyle. Review 1.2 and 2.2 for distributions and densities; 4.1-4.2 for conditioning; 6.1-6.3 for expectation and variance; 8.1 and 9.1-9.2 for laws of large numbers and central limits; then 11.1 and 11.3-11.4 for finite Markov chains.

Choosing what to read

Describe the experiment and the events before doing arithmetic. For a Bernoulli sample mean, calculate its expectation and variance, then use the variance bound to explain convergence in probability. Later, translate those same objects into random variables on a probability space. Keep almost sure convergence separate from convergence in probability, and test a two-state periodic chain before assuming that every stationary distribution describes a limiting distribution.

Grinstead and Snell's Introduction to Probability, CHANCE version dated July 4, 2006

Charles M. Grinstead and J. Laurie Snell; CHANCE version by Peter G. Doyle

Learn conditioning and expectation before elementary laws of large numbers.

How it reads: Calculus-based examples, computation and simulation.
What it expects: Calculus including multiple integrals; matrix basics for Markov chains.
What to keep in mind: The linked text is Doyle's 2006 CHANCE adaptation; its numbering is not guaranteed to match every 2003 answer.
Practice: Exercises after 4.1, 6.1-6.2, 8.1 and 11.3. The source page links public odd-numbered answers; complete solutions are offered to instructors by request.

Complete Dartmouth PDF; the public answer file is separate from the restricted instructor solution set.

Edition, reading range and source

CHANCE July 4, 2006: 4.1-4.2, 6.1-6.3, 8.1, 9.1-9.2, then 11.1 and 11.3-11.4.

Source · Checked 2026-10-17

Notes on Probability

Gregory F. Lawler

Recast elementary random variables and independence in a measure-theoretic framework.

How it reads: Concise foundations expressed through measure and integration.
What it expects: Measure spaces, Lebesgue integration and modes of convergence; Fubini for section 3.
What to keep in mind: Assumes graduate real analysis; later Fourier arguments need additional preparation.
Practice: Exercises 3.1-3.2 and 3.6; then Exercise 4.1 on convergence in probability versus almost sure convergence. Public worked solutions not verified.

Complete 39-page author PDF on the University of Chicago site, dated March 21, 2016.

Edition, reading range and source

Sections 1-3, PDF pp. 1-12, from probability spaces through independence; then Exercise 4.1 on PDF p. 13.

Source · Checked 2026-10-17

Practice

Probability exercises and odd-numbered answers. Use exercise sets after sections 4.1, 6.1-6.2, 8.1 and 11.3. On the source page follow Answers and Errata to the odd-numbered answer PDF; match section and exercise labels.. The authors' public page links answers to odd-numbered exercises; that PDF was opened and sampled. The complete solution set is offered to instructors by request and is not described as publicly available.

Try these before moving on

Try a proof or calculation before opening the hint. These solutions address the checks below, not the exercises in the linked textbooks.

For independent Bernoulli(p) variables X1,...,Xn with 0<p<1, compute the expectation and variance of their average and prove convergence in probability to p using Chebyshev's inequality.
A hint
Compute one variable's first two moments. Linearity of expectation does not require independence; adding variances does here.
Work through the solution
Let $(X_i)_{i\ge1}$ be independent Bernoulli variables with the same parameter $p\in(0,1)$ on a probability space, and set $\bar X_n=n^{-1}\sum_{i=1}^nX_i$ . Since $X_i^2=X_i$ , $\mathbb EX_i=p$ and $\operatorname{Var}(X_i)=p-p^2=p(1-p)$ . Linearity gives $\mathbb E\bar X_n=p$ . Independence makes all cross covariances zero, so

$\operatorname{Var}(\bar X_n)=\frac{1}{n^2}\sum_{i=1}^n p(1-p)=\frac{p(1-p)}n.$

For every fixed $\varepsilon>0$ , Chebyshev's inequality yields $\mathbb P(|\bar X_n-p|\ge\varepsilon)\le p(1-p)/(n\varepsilon^2)\to0$ . This is convergence in probability to $p$ . The expectation identity remains true for dependent variables with these marginals, but the displayed variance computation need not; independence was used at that step.

If this was the sticking point:
- Calculus — Review finite sums and scaling of quadratic expressions.
- Real Analysis — Review the quantifiers in convergence and limits of bounds.
Toss a fair coin twice independently. Let A mean the first toss is heads and B mean at least one toss is heads. Compute P(A), P(B), P(A given B) and P(B given A), and decide whether A and B are independent or disjoint.
A hint
Use the four ordered outcomes HH, HT, TH and TT, each with probability 1/4. Conditioning changes the denominator.
Work through the solution
The sample space $\Omega=\{HH,HT,TH,TT\}$ has four equally likely outcomes by fairness and independence of the tosses. Here $A=\{HH,HT\}$ , $B=\{HH,HT,TH\}$ and $A\cap B=A$ . Hence $\mathbb P(A)=1/2$ , $\mathbb P(B)=3/4$ and

$\mathbb P(A\mid B)=\frac{\mathbb P(A\cap B)}{\mathbb P(B)}=\frac{1/2}{3/4}=\frac23,\qquad \mathbb P(B\mid A)=1.$

The conditioning events have positive probability, so both ratios are defined. The events are not independent because $\mathbb P(A\cap B)=1/2\ne(1/2)(3/4)=3/8$ , and not disjoint because their intersection contains two outcomes. Independence of the two tosses does not imply independence of all events constructed from them.

If this was the sticking point:
- Combinatorics & Graph Theory — Review ordered sample spaces and counting equally likely outcomes.
- Set Theory — Review intersections, subsets and disjointness of events.

Already comfortable with this? Skip elementary probability if you can derive the first check and handle conditioning with a specified sample space. Skip Markov-chain review if you can justify convergence using irreducibility and aperiodicity and explain the periodic counterexample.

Expectation is a weighted mean of outcomes

For four independent Bernoulli

(1/2)

variables,

\bar X_4

takes

k/4

with probability

\binom4k/16

. The bar heights sum to

1

; their weighted mean is

1/2

, and the weighted squared deviation is

1/16

. These are discrete probability masses, not areas under a density. Generally

\mathbb E\bar X_n=p

and

\operatorname{Var}(\bar X_n)=p(1-p)/n

under independence. Chebyshev then gives

\mathbb P(|\bar X_n-p|\ge\varepsilon)\le p(1-p)/(n\varepsilon^2)

Enlarge: Expectation is a weighted mean of outcomes

Review the prerequisites · Compare the full subject collection

Where this can lead

Measure & Integration — Learn measurable functions and convergence theorems before taking Lawler's measure-based route.
Dynamical Systems & Ergodic Theory — Continue from finite Markov chains to invariant behavior, adding measure theory for ergodic questions.

Reading-source dates

Grinstead and Snell's Introduction to Probability, CHANCE version dated July 4, 2006 · 2026-10-16
Probability exercises and odd-numbered answers · 2026-10-16

Reading choices revised 2026-10-17.

Suggest an edit · Report a correction

Condition on a specified event

A conditioning formula is only useful after the sample space is fixed. Two fair tosses already separate three ideas that are easy to confuse: independent trials, independent events and disjoint events.

What to read for this step

Use the July 4, 2006 CHANCE version of Grinstead and Snell named in the course guide. Read §§4.1–4.2 for conditioning, then §§6.1–6.3 for expectation and variance. The practice source links odd-numbered answers; the instructor solution set is not a public complete key.

When elementary conditioning is secure, use the existing Lawler comparison, §§1–3 and Exercise 4.1, to revisit probability spaces and independence. Add real analysis and measure theory before expecting conditional expectation to follow from finite ratios alone. The route introduction also cites a July 22, 2006 CHANCE file: check which edition you opened before using locators.

Starting text · Compare reading approaches · Exercises and available answers

One example to keep in view

For two fair independent tosses, write $\Omega=\{HH,HT,TH,TT\}$ with mass $1/4$ on each ordered outcome. Let $A=\{HH,HT\}$ mean “first toss heads” and $B=\{HH,HT,TH\}$ mean “at least one heads.”

Conditioning on $B$ leaves three outcomes, each with conditional mass $1/3$ . Thus $\mathbb P(A\mid B)=2/3$ , whereas $\mathbb P(B\mid A)=1$ . The tosses are independent; $A$ and $B$ are not. Use the existing diagram and check for the intersection calculation rather than memorizing the answer $2/3$ .

When to review, when to continue

If you can justify both conditional probabilities and distinguish dependence from disjointness, go on to the Bernoulli-average check. If the sample space is uncertain, review ordered outcomes first. Passing these finite checks does not replace the integration theory needed for measure-theoretic probability.

Editorial choice and revision

Grinstead and Snell lets you calculate before abstracting; Lawler is the change of language afterward. Keep the conditioning example as a reference when sigma-algebras enter, but do not assume the finite formula covers conditioning on a null event.

Added 2026-10-19. AI-assisted drafting; independent human mathematical review has not been recorded. Existing reading-source dates are unchanged.

The study guide above is translated; catalogue annotations below retain their original language.

Introduction to Probability, 2nd edition

Online text

Charles M. Grinstead and J. Laurie Snell · Textbook · Introductory

Examples and exercises develop discrete and continuous probability, with Markov chains and generating functions. Reviewers note broad coverage; graduate measure theory lies beyond its starting point.

Preparation: Calculus; elementary counting.

Full book & PDF Reviews / assessment

Probability Theory: STAT310/MATH230

PDF

Amir Dembo · Course notes · Advanced

Graduate course notes for measure-theoretic probability. More concise than a first undergraduate textbook; readers should work through proofs rather than rely only on examples.

Preparation: Real analysis; measure theory.

Open official PDF About / terms

Additional reading

Search the complete catalogue.

Introductory reading

Introduction to Probability

Joseph K. Blitzstein and Jessica Hwang · Textbook · Introductory · English · Online text

Harvard’s official Stat 110 page links this free online second edition. The short URL opens the authorized PDF in a Google Drive viewer rather than a direct PDF endpoint.

A calculus-level introduction to probability using conditioning, distributions and simulation. This is distinct from the existing Grinstead–Snell book with a similar title.

Preparation: Calculus; elementary counting.

Use: Prepare for measure-theoretic probability.

Open authorized online PDF viewer Source / terms

Edition and sources

Version: Second edition, 2019; author-authorized free online edition · Checked 2026-10-11

Probability

Richard Weber · Lecture notes · Introductory · English · PDF

The complete PDF opens publicly from the author’s university resource page.

A first course covering counting, conditional probability, distributions, random walks, branching and limit laws.

Preparation: Elementary calculus; algebra; basic proof-writing.

Use: Beginner bridge to stochastic processes and graduate probability.