Bayes' theorem: Flipping a conditional probability around, going from P(evidence given cause) to P(cause given evidence).
Random variables
Discrete random variables: A number assigned to each outcome of a random experiment, when only countably many values are possible.
Continuous random variables: A random variable that can take any value in a range, described by a density curve instead of a list of probabilities.
Expectation and variance: The long-run average value of a random variable, and how spread out it typically is around that average.
Joint distributions: How two random variables behave together, including whether knowing one tells you anything about the other.
Distributions
Common discrete distributions: The handful of discrete shapes, Bernoulli, Binomial, Poisson, Geometric, that keep showing up because they model real repeated-trial situations.
Common continuous distributions: The handful of continuous shapes, Uniform, Normal, Exponential, that keep showing up in real measurements and waiting times.
Limit theorems
Law of large numbers: A sample average gets closer to the true average as you collect more data.
Central limit theorem: Why the average of almost any large sample ends up looking approximately Normal, regardless of the original distribution's shape.
Point estimation: Using sample data to produce a single best-guess number for an unknown population value.
Confidence intervals: A range, instead of a single number, built to contain the true value a stated percentage of the time.
Resampling and the bootstrap: Estimating how uncertain a statistic is by repeatedly resampling from the data you already have, instead of relying on a formula.
Hypothesis testing
Hypothesis testing basics: Checking whether observed data is surprising enough to reject a specific default assumption, the null hypothesis.
t-tests: Comparing one or two sample means against a hypothesized value, when the true variance is unknown.
Chi-squared tests: Checking whether observed category counts match an expected pattern, or whether two categorical variables are related.
ANOVA: Comparing the means of three or more groups at once without inflating the false-positive rate.
Regression
Correlation: A single number summarizing how strongly, and in what direction, two variables move together.
Linear regression: Fitting a straight line that best predicts one variable from another, and reading its slope as an effect size.
Multiple regression and model fit: Predicting an outcome from several variables at once, and judging how well the resulting model actually fits.
Bayesian methods
Bayesian inference: Updating a starting belief about a parameter into a revised belief, using Bayes' theorem and observed data.
Stochastic processes
Markov chains: A sequence of states where the next state depends only on the current one, not the full history.