The Law of Large Numbers states that as a random experiment is repeated more and more times, the average of the observed results converges to the true expected value. First proven rigorously by Jacob Bernoulli (1713), this theorem is the mathematical justification for why casinos, insurance companies, and opinion pollsters can reliably predict long-run outcomes despite genuine unpredictability in any single individual trial.
Flip a fair coin 10 times. You might get 7 heads and 3 tails — a lopsided result far from the "expected" 50-50 split. But flip that same coin 10,000 times, and the proportion of heads will almost certainly land extremely close to 50%. This is the Law of Large Numbers: individual outcomes are unpredictable, but the long-run average reliably converges to the true underlying probability.
If a fair coin has landed heads five times in a row, does tails become "due" — more likely on the next flip? No. The Law of Large Numbers describes what happens over very many flips overall, but it says nothing at all about compensating for a short-term imbalance. Each new flip remains an entirely independent, fair 50-50 event. The mistaken belief that a "correction" must be coming soon is called the gambler's fallacy — a famous, extremely common, and costly misunderstanding of this theorem.
For independent, identically distributed random variables X₁, X₂, ..., Xₙ with finite expected value μ, let X̄ₙ = (X₁+X₂+...+Xₙ)/n be the sample average. The Law of Large Numbers states that X̄ₙ converges to μ as n→∞.
The weak version states X̄ₙ converges to μ in probability: for any small ε>0, the probability that |X̄ₙ−μ| exceeds ε shrinks to zero as n grows. This version, essentially proven by Jacob Bernoulli (1713) for the specific case of a binomial-type experiment, can be proven relatively directly using Chebyshev's inequality.
The strong version, proved rigorously by Andrey Kolmogorov (1930s) using more sophisticated measure-theoretic tools, makes a considerably more powerful claim: X̄ₙ converges to μ almost surely — meaning that, with probability exactly 1, the actual observed sequence of sample averages will genuinely settle down and converge to μ (rather than merely becoming increasingly unlikely to deviate significantly, as the weaker version states). Almost sure convergence implies convergence in probability, but the reverse implication does not hold in general — the strong law is a genuinely stronger and more difficult mathematical statement.
The Law of Large Numbers tells us that X̄ₙ converges to μ, but not how quickly, or how much variability to expect for finite n. The Central Limit Theorem (see that entry) supplies exactly this missing information: it describes the specific distribution of the remaining fluctuations around μ for large but finite n, showing they are approximately normally distributed with a spread shrinking proportionally to 1/√n.
Rolling a fair six-sided die repeatedly, the expected value per roll is exactly 3.5. After just 10 rolls, the running average might sit anywhere from roughly 2.5 to 4.5. After 1,000 rolls, the Law of Large Numbers guarantees the running average will almost certainly be extremely close to 3.5 — this predictable narrowing of the plausible range as sample size grows is the law in direct, observable action.
Chebyshev's inequality states P(|X−μ|≥k) ≤ Var(X)/k² for any random variable X. Applying this to X̄ₙ (whose variance is σ²/n, shrinking as n grows, since averaging independent variables reduces variance) directly gives P(|X̄ₙ−μ|≥ε) ≤ σ²/(nε²) → 0 as n→∞ — establishing convergence in probability with a comparatively simple, elementary argument requiring no advanced machinery beyond basic variance properties.
Jacob Bernoulli's Ars Conjectandi (1713, published posthumously by his nephew) contains the first rigorous proof of a version of the law, which Bernoulli himself modestly but tellingly called his "golden theorem," recognising its fundamental importance even at that early stage in the development of formal probability theory. Siméon Denis Poisson later coined the actual term "law of large numbers" (1837). The considerably more difficult strong law required the full development of modern measure-theoretic probability (Kolmogorov, 1933 — see the Probability entry) before it could be stated and proven with complete rigour.
The Law of Large Numbers requires the underlying random variables to have a finite expected value (and, for some versions and convergence rate estimates, finite variance). For certain "heavy-tailed" probability distributions (such as the Cauchy distribution, which has neither a finite mean nor variance), the sample average does not converge at all as n increases — it can instead fluctuate wildly and unpredictably forever, regardless of how large a sample is taken. This is an important and often-overlooked caveat: the law's near-universal reliability in everyday standard applications depends on an underlying finite-mean assumption that, while satisfied by most common real-world distributions, is not automatically guaranteed for every conceivable probability distribution.
The Law of Large Numbers provides the fundamental theoretical justification for why the sample mean is considered a consistent estimator of the true population mean in statistics — meaning that, as more data is collected, the estimate is mathematically guaranteed to become more and more reliable and accurate. This single foundational theoretical property underlies the validity of essentially the entire practice of statistical estimation and inference from sampled data across all of the sciences.
The Law of Large Numbers is also the essential theoretical foundation of Monte Carlo simulation methods — a broad and widely used computational technique for numerically estimating a fixed but analytically hard-to-calculate value (such as a difficult multi-dimensional integral) by averaging over a very large number of independent random samples. Named whimsically after the famous Monte Carlo casino (owing to their shared essential reliance on repeated random chance), these methods are extensively used throughout modern physics, finance, and computational engineering.