Statistics

Chebyshev’s inequality Chebyshev’s inequality guarantees that, for a wide class of probability distributions, no more than a certain fraction of values can be more than a certain distance from the mean. Specifically, no more than 1/k^2 of the distribution’s values can be more than k standard deviations away from the mean (or equivalently, at least 1 − 1/k^2 of the distribution’s values are within k standard deviations of the mean). The rule is often called Chebyshev’s theorem, about the range of standard deviations around the mean, in statistics. The inequality has great utility because it can be applied to any probability distribution in which the mean and variance are defined. For example, it can be used to prove the weak law of large numbers.

Statement: Let X be a random variable with finite expected value μ and finite non-zero variance σ2. Then for any real number k > 0.

    Pr(|X-μ|>= kσ) <= 1/k^2

Only the case of k>1 is useful, when k<=1 the inequality is trivial.

Example: Suppose we randomly select a journal article from a source with an average of 1000 words per article, with a standard deviation of 200 words. We can then infer that the probability that it has between 600 and 1400 words (i.e. within k = 2 standard deviations of the mean) must be at least 75%, because there is no more than ​1⁄k^2 = 1/4 chance to be outside that range, by Chebyshev’s inequality. But if we additionally know that the distribution is normal, we can say there is a 75% chance the word count is between 770 and 1230 (which is an even tighter bound).

Law of Large numbers The law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer to the expected value as more trials are performed. The LLN is important because it guarantees stable long-term results for the averages of some random events. Weak Law The weak law of large numbers states that the sample average converges in probability towards the expected value.