Probability and Statistics

Measures of dispersion

Dispersion describes how far the observations of a data set spread out around its central value, so it is always reported alongside a measure of location such as the mean or median. The main measures are the range, the variance, the standard deviation and the interquartile range. The range is the difference between the largest and smallest values and is very sensitive to outliers. The variance of a sample is the average squared deviation from the mean, s^2 = \frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})^2, and the standard deviation s is its positive square root, expressed in the same unit as the data. The interquartile range \text{IQR} = Q_3 - Q_1 covers the middle fifty percent of the data and is robust to extreme values. The median and the mode are measures of central tendency, not of dispersion, and must not be grouped with the spread measures. On a box plot, the box runs from the first quartile to the third quartile, so the IQR is the vertical length of the box; the whiskers and the median line do not enter the calculation. For example, if Q_1 = 12 and Q_3 = 20, then \text{IQR} = 8.

Correlation coefficient interpretation

Pearson's correlation coefficient \rho_{xy} (or r for a sample) measures the strength and direction of a linear relationship between two variables, and it always lies in [-1, +1]. A value of +1 or -1 means that all points lie exactly on a straight line, the sign gives the direction of the trend, and a value near zero indicates a weak linear association, which does not by itself prove independence. The slope of the least-squares line of y on x is b = r\, s_y / s_x, so the shape of a scatter plot depends on both the correlation and the ratio of the spreads. A perfect correlation with unit slope requires equal standard deviations, while a steeper perfect line means that y varies more than x. When |r| < 1 the points scatter around the line, and a steep cloud with s_y > s_x indicates a weaker linear link than a narrow cloud with the same spread ratio. Matching scatter plots to their attributes therefore requires checking both the value of the correlation and the relative size of the two standard deviations. The coefficient of determination r^2 is the fraction of the total variation in y explained by the regression, r^2 = 1 - \frac{SSR}{S_{yy}}, where SSR is the residual sum of squares, and it also equals \frac{S_{xy}^2}{S_{xx}S_{yy}}. Because it is a square, r^2 always lies between 0 and 1 and loses the sign of r. For example, if S_{yy} = 80 and the residual sum of squares is 20, then r^2 = 0.75.

Normal distribution

The normal (Gaussian) distribution is a continuous, symmetric, bell-shaped density determined by two parameters, the mean \mu that fixes its location and the standard deviation \sigma that fixes its scale, written X \sim N(\mu, \sigma^2). A distribution is parametric when its shape is fixed by a finite number of parameters, so the normal distribution is a parametric, two-parameter family and is not non-parametric. For comparison, the Poisson distribution has one parameter \lambda, and the uniform distribution on [a, b] has two. The standard normal distribution Z \sim N(0, 1) is a particular member of the normal family in which \mu = 0 and \sigma = 1 are fixed; fixing the values does not remove the parameters from the family, so it remains a two-parameter family standardized to a single shape. Any normal variable is converted to the standard normal by z = (x - \mu)/\sigma. For example, the probability that a normal variable lies below its mean is 0.5 for any \mu and any \sigma.

Variance of independent random variables

For random variables X and Y, the variance of their sum is \operatorname{Var}(X+Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X,Y), and the variance of their difference is \operatorname{Var}(X-Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) - 2\operatorname{Cov}(X,Y). Statistical independence implies zero covariance, so the cross term vanishes and the variances add: \operatorname{Var}(aX+bY) = a^2\operatorname{Var}(X) + b^2\operatorname{Var}(Y) for constants a and b. Two points follow from this. First, a coefficient enters the variance squared, not linearly. Second, the sign of a subtraction disappears, so \operatorname{Var}(X-Y) equals \operatorname{Var}(X+Y) when X and Y are independent. Zero covariance does not always imply independence, but independence always implies zero covariance. For example, if \operatorname{Var}(X) = 4 and \operatorname{Var}(Y) = 9 with X and Y independent, then \operatorname{Var}(X+Y) = 13 and \operatorname{Var}(2X - Y) = 16 + 9 = 25.

System reliability of parallel components

In a parallel configuration, the system works as long as at least one component works, so the system fails only when every component fails at the same time. If the components are independent, the failure probabilities multiply, and the system reliability is R_s = 1 - \prod_{i=1}^{n}(1 - R_i), where R_i is the reliability of component i. Adding a component to a parallel arrangement always increases the system reliability, while the reliability of a parallel system is never lower than that of its best component. The calculation proceeds by first computing each failure probability 1 - R_i, multiplying them, and subtracting the product from one. For example, two independent components each with reliability 0.8 give failure probabilities 0.2 each, so the product is 0.04 and R_s = 0.96.

Hypothesis testing for mean

When the population standard deviation is unknown and the sample is small, the mean is tested with the one-sample t-statistic t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}, where \mu_0 is the value stated in the null hypothesis, s is the sample standard deviation computed with divisor n-1, and the statistic follows a t-distribution with n-1 degrees of freedom. The sign of t always follows the sign of \bar{x} - \mu_0, so a sample mean below \mu_0 gives a negative statistic. The direction of the alternative hypothesis fixes the rejection region: for H_a: \mu < \mu_0 the test is left-tailed and rejects H_0 when t is less than -t_{\alpha, n-1}. The standard error s/\sqrt{n} in the denominator must be computed from the sample, not from the population. For example, a sample mean of 52 tested against \mu_0 = 50 with s = 4 and n = 16 gives t = 2/(4/4) = 2.

Chi-square test of independence

The chi-square test of independence checks whether two categorical classifications of the same sample are associated. The data are arranged in a contingency table with r rows and c columns, the null hypothesis states that the row and column classifications are independent, and the expected count in each cell is E_{ij} = \frac{(\text{row total})(\text{column total})}{N}. The test statistic \chi^2 = \sum \frac{(O_{ij} - E_{ij})^2}{E_{ij}} has (r-1)(c-1) degrees of freedom. For a 2 \times 2 table with cells a, b, c, d, the odds of the outcome in the first group are a/b and in the second group are c/d, so the odds ratio is OR = \frac{ad}{bc}. An odds ratio of 1 corresponds to independence, values above 1 indicate that the outcome is more likely in the first group, and values below 1 indicate that it is less likely. For example, cell counts of 20, 10, 5 and 15 give OR = \frac{20 \times 15}{10 \times 5} = 6.

Expectation of discrete random variable

For a discrete random variable X with probability mass function p(x), the expectation of any function g(X) is E[g(X)] = \sum_x g(x)\,p(x), and the probabilities must sum to one before the result is trusted. The expected value of the square is therefore not the square of the expected value; E(X^2) \neq [E(X)]^2 in general. The variance is obtained from the second moment as \operatorname{Var}(X) = E(X^2) - [E(X)]^2. Each value is squared first, then multiplied by its own probability, and the products are added. For example, if X takes the values 0, 1 and 2 with probabilities 0.5, 0.3 and 0.2, then E(X) = 0.7, E(X^2) = 0 + 0.3 + 0.8 = 1.1, and the variance is 1.1 - 0.49 = 0.61.

System reliability of series components

A series system works only if every component works, so the system fails as soon as any one component fails. For independent components with reliabilities R_i, the system reliability is R_s = \prod_{i} R_i, and the probability that at least one component fails is 1 - R_s. When the question asks for the probability that an accident occurs if either of two independent critical components fails, the required event is the union of the two failure events. For two events, P(A \cup B) = P(A) + P(B) - P(A)P(B) when they are independent, and the product term is subtracted once because the case in which both fail is already counted in each single term. Equivalently, one can compute 1 - (1 - P_A)(1 - P_B). For example, if the failure probabilities are 0.1 and 0.3, the probability of at least one failure is 0.1 + 0.3 - 0.03 = 0.37.