Econometrics and Regression Analysis

Simple linear regression estimation

In the simple linear regression y_i=\beta_0+\beta_1x_i+u_i, ordinary least squares chooses \hat\beta_0 and \hat\beta_1 to minimise the residual sum of squares. The slope is \hat\beta_1=\sum(x_i-\bar x)(y_i-\bar y)/\sum(x_i-\bar x)^2, and the intercept is \hat\beta_0=\bar y-\hat\beta_1\bar x. To test H_0:\beta_1=0, the t-statistic is t=\hat\beta_1/SE(\hat\beta_1), which is compared with the t critical value on n-2 degrees of freedom. Keep full precision in intermediate steps and round only the final answer to the number of decimal places requested.

Goodness of fit is measured by R^2=1-RSS/TSS, where TSS=\sum(y_i-\bar y)^2. With an intercept in the model, 0\le R^2\le 1, and R^2 is the share of the variation in y explained by the regressor.

When the regressor is a single 0/1 dummy, the OLS slope equals the difference between the two group means of y. In a logit model with one binary regressor, the fitted probabilities reproduce the observed group proportions, so the slope equals the difference in log-odds, \ln(\text{odds}_1)-\ln(\text{odds}_0), where odds =p/(1-p). A negative slope therefore means that the odds of the outcome fall when the regressor moves from 0 to 1.

In a simple regression with an intercept, R^2=b^2S_{xx}/S_{yy}, where b is the slope estimate. Hence a zero slope gives R^2=0 exactly, not a value inside some interval.

In a log-linear trend model \ln Y_t=\beta_0+\beta_1 t+u_t, the slope \beta_1 approximates the constant growth rate of Y per period. A slope of 0.05 therefore corresponds to about 5 percent growth, obtained by multiplying by 100. The t-statistics attached to the coefficients test significance and do not change this growth rate.

Classical linear regression assumptions

The classical linear model assumes linearity in parameters, random sampling, no perfect multicollinearity, zero conditional mean of the errors, homoscedasticity, and no autocorrelation. Perfect multicollinearity occurs when one regressor is an exact linear function of others, for example X_1=cX_2. Then X'X is singular and the OLS coefficients cannot be uniquely determined. Homoscedasticity and no autocorrelation concern the variance and covariance of the errors, and linearity in parameters concerns the form of the model, so an exact relation between regressors violates none of these.

• Shapiro-Wilk test — a formal test of normality of residuals.
• Jarque-Bera test — uses sample skewness and kurtosis, with a chi-square distribution on 2 degrees of freedom.
• Breusch-Godfrey test — tests for serial correlation, not normality.
• Park test — a procedure for detecting heteroscedasticity.

Normality of the errors is not needed for unbiasedness of OLS, but it is needed for exact t and F inference in small samples. The Shapiro-Wilk and Jarque-Bera tests check it, while the Breusch-Godfrey and Park tests check other assumptions.

Elasticity in log-linear regression models

The elasticity of y with respect to x is \varepsilon=\frac{dy}{dx}\cdot\frac{x}{y}. Its value depends on the functional form. For a log-log model \ln y=\beta_1+\beta_2\ln x, the elasticity is the constant \beta_2. For a semi-log model \ln y=\beta_1+\beta_2x, the elasticity is \beta_2x and changes with x. For y=\beta_1-\beta_2\ln x, the derivative is -\beta_2/x, so the elasticity is -\beta_2/y. For example, if \ln y=1+0.4x, the elasticity at x=2 is 0.8.

In a log-log demand or energy model, the coefficient on the log of price is the constant price elasticity. If |\varepsilon|<1 demand is price inelastic, if |\varepsilon|>1 it is elastic, and if |\varepsilon|=1 it is unit elastic. The statistical significance of the coefficient is judged separately by its t-ratio, and a large t-value means the estimate is precisely estimated, whatever its size.

Omitted variable bias in OLS

When a relevant regressor x_2 is omitted, the short-regression slope on x_1 is \tilde\delta_1=\hat\beta_1+\hat\beta_2\hat\delta_{21}, where \hat\delta_{21} is the slope from regressing x_2 on x_1. The difference between the short and long slopes is \hat\beta_2\hat\delta_{21}. This difference is zero only if the omitted variable has a zero coefficient or is uncorrelated with x_1 in the sample. Equal slopes on x_1 therefore imply \hat\beta_2\,\mathrm{Cov}(x_1,x_2)=0. This does not imply that \mathrm{Cov}(x_1,y)=0, because the marginal association of x_1 with y can remain nonzero.

Demand function estimation

In a log-log demand function \ln Q=\alpha+\beta_1\ln P+\beta_2\ln M+\beta_3\ln S+\varepsilon, the coefficient on \ln P is the price elasticity and the coefficient on \ln M is the income elasticity. A positive income elasticity indicates a normal good and a negative one indicates an inferior good. Each coefficient must be read with its standard error: the t-ratio is the coefficient divided by its standard error, and |t|<1.96 means the estimate is not significant at the 5 percent level in a large sample. The number in parentheses is a standard error, not an elasticity. A large point estimate with a large standard error does not establish high elasticity.