Intermediate Econometrics

• If we have evidence of non-linearity we can explore the following options
• Transforming the predictors \(Y_i = \beta_1 + \beta_2 g(x_{i2}) + \ldots + \beta_p g(x_{ip}) + \varepsilon_i\)

1. Several transformations (log, square root, inverse, …)
2. Polynomial regression

• Transforming the outcome: \(f(Y_i) = \beta_1 + \beta_2 x_{i2} + \ldots + \beta_p x_{ip} + \varepsilon^*_i\)

1. For instance \(f(Y_i) = \log(Y_i)\); in such a case we are assuming that \(\log (Y_i)\) (and \(\varepsilon^*_i\)) are normally distributed, so \(Y_i\) and \(\varepsilon_i = {\rm exp}(\varepsilon^*_i)\) are distributed accordingly to he lognormal distribution

• Transform either the predictors and the outcome

• Also without assuming the normality of errors, the OLS estimator has good properties
• However, without the normality assumption, the inferential results (confidence intervals and hypothesis test) cannot be obtained easily
• Sometimes, we try to recover the normality assumption via transformations, for instance \(\log()\)

1. A very popular transformation is the Box-Cox transformation, which includes the logarithmic transformation as a special case (we do not see)
2. Considering a different class of models: Generalized Linear Models (GLMs), that also includes the normal linear model, where we assume that the response variable is distributed according to a certain distribution (we will see some of them)

• Let’s suppose that we are in a case where \(V(\varepsilon_i) = V(Y_i) = \sigma^2_i\), for \(i=1, \ldots, n\): the variance is not constant \(\implies\) heteroschedasticity

1. The OLS estimator \(\hat \beta\) has mean \(\beta\) (unbiased)
2. The OLS estimator \(\hat \beta\) is no more efficient; indeed its variance covariance matrix is \[V(\hat \beta) = V((X^\top X)^{-1}X^\top Y) = (X^\top X)^{-1}X^\top V(Y) X (X^\top X)^{-1}\]
3. The distribution is still normal \(\hat \beta \sim \mathcal{N}(\beta, V(\hat \beta))\)

1. Trasforming the outcome (there are some trasformations that are able to stabilize the variance, e.g. square root, logarithm, inverse square root, arcsin of the square root). Consider the following data

1. Trasforming the outcome: Consider the square root
   

2. Change the estimation approach: GLS (generalized least square)

• The model structure remains the same: \[ Y = X\beta + \varepsilon\] but we are substituting the homoschedasticity (\(V(\varepsilon) = \sigma^2I\)) assumption with \[V(\varepsilon) = \sigma^2 \Omega\] where \(\Omega\) is a known diagonal matrix (obviously not the identity)

2. Change the estimation approach: GLS (generalized least square)

• The log-likelihood for (\(\beta, \sigma^2\)) is \[\ell(\beta, \sigma^2) = - \frac{n}{2}\log \sigma^2 - \frac{1}{2\sigma^2} (y - X\beta)^\top \Omega^{-1} (y - X\beta)\]
• The maximum likelihood estimate of \(\beta\) is \[\hat \beta = \arg\min_\beta (y - X\beta)^\top \Omega^{-1}(y - X\beta) = (X^\top \Omega^{-1} X)^{-1}X^\top \Omega^{-1}y\]
• So, it is easy to derive \(V(\hat \beta)\) (\(\hat \beta\) is still normally distributed)
• Since \(\Omega\) is diagonal, by using the generalized least squares we are minimizing the function \[RSS_{g}(\beta) = \sum_{i=1}^{n}\frac{1}{w_{ii}} (y_i - x^\top_i \beta)^2\] where \(\omega_{ii}\) is the \(i\)-th diagonal element of \(\Omega\)

2. Change the estimation approach: GLS (generalized least square)

• In summary, the contribution to the sum of squares are weighted by \(1/\omega_{ii}\): greater is \(\omega_{ii}\), that is greater is the variance of the error \(\varepsilon_i\) for the \(i\)-th observations, and lower is the weight associated to the contribution of the sum
• Note: If \(\Omega\) is not diagonal, the generalized least squares can be used to deal with error’s dependence, in addition to the heteroschedasticity
• The case illustrated above belongs to the class of non-spherical disturbance errors

• The dataset is extracted from Wooldridge’s Introductory Econometrics (2016)

• Goal:

1. detect heteroschedasticity (non-constant error variance) in a simple wage regression
2. show how it can be mitigated by applying a logarithmic transformation to the dependent variable, or alternatively by estimating the model using Generalized Least Squares (GLS).

• Estimate a linear regression: \[wage_i = \beta_0 + \beta_1 educ_i + \beta_2 exper_i + \beta_3 tenure_i + \varepsilon_i\]

• Testing for heteroschedasticity, fit via OLS and GLS, then considering a transformation of the outcome

• The dataset wage1 is included in the wooldridge R package

• It is a classic example in labor economics

• It contains cross-sectional data on 526 working individuals in the U.S.

• Variables

1. wage: Hourly wage (in USD)
2. educ: Years of education
3. exper: Years of labor market experience
4. tenure: Years with current employer
5. nonwhite, female, etc.: Demographic indicators

• educ: Each additional year of education increases the hourly wage by about 0.60 dollars per hour, on average, holding fixed the other predictors
• exper and tenure typically have positive but a lower effect

• Residual variance increases for higher predicted wages — a clear sign of heteroskedasticity

• Model \(y_i = x^\top_i \beta + \varepsilon_i\), with \(\varepsilon_i \sim \mathcal{N}(0, \sigma^2_i)\), \(i = 1, \ldots, n\)

• Test statistic: \[LM = \frac{1}{2}f^{t} W (W^TW)^{-1} W f\] where

1. \(f\) is a n-dimensional vector composed by \((e^2_i/\hat \sigma^2 - 1)\), where \(e_i\) is the \(i\)-th residual of the OLS regression and \(\hat \sigma^2\) is the estimate of the variance of the error term
2. \(W\) is a matrix of covariates. Indeed, we assume that \(\sigma^2_i\) is a function of \(J\) covariates denoted by \(w_i\), that is \(\sigma^2_i = h(\mathbf{w}^\top_i\delta)\), where the first element of the vector \(w_i\) is equal to 1

• The test statistic is, under \(H_0: \sigma^2_1 = \sigma^2_2 = \ldots = \sigma^2_n = \sigma^2\) (homoschedasticity), distributed according to a Chi-square distribution with \(J\) degrees of freedom

• Breusch - Pagan Test: p-value close to 0 suggesting a clear evidence for rejecting \(H_0\) (so confirming the results of the graphical inspection)

• Let’s fit a GLS model, the R function is gls (in addition to the model and the data formula you must provide the weights)
• In this case we are saying that the variance of the errors is proportional to \(\hat y_i^\delta\), with \(\delta\) a parameter to be estimated (you have several options)

• The results are quite different (especially the estimated coefficients for educ)
• If there is no heteroschedasticity we expect similar estimated regression coefficients and standard errors
• In the following slide, the residuals are plotted

• The results are no more too different (especially the estimated coefficients for educ)

• We discussed that we can compare models in terms of model accuracy by analyzing the RSE or the \(R^2\) but only if we are working in the same scale
• However, for the GLS models the \(R^2\) is no longer available because the variance decomposition is no more unique (there exists some pseudo \(R^2\))
• We can compare them using the RSE (but only within the model having the same scale of the answer)

• It is based on the log-likelihood of the fitted model and a penalization term

\[AIC = 2(p+1) - 2 \ell(\hat {\boldsymbol \beta}; \hat \sigma^2))\]

• In a linear model it is simply

\[AIC = 2(p+1) + n \log {\hat \sigma}^2\]

• To compare models fitted in different scales, you must consider an additive term in the log-likelihood

• Lower is the best

• The comparison

• It refers to situations in which two or more predictors are closely related to one other.
• See the example below: while Limit and Age have not an obvious relationship, limit and rating are highly correlated (we say that they are collinear)

• Collinearity can pose problems in the context of regressions, since it can be difficult to separate individual effects of collinear variables on the response
• In other words, since limit and rating tend to increase or decrease togheter , it can be difficult to determine how each one separately is associated with the response (balance)
• Collinearity reduces the accuracy of the estimates: so it causes the standard error of \(\hat \beta_j\) to grow
• Remember that the t-statistic for testing the nullity of the single coefficients is obtained by dividing the estimate of \(\hat \beta_j\) for its standard error
• Consequently, collinearity results in a decline of the t-statistic and we may fail to reject \(H_0: \beta_j=0\)

• We fit the simple linear regression models

• The estimated coefficient for limit is almost similar when including age in the model, as well as its standard error

• Here we can see that there is a drastic change on both the estimated coefficients and they associated standard errors (increase of 12 times implying a p-value of 0.7)
• The importance of the limit variable is masked due to the presence of collinearity

• A simple way to detect a potential problem of collinearity is to see the correlation matrix
• Unfortunately, not all the problems related with collinearity can be detected by analyzing the correlation matrix: it is possible for collinearity to exist between three or more variable, even if no pair of variables has a particularly high correlation (we call this situation multicollinearity)

• An alternative is to explore the Variance Inflaction Factor (VIF): the ratio between the variance of \(\hat \beta_j\) when fitting the full model and the variance of \(\hat \beta_j\) when fitting the model on its own
• The lower value is 1 and values of 5/10 indicates a problematic amount of multicollinearity
• In this case, it is clear the presence of multicollinearity (values of 160)

• Possible solutions to deal with this problem

1. Removing one of the variables (for instance Rating)
2. Combining the predictors into a single one
3. Use regularization techniques (we do not see)

• The unbiasedness and the consistency of the OLS estimator rest on the hypothesis that the conditional expectation of the error is constant (and can be set to zero if the model contains an intercept)

• Consider the simple linear model: \[y_i = \beta_1 + \beta_2 x_i + \varepsilon_i\] with \[\mathbb{E}(Y_i|x_i) = \mathbb{E}[\beta_1+\beta_2x_i + \varepsilon_i|x_i] = \beta_1 +\beta_2x_i + \mathbb{E}[\varepsilon_i| x_i] = \beta_1 +\beta_2x_i\] if \(\mathbb{E}(\varepsilon_i|x_i) = 0\)

• When \(x\) is correlated with \(\varepsilon\), we face endogeneity

• Leads to biased and inconsistent OLS estimates

• Common in observational data

• The same property can also be described using the covariance between the covariate and the error that can be written, using the rule of repeated expectation (tower’s property): \[\text{cov}(x, \varepsilon) = \mathbb{E}[(x-\mu_x)\varepsilon] = \mathbb{E}_x[\mathbb{E}_\varepsilon[(x-\mu_x)\varepsilon |x]] = \mathbb{E}_x[(x-\mu_x)\mathbb{E}_\varepsilon[\varepsilon |x]]\]

• Key consequence: \[\text{cov}(x,\varepsilon) \neq 0 \Rightarrow \hat{\beta_2}_{OLS} \xrightarrow{p} \beta_2 + \frac{\text{cov}(x,\varepsilon)}{\text{var}(x)}\] where \(\xrightarrow{p}\) stay for convergence in probability

• Indeed, considering a simple linear regression model \[\hat{\beta_2}_{OLS}=\frac{cov(x,y)}{var(x)} = \frac{cov(x,\beta_1+\beta_2x+\varepsilon)}{var(x)} = \beta_2 + \frac{cov(x,\varepsilon)}{var(x)}\]

• Cases that we could encounter in application:

1. if \(cov(x,\varepsilon)>0 \implies\) OLS overestimes \(\beta_2\)
2. if \(cov(x,\varepsilon)<0 \implies\) OLS underestimes \(\beta_2\)
3. if \(cov(x,\varepsilon)=0 \implies\) OLS is consistent (converges to \(\beta_2\))

• OLS incorrectly attributes variation in \(y\) to \(x\)
• Standard errors no longer meaningful
• Inference becomes unreliable

• If the conditional expectation of \(\varepsilon\) is a constant, \(\mathbb{E}_\varepsilon[\varepsilon |x] = \mu_\varepsilon\) (not necessarily 0), the covariance is \[cov(x, \varepsilon) = \mu_\varepsilon \mathbb{E}_x[x-\mu_x] = 0\]

• Stated in a different way, \(x\) is supposed to be exogenous, or \(x\) is assumed to be uncorrelated with \(\varepsilon\)

• Endogeneity when \(cov(x, \varepsilon)\neq 0\)

• Sources of endogeneity:

1. There are errors in the variables
2. There are omitted variables
3. Simultaneity bias

• Data used in economics, especially micro-data, are prone to errors of measurement (either outcome and predictors)

• Suppose that the model that we seek to estimate is \[y^{*}_i = \beta_1 + \beta_2 x^{*}_i + \varepsilon^{*}_i\] where the covariates is exogenous (\(cov(x^*, \varepsilon^*)=0\))

• Suppose that the response is observed with error, namely that the observed value of the response is \[y^{*}_i = y_i - \nu_i\] where \(\nu_i\) is the measurement error of the respnse. Then

\[y_i= \beta_1 + \beta_2 x^*_i + (\varepsilon^*_i + \nu_i) \]

• The error of the model is \(\varepsilon_i = \varepsilon^*_i + \nu_i\), which is still uncorrelated with \(x\) if \(\nu\) is uncorrelated with \(x\), which means that the error of measurement of the response is uncorrelated with the covariate

• The measurement error only increases the size of the error, which implies that the coefficients are estimated less precisely and that the \(R^2\) is lower compared to a model with a correctly measured response

• Let’s suppose now that the covariate is observed with error, namely that the observed value of the covariate is \[x^{*}_i = x_i - \nu_i\] where \(\nu_i\) is the measurement error of the covariate.

• If the measurement error is uncorrelated with the value of the covariate, the variance of the observed covariate is \[\sigma^2_x = \sigma^{*2}_x + \sigma^2_{\nu}\] and the covariance between the observed covariate and the measurement error is equal to the variance of the measurement error, that is \({\text cov}(x, \nu) = \mathbb{E}(x^* + \nu - \mu_x) \nu) = \sigma^2_\nu\) because the measurement error is uncorrelated with the covariate

• So, rewriting the model in terms of \(x\), we get \[y_i= \beta_1 + \beta_2 x_i + \varepsilon_i\] with \(\varepsilon_i= \varepsilon^{*}_i - \beta_2 \nu_i\)

• The error of the model is correlated with \(x\), as \(cov(x,\varepsilon) = cov(x^* + \nu, \varepsilon - \beta \nu) = -\beta\sigma^2_\nu\)

• The OLS estimator can be written as usual as

\[\hat \beta_2 = \frac{\sum_{i=1}^{n}(x_i - \bar x)(y_i - \bar y)}{\sum_{i=1}^{n}(x_i - \bar x)^2} = \beta_2 + \frac{\sum_{i=1}^{n}(x_i - \bar x)\varepsilon_i}{\sum_{i=1}^{n}(x_i - \bar x)^2}\]

• Taking the expectation, we have \(\mathbb{E}[(x-\bar x) \varepsilon]) = -\beta_2\sigma^2_\nu\) and the expected value of the estimator is \[\mathbb{E}(\hat \beta_2) = \beta_2 \left(1- \frac{\sigma^2_\nu}{\sum_{i=1}^{n}(x_i - \bar x)^2/n} \right) = \beta_2\left(1-\frac{\sigma^2_\nu}{\hat \sigma^2_x}\right)\]

• The OLS estimator is biased and the term in brackets is the minus the share of the variance of that is due to measurement errors

\[\mathbb{E}(\hat \beta_2) = \beta_2 \left(1- \frac{\sigma^2_\nu}{\sum_{i=1}^{n}(x_i - \bar x)^2/n} \right)= \beta_2\left(1-\frac{\sigma^2_\nu}{\hat \sigma^2_x}\right)\]

• Then, \(|\hat \beta_2| < \beta_2\)

• This is called attenuation bias: it can be either a lower or an upper bias depending on the sign of \(\beta\)

• This bias clearly doesn’t attenuate in large samples. As \(n\) grows, the empirical variances/covariances converge to the population ones, and the estimator therefore converges to \(\beta_2(1 - \sigma^2_\nu/\sigma^2_x)\)

• Suppose that the true model is: \[y_i = \beta_1 + \beta_2x_i + \beta_3 z_i + \varepsilon_i\] where \(E[\varepsilon|x,z]=0\) and the model can be estimated consistently using OLS

• Consider that \(z\) is unobserved

• The model to be estimated is \[y_i = \beta_1 + \beta_2x_i + \varepsilon^*_i\] where \(\varepsilon^*_i = \varepsilon_i + \beta_3 z_i\)

• The omission of relevant covaritate \(\beta_3 \neq 0\) has two consequences

1. The variance of the error is \[\sigma^2_{\varepsilon^*} = \beta^2_3 \sigma^2_z + \sigma^2_{\varepsilon}\] and it is greater than the one of the initial model for which \(z\) is observed and used as a covariate
2. the covariance between the error and \(x\) is \[\text{cov}(x, \varepsilon^*) = \beta_3 \text{cov}(x,z)\] if the covariate is correlated with the omitted variable, the covariate and the error of the model are correlated.

• As the variance of the OLS estimator is proportional to the variance of the errors, omission of a relevant covariate will always induce a less precise estimation of the slopes and a lower \(R^2\)

• Moreover, if the omitted covariate is correlated with the covariate used in the regression, the estimation will be biased and inconsistent

• This omitted variable bias can be computed as follows:

\[\hat \beta_2 = \beta_2 + \frac{\sum_{i=1}^{n}(x_i -\bar x)(\beta_3z_i + \varepsilon_i)}{\sum_{i=1}^{n}(x_i - \bar x)^2} = \beta_2 + \beta_3 \frac{\sum_{i=1}^{n}(x_i -\bar x)(z_i + \bar z)}{\sum_{i=1}^{n}(x_i - \bar x)^2} + \frac{\sum_{i=1}^{n}(x_i -\bar x)\varepsilon_i}{\sum_{i=1}^{n}(x_i - \bar x)^2}\]

• Taking the conditional expectation, the last term disappear, so that \[\mathbb{E}(\hat \beta_2|x,z) = \beta_2 + \beta_3 \hat{\text cov}(x,z)/\hat\sigma^2_x\]

1. upper bias if the signs of the covariance between \(x\) and \(z\) and \(\beta_3\) are the same
   
2. a lower bias if they have opposite signs

• There is an upper bias if the signs of the covariance between \(x\) and \(z\) and \(\beta_3\) are the same, and a lower bias if they have opposite signs.
• As \(n \to \infty\) the OLS estimator converges to \[\hat \beta_2 \xrightarrow{p}\beta_2 + \beta_3 {\text cov}(x,z)/{\text var(x)} = \beta_2 + \beta_3 \times \beta^{*2}\] where
• \(\beta^{*2}\) is the true value of the slope of the regression of \(z\) on \(x\)
• This formula makes clear what \(\hat \beta_2\) really estimates in a linear regression:

1. The direct effect of \(x\) on \(y\)
2. The indirect effect of \(x\) on \(y\) which is the product of \(x\) in \(z\) (\(\beta^{2*}\)) times the effect of \(z\) on \(\varepsilon^*\)

• A classic example of omitted variable bias occurs in the estimation of a Mincer earning function, which relates wage (w), education (e) and experience (s)

\[\log(w_i) = \beta_1 + \beta_2 e_i + \beta_3 s_i + \beta_4 s^2_i + \varepsilon_i\] where \(\beta_2\) is the percentage increase of the wage for one more year of education, holding fixed the other predictors. Indeed \[ \beta_2 = \frac{d \log(w)}{d e} = \frac{dw/w}{de}\]

• Numerous studies of the Mincer function deal with this problem of endogeneity of the education level

• Sample of 303 white males taken from the National Longitudinal Survey of Youth in 1992

• Variables:

1. wage: Log hourly wage
2. educ: Education (years of schooling)
3. AFQT: Ability based on standardized AFQT test score
4. educSibl: Education of oldest sibling (years of schooling)
5. exper: Experience (total weeks of labor market experience)
6. tenure: Tenure (weeks on current job)
7. educM: Mother’s education (years of schooling)
8. educF: Father’s education (years of schooling)
9. urban: Dummy variable for urban residence
10. home: Broken home dummy (dummy for living with both parents at age 14)

• Let’s fit an OLS regression: one more year of education increases the average by 10%, holding fixed the experience

• Concern: the individuals have different abilities (a), and that more abilities have a positive effect on wage, but may also have a positive effect on education

• If so, adding ability in the model \[\log(w_i) = \beta_1 + \beta_2 e_i + \beta_3 s_i + \beta_4 s^2_i + \beta_5 a_i + \varepsilon_i\]

will provide \(\beta_5>0\) and regressing ability on the education, that is \[e_i = \beta^*_1+ \beta^*_2 a_i + \varepsilon^*_i\] will provide \(\beta^*_2>0\)

• Thus \[\hat \beta_2 \xrightarrow{p} \beta_2 + \beta_5 \times \beta^*_2>\beta_2\] and the OLS estimator is upwarded biased

• This is the case, because more education

1. increases, for a given level of ability, the expected wage is \(\beta_2\)
2. on average, the level of ability is higher, this effect being \(\beta^*_2\), so the wage will also be higher

• In our data set we have the ability (AFQT), which is the standardized AFQT test score.

• If we introduce ability in the regression, education is no more endogenous and least squares will give a consistent estimation of the effect of education on wage.

• We first check that education and ability are positively correlated:

• Adding ability as a covariate should decrease the coefficient on education
• The effect of one more year of education is now an increase of 8.7% of the wage (compared to the 10%)
• The relatively small decrease suggests that the correlation between education and ability is limited, so the omitted variable bias in this example is not very large

• Often in economics, the phenomenon of interest is not described by a single equation, but by a system of equations
• Consider for example a market equilibrium. The two equations relate the quantity demanded / supplied (\(q^d\) and \(q^s\)) to the unit price and to some specific covariates to the demand and to the supply side of the market
• The equilibrium on the loan market is then defined by a system of three equations:

\[\begin{cases} q^d = \beta^d_1 + \beta^d_2 p + \beta^d_3 d +\varepsilon^d \\q^s = \beta^s_1 + \beta^s_2 p + \beta^o_3 s +\varepsilon^s\\q^d = q^s\end{cases}\] where

1. \(q\) is the quantity (in logarithm)
2. \(p\) is the price
3. \(d\) and \(s\) two rates

\[\begin{cases} q^d = \beta^d_1 + \beta^d_2 p + \beta^d_3 d +\varepsilon^d \\q^s = \beta^s_1 + \beta^s_2 p + \beta^o_3 s +\varepsilon^s\\q^d = q^s\end{cases}\]

• The demand curve should be decreasing: \(\beta^d_2<0\)

• The supply curve should be increasing: \(\beta^s_2>0\)

• By fitting the OLS regression we can identify the correct signs…

• However, the fit of the supply equation is very bad

• What is actually observed for each observation in the sample is a price-quantity combination at an equilibrium.

• A positive shock on the demand equation will move upward the demand curve and will lead to a new equilibrium with a higher equilibrium quantity \(q'\) and also a higher equilibrium price \(p'\) (except in the special case where the supply curve is vertical, which means that the price elasticity of supply is infinite).

• This means that \(p\) is correlated with \(\varepsilon^{d}\), which leads to a bias in the estimation of \(\beta^d_2\) via OLS

• The same reasoning applies of course to the supply curve.

• Instrumental variable regression can eliminate the bias when \(\mathbb{E}(\varepsilon|x)\neq 0\) using an instrumental variable (let’s say \(z\))
• We will explore very quickly the theoretical part (see the iv2 slides), in particular we will see an application in R