2.5 Explanation and prediction

In practice, there exist two critical purposes of any estimated model: Explanation (also known as inference) and Prediction. With the former, we focus the search on detecting any causal effect relationship in the model by the potential independent variables, While with the latter, we focus on getting accurate predictions for the dependent variables regardless of the regressors included.

2.5.1 Explanation:

All the regressors that may affect the dependent variable are vaguely determined by the investigator, based on her experience or economic theory, then the estimated model from the data at hand can decide about the effect significance of each regressor based on some statistic tests such as t-test and F-test.

The common effect types are:

Negative effect if \(\widehat\beta_1<0\) such as the price effect on the demand of some ordinary product.

Positive effect if \(\widehat\beta_1>0\) such as the income effect on the demand of some ordinary product.

Marginal effect of a regressor is the amount of change in the dependent variable as a result of changing the corresponding regressor by one unit. Formally, it is written (in the simple linear regression):

\[\begin{equation*} \frac{\partial \widehat Y_t}{\partial X_t}=\widehat\beta_1 \end{equation*}\]

Elasticity is the percentage change in the dependent variable given the percentage change in the regressor. It is also equal to the marginal effect when both variables (the dependent and the regressor) are expressed in \(log\):

\[\begin{equation*} E=\frac{\partial log(\widehat Y_t)}{\partial log(X_t)}=\frac{\partial \widehat Y_t\big/\widehat Y_t}{\partial X_t\big/X_t} \end{equation*}\]

2.5.2 Prediction:

Let \(X_{\theta}\) be the regressor value at time \(t=\theta\), the predicted value then will be:

\[\begin{equation} \widehat Y_{\theta}=\widehat\beta_0+\widehat\beta_1X_{\theta} \tag{2.50} \end{equation}\]

While the corresponding value in the true model is:

\[\begin{equation} Y_{\theta}=\beta_0+\beta_1X_{\theta}+\varepsilon_{\theta} \tag{2.51} \end{equation}\]

Consequently, we define the prediction error as:

\[\begin{equation} \widehat Y_{\theta}-Y_{\theta}=(\widehat\beta_0-\beta_0)+(\widehat\beta_1-\beta_1)X_{\theta}-\varepsilon_{\theta} \tag{2.52} \end{equation}\]

Clearly, the mean of this quantity is zero (using assumption A2 and the unbiasedness of the two parameters):

\[\begin{equation} E(\widehat Y_{\theta}-Y_{\theta})=0 \tag{2.53} \end{equation}\]

While the variance will be defined as follows:

\[\begin{align*} &V(\widehat Y_{\theta}-Y_{\theta})=\\ &E(\widehat Y_{\theta}-Y_{\theta})^2=\\ &E\bigg[\left(\widehat\beta_1-\beta_1\right)^2X^2_{\theta}+\left[(\widehat\beta_0-\beta_0)-\varepsilon_{\theta}\right]^2+2\left(\widehat\beta_1-\beta_1\right)X_{\theta}\left[(\widehat\beta_0-\beta_0)-\varepsilon_{\theta}\right]\bigg]=\\ &E\bigg[\left(\widehat\beta_1-\beta_1\right)^2X^2_{\theta}+(\widehat\beta_0-\beta_0)^2-2(\widehat\beta_0-\beta_0)\varepsilon_{\theta}+\varepsilon_{\theta}^2+\\ &2\left(\widehat\beta_1-\beta_1\right)(\widehat\beta_0-\beta_0)X_{\theta}-2\left(\widehat\beta_1-\beta_1\right)X_{\theta}\varepsilon_{\theta}\bigg]=\\ &X^2_{\theta}\underbrace{E\left(\widehat\beta_1-\beta_1\right)^2}_{V(\widehat\beta_1)}+\underbrace{E(\widehat\beta_0-\beta_0)^2}_{V(\widehat\beta_0)}-2\underbrace{E\left[(\widehat\beta_0-\beta_0)\varepsilon_{\theta}\right]}_{=0}+\underbrace{E(\varepsilon_{\theta}^2)}_{\sigma^2}+\\ &2X_{\theta}\underbrace{E\left[\left(\widehat\beta_1-\beta_1\right)(\widehat\beta_0-\beta_0)\right)}_{cov(\widehat\beta_1,\widehat\beta_0)}-2X_{\theta}\underbrace{E\left[\left(\widehat\beta_1-\beta_1\right)\varepsilon_{\theta}\right]}_{=0}=\\ &\frac{\sigma^2X^2_{\theta}}{\sum \left(X_t-\overline X\right)^2}+\frac{\sigma^2}{n}+\frac{\sigma^2\overline X^2}{\sum \left(X_t-\overline X\right)^2}+\sigma^2-\frac{2\sigma^2X_{\theta}\overline X}{\sum \left(X_t-\overline X\right)^2} \end{align*}\]

Finally, the prediction error variance is defined by:

\[\begin{equation} \sigma^2_{\theta}=\sigma^2\Bigg[1+\frac{1}{n}+\frac{(X_{\theta}-\overline X)^2}{\sum(X_t-\overline X)^2}\Bigg] \tag{2.54} \end{equation}\]

By looking at this formula, we see that large sample size \(n\), or large variability of the regressor \(\sum(X_t-\overline X)^2\), or the regressor value \(X_{\theta}\) is closer to the regressor mean, all of which lead to having a smaller variance, which what we want.

Now we can use this result to construct the prediction interval with the significance level \(\alpha\), as we did with the parameter estimates, after the replacement of the unknown error variance \(\sigma^2\) by the estimated one \(s^2\):

\[\begin{equation} Y_{\theta}\in \bigg[\widehat Y_{\theta}-t^{\alpha}_{(n-2)}s_{\theta}\quad , \quad \widehat Y_{\theta}+t^{\alpha}_{(n-2)}s_{\theta}\bigg] \tag{2.55} \end{equation}\]