Lecture 6: Effects at quantiles and quantile regression

Economics 527: Econometric Methods

Effects at the mean and at quantiles

Effect at the mean

• One regressor: Y_i = \beta_0 + \beta_1 X_i + U_i, \qquad \mathrm{E}\left[U_i \mid X_i\right] = 0

• Effect for the “average” individual, with U_i = 0: \beta_1 = \frac{\partial(\beta_0 + \beta_1 X_i)}{\partial X_i}

• Under normality: \begin{aligned} U_i \mid X_i &\sim N(0,\ \sigma^2) \\ \Longrightarrow\quad \mathrm{E}\left[Y_i \mid X_i\right] &= \beta_0 + \beta_1 X_i \end{aligned}

• Average effect: \beta_1.

Conditional quantile q^Y_\tau(X_i)

• z_\tau for Z \sim N(0,1): \mathrm{P}(Z \le z_\tau) = \tau, \qquad \tau \in (0,1)

• Definition. q^Y_\tau(X_i), the \tau-th quantile of Y_i given X_i: \mathrm{P}\big(Y_i \le q^Y_\tau(X_i) \mid X_i\big) = \tau

• Claim. q^Y_\tau(X_i) = \beta_0 + \beta_1 X_i + q^U_\tau(X_i)

• Claim. Under U_i \mid X_i \sim N(0,\ \sigma^2): q^U_\tau(X_i) = \sigma z_\tau

Verification

• Check that \beta_0 + \beta_1 X_i + \sigma z_\tau is the \tau-th quantile of Y_i given X_i: \begin{aligned} & \mathrm{P}\Big(Y_i \le \beta_0 + \beta_1 X_i + \underbrace{\sigma z_\tau}_{q^U_\tau(X_i)} \mid X_i\Big) \\ &= \mathrm{P}\big({\color{red} \beta_0 + \beta_1 X_i} + U_i \le {\color{red} \beta_0 + \beta_1 X_i} + \sigma z_\tau \mid X_i\big) \\ &= \mathrm{P}\big(U_i \le \sigma z_\tau \mid X_i\big) \\ &= \mathrm{P}\Big(\underbrace{\frac{U_i}{\sigma}}_{N(0,1)} \le z_\tau \mid X_i\Big) \\ &= \tau \end{aligned}

Same effect at every quantile

• Therefore: \begin{aligned} &\boxed{\ q^Y_\tau(X_i) = \beta_0 + \beta_1 X_i + \sigma z_\tau\ } \\ &= (\beta_0 + \sigma z_\tau) + \beta_1 X_i \end{aligned}

• Effect at the \tau-th quantile: \begin{aligned} \frac{\partial q^Y_\tau(X_i)}{\partial X_i} &= \frac{\partial\big[\beta_0 + \beta_1 X_i + \overbrace{\sigma z_\tau}^{\text{const.}}\big]}{\partial X_i} \\ &= \beta_1 \end{aligned}

• Same effect \beta_1 at every quantile and at the mean.

• For example, z_{0.05} \approx -1.64: q^Y_{0.05}(X_i) \approx \beta_0 + \beta_1 X_i - 1.64\,\sigma

Equal spread and parallel quantile lines

• Densities of Y_i given X_i = x on \beta_0 + \beta_1 x, and q^Y_{0.75}(x), q^Y_{0.25}(x):

  

Heteroskedastic normal model

• Suppose: \begin{aligned} U_i \mid X_i &\sim N\big(0,\ \underbrace{\sigma^2(X_i)}_{\text{not const.}}\big) \\ \Longrightarrow\quad & \boxed{\ q^Y_\tau(X_i) = \beta_0 + \beta_1 X_i + \sigma(X_i)\, z_\tau\ } \end{aligned}

• Exponential scale: \sigma(X_i) = e^{\gamma_0 + \gamma_1 X_i}

• Effect at the \tau-th quantile: \begin{aligned} \frac{\partial q^Y_\tau(X_i)}{\partial X_i} &= \frac{\partial\big[\beta_0 + \beta_1 X_i + \sigma(X_i)\, z_\tau\big]}{\partial X_i} \\ &= \beta_1 + z_\tau\, \frac{\partial \sigma(X_i)}{\partial X_i} \\ &= \beta_1 + z_\tau\, \frac{\partial e^{\gamma_0 + \gamma_1 X_i}}{\partial X_i} \\ &= \beta_1 + z_\tau\, \overbrace{\underbrace{\sigma(X_i)}_{e^{\gamma_0 + \gamma_1 X_i}}\, \gamma_1}^{>\,0 \ \text{ if } \gamma_1 > 0} \end{aligned}

Effects above and below the median

• Sign of z_\tau: \begin{aligned} z_\tau &> 0 \quad \text{if } \tau > 0.5 \\ z_{0.5} &= 0 \\ z_\tau &< 0 \quad \text{if } \tau < 0.5 \\ z_\tau &\to -\infty \quad \text{as } \tau \to 0 \end{aligned}

• Suppose \gamma_1 > 0: \frac{\partial q^Y_\tau(X_i)}{\partial X_i} = \beta_1 + z_\tau\, \sigma(X_i)\, \gamma_1 \ \begin{cases} > \beta_1 & \text{if } \tau > 0.5 \\ = \beta_1 & \text{if } \tau = 0.5 \\ < \beta_1 & \text{if } \tau < 0.5 \end{cases}

• At the median: \begin{aligned} q^Y_{0.5}(X_i) &= \beta_0 + \beta_1 X_i \\ &= \mathrm{E}\left[Y_i \mid X_i\right] \end{aligned}

Where heterogeneity comes from

• Given X_i, a distribution of U_i: \mathrm{E}\left[U_i \mid X_i\right] = 0, \qquad \text{but } U_i \text{ and } X_i \text{ not independent}

• Scale times a standard normal: \begin{aligned} U_i &= \sigma(X_i)\, \varepsilon_i, \qquad \varepsilon_i \mid X_i \sim N(0,1) \\ \Longrightarrow\quad q^Y_\tau(X_i) &= \beta_0 + \beta_1 X_i + \sigma(X_i)\, z_\tau \end{aligned}

• Individual at the \tau-th quantile of Y_i given X_i: \begin{aligned} \varepsilon_i &= z_\tau \\ \Longleftrightarrow\quad Y_i &= \beta_0 + \beta_1 X_i + \sigma(X_i)\, z_\tau \\ &= q^Y_\tau(X_i) \end{aligned}

Widening densities and fanning quantile curves

• Densities on \beta_0 + \beta_1 x, then q^Y_{0.8}(x) and q^Y_{0.2}(x):

  

Quantile regression

Quantile regression without normality

• Explain q^Y_\tau(X_i) instead of \mathrm{E}\left[Y_i \mid X_i\right].

• No model for \sigma(X_i).

• Mean as an optimizer: \underbrace{\mathrm{E}\left[Y_i\right]}_{\mu} = \underset{c \in \mathbb{R}}{\arg\min}\ \mathrm{E}\left[(\underset{1\times 1}{Y_i} - \underset{1\times 1}{c})^2\right]

• Definition. Check function, \tau \in (0,1): \rho_\tau(u) = u\, \big(\tau - \mathbf{1}(u < 0)\big)

• Two branches: \begin{aligned} u < 0 \quad &\Longrightarrow\quad \rho_\tau(u) = (\tau - 1)\, u \\ u > 0 \quad &\Longrightarrow\quad \rho_\tau(u) = \tau\, u \end{aligned}

• Definition. q^Y_\tau, the \tau-th quantile of Y_i: \mathrm{P}\big(Y_i \le q^Y_\tau\big) = \tau

• Quantile as an optimizer: q^Y_\tau = \underset{c \in \mathbb{R}}{\arg\min}\ \mathrm{E}\left[\rho_\tau(Y_i - c)\right]

Median: \tau = 0.5

• Check function: \begin{aligned} \rho_{0.5}(u) &= u\, \big(0.5 - \mathbf{1}(u < 0)\big) \\ &= \begin{cases} -\tfrac12\, u & \text{if } u < 0 \\ \tfrac12\, u & \text{if } u > 0 \end{cases} \\ &= \tfrac12\, |u| \end{aligned}

• Median as an optimizer: \begin{aligned} q^Y_{0.5} &= \underset{c \in \mathbb{R}}{\arg\min}\ \mathrm{E}\left[\rho_{0.5}(Y_i - c)\right] \\ &= \underset{c \in \mathbb{R}}{\arg\min}\ \mathrm{E}\left[|Y_i - c|\right] \end{aligned}

• Median: \mathrm{P}\big(Y_i \le q^Y_{0.5}\big) = 0.5

Why “check function”

• Graph of \rho_\tau:

  

• \tau < 0.5: a check mark, \checkmark.

Mean regression as a minimization

• X_i now the k \times 1 vector of regressors, \operatorname{rank}\left(\mathrm{E}\left[X_i X_i^\top\right]\right) = k; \beta, b \in \mathbb{R}^k.

• Population, then sample: \begin{aligned} \mu = \mathrm{E}\left[Y_i\right] &= \underset{c \in \mathbb{R}}{\arg\min}\ \mathrm{E}\left[(Y_i - c)^2\right] \\ \hat\mu &= \underset{c \in \mathbb{R}}{\arg\min}\ \frac{1}{n} \sum_{i=1}^n (Y_i - c)^2 \end{aligned}

• Conditional mean: \mathrm{E}\left[Y_i \mid X_i\right] = \underset{c \in \mathbb{R}}{\arg\min}\ \mathrm{E}\left[(Y_i - c)^2 \mid X_i\right]

• Assume \mathrm{E}\left[Y_i \mid X_i\right] = X_i^\top \beta: \beta = \underset{b \in \mathbb{R}^k}{\arg\min}\ \mathrm{E}\left[(Y_i - X_i^\top b)^2\right]

• OLS: \hat\beta = \underset{b \in \mathbb{R}^k}{\arg\min}\ \frac{1}{n} \sum_{i=1}^n (Y_i - X_i^\top b)^2

Quantile regression by analogy

• Population, then sample: \begin{aligned} q^Y_\tau &= \underset{c \in \mathbb{R}}{\arg\min}\ \mathrm{E}\left[\rho_\tau(Y_i - c)\right] \\ \hat q^Y_\tau &= \underset{c \in \mathbb{R}}{\arg\min}\ \frac{1}{n} \sum_{i=1}^n \rho_\tau(Y_i - c) \end{aligned}

• Assume q^Y_\tau(X_i) = X_i^\top \beta_\tau: \beta_\tau = \underset{b \in \mathbb{R}^k}{\arg\min}\ \mathrm{E}\left[\rho_\tau(Y_i - X_i^\top b)\right]

• Quantile regression: \boxed{\ \hat\beta_\tau = \underset{b \in \mathbb{R}^k}{\arg\min}\ \frac{1}{n} \sum_{i=1}^n \rho_\tau(Y_i - X_i^\top b)\ }

• Estimated conditional quantile: \hat q^Y_\tau(X_i) = X_i^\top \hat\beta_\tau

• Koenker and Bassett (1978), Econometrica.

Median regression

• Assume q^Y_{0.5}(X_i) = X_i^\top \beta_{0.5}: \beta_{0.5} = \underset{b \in \mathbb{R}^k}{\arg\min}\ \mathrm{E}\left[\rho_{0.5}(Y_i - X_i^\top b)\right]

• Estimator, with \rho_{0.5}(u) = \tfrac12\, |u|: \begin{aligned} \hat\beta_{0.5} &= \underset{b \in \mathbb{R}^k}{\arg\min}\ \frac{1}{n} \sum_{i=1}^n \rho_{0.5}(Y_i - X_i^\top b) \\ &= \underset{b \in \mathbb{R}^k}{\arg\min}\ \frac{1}{n} \sum_{i=1}^n |Y_i - X_i^\top b| \end{aligned}

• Estimated conditional median: \hat q^Y_{0.5}(X_i) = X_i^\top \hat\beta_{0.5}

Summary

Effects at quantiles

• Normal, homoskedastic: q^Y_\tau(X_i) = \beta_0 + \beta_1 X_i + \sigma z_\tau; \partial q^Y_\tau(X_i)/\partial X_i = \beta_1 at every \tau, the same as at the mean.

• Normal, heteroskedastic: q^Y_\tau(X_i) = \beta_0 + \beta_1 X_i + \sigma(X_i) z_\tau; with \sigma(X_i) = e^{\gamma_0 + \gamma_1 X_i}, \partial q^Y_\tau(X_i)/\partial X_i = \beta_1 + z_\tau \sigma(X_i) \gamma_1.

• \gamma_1 > 0: effect above \beta_1 for \tau > 0.5, equal to \beta_1 at \tau = 0.5, below \beta_1 for \tau < 0.5.

• U_i = \sigma(X_i)\,\varepsilon_i, \varepsilon_i \mid X_i \sim N(0,1); \varepsilon_i = z_\tau \Longleftrightarrow Y_i = q^Y_\tau(X_i).

Check function and quantile regression

• \rho_\tau(u) = u\,(\tau - \mathbf{1}(u < 0)), slope \tau - 1 for u < 0 and \tau for u > 0; \rho_{0.5}(u) = \tfrac12 |u|.

• \mu = \arg\min_c \mathrm{E}\left[(Y_i - c)^2\right] and q^Y_\tau = \arg\min_c \mathrm{E}\left[\rho_\tau(Y_i - c)\right].

• q^Y_\tau(X_i) = X_i^\top \beta_\tau: \hat\beta_\tau = \arg\min_b \frac{1}{n} \sum_{i=1}^n \rho_\tau(Y_i - X_i^\top b) and \hat q^Y_\tau(X_i) = X_i^\top \hat\beta_\tau; Koenker and Bassett (1978).

• \tau = 0.5: \hat\beta_{0.5} = \arg\min_b \frac{1}{n} \sum_{i=1}^n |Y_i - X_i^\top b|.