Exercise 7.1 Solution Example - Hoff, A First Course in Bayesian Statistical Methods
標準ベイズ統計学 演習問題 7.1 解答例

For \(p_J\) to be a probability density function, it must satisfy \(\int \int p_J(\boldsymbol{\theta}, \Sigma) d\boldsymbol{\theta} d\Sigma = 1\).

However, \(\int p_J(\boldsymbol{\theta}, \Sigma) d\boldsymbol{\theta} = \infty\), so \(p_J\) cannot be a proper probability density function.

\begin{align*} p_J(\boldsymbol{\theta}, \Sigma | \boldsymbol{y}_1, \ldots, \boldsymbol{y}_n) &\propto p_J(\boldsymbol{\theta}, \Sigma) \times p(\boldsymbol{y}_1, \ldots, \boldsymbol{y}_n | \boldsymbol{\theta}, \Sigma) \\ &\propto |\Sigma|^{-(p+2)/2} \times |\Sigma|^{-n/2} \exp \left\{ -\frac{1}{2} \sum_{i=1}^n (\boldsymbol{y}_i - \boldsymbol{\theta})^T \Sigma^{-1} (\boldsymbol{y}_i - \boldsymbol{\theta}) \right\} \\ &= |\Sigma|^{-(p+n+2)/2} \exp \left\{ -\frac{1}{2} \sum_{i=1}^n (\boldsymbol{y}_i - \boldsymbol{\theta})^T \Sigma^{-1} (\boldsymbol{y}_i - \boldsymbol{\theta}) \right\} \\ &= |\Sigma|^{-(p+n+2)/2} \exp \left\{ -\frac{1}{2} \sum_{i=1}^n (\boldsymbol{y}_i - \bar{y} + \bar{y} - \boldsymbol{\theta})^T \Sigma^{-1} (\boldsymbol{y}_i - \bar{y} + \bar{y} - \boldsymbol{\theta}) \right\} \\ &= |\Sigma|^{-(p+n+2)/2} \exp \left\{ -\frac{1}{2} \sum_{i=1}^n \left[ (\boldsymbol{y}_i - \bar{y})^T \Sigma^{-1} (\boldsymbol{y}_i - \bar{y}) + 2(\boldsymbol{y}_i - \bar{y})^T \Sigma^{-1} (\bar{y} - \boldsymbol{\theta}) + (\bar{y} - \boldsymbol{\theta})^T \Sigma^{-1} (\bar{y} - \boldsymbol{\theta}) \right] \right\} \\ &= |\Sigma|^{-(p+n+2)/2} \exp \left\{ -\frac{1}{2} \left[ \sum_{i=1}^n (\boldsymbol{y}_i - \bar{y})^T \Sigma^{-1} (\boldsymbol{y}_i - \bar{y}) + 2 \sum_{i=1}^n (\boldsymbol{y}_i - \bar{y})^T \Sigma^{-1} (\bar{y} - \boldsymbol{\theta}) + n(\bar{y} - \boldsymbol{\theta})^T \Sigma^{-1} (\bar{y} - \boldsymbol{\theta}) \right] \right\} \\ &= |\Sigma|^{-(p+n+2)/2} \exp \left\{ -\frac{1}{2} \left[ \sum_{i=1}^n (\boldsymbol{y}_i - \bar{y})^T \Sigma^{-1} (\boldsymbol{y}_i - \bar{y}) + n(\bar{y} - \boldsymbol{\theta})^T \Sigma^{-1} (\bar{y} - \boldsymbol{\theta}) \right] \right\} \\ &= |\Sigma|^{-\frac{1}{2}} \exp \left\{ -\frac{1}{2} (\boldsymbol{\theta} - \bar{y})^T \left( \frac{\Sigma}{n} \right)^{-1} (\boldsymbol{\theta} - \bar{y}) \right\} \times |\Sigma|^{-\frac{n + p + 1}{2}} \exp \left\{ -\frac{1}{2} \text{tr}(S \Sigma^{-1}) \right\} \\ &\quad \text{where} \quad S = \sum_{i=1}^n (\boldsymbol{y}_i - \bar{y})(\boldsymbol{y}_i - \bar{y})^T \end{align*}

Thus, we have

\begin{align*} p_J(\boldsymbol{\theta} | \Sigma, \boldsymbol{y}_1, \ldots, \boldsymbol{y}_n) &\propto \exp \left\{ -\frac{1}{2} (\boldsymbol{\theta} - \bar{y})^T \left( \frac{\Sigma}{n} \right)^{-1} (\boldsymbol{\theta} - \bar{y}) \right\} \\ &\propto \text{dmultivariate normal}(\bar{y}, \Sigma/n), \\ \\ p_J(\Sigma | \boldsymbol{y}_1, \ldots, \boldsymbol{y}_n) &\propto |\Sigma|^{-\frac{n + p + 1}{2}} \exp \left\{ -\frac{1}{2} \text{tr}(S \Sigma^{-1}) \right\} \\ &\propto \text{dinverse Wishart}(n, S^{-1}). \end{align*}