Computes a shrunken version of the maximum likelihood estimator for the sample covariance matrix under the assumption of multivariate normality.

For a sample matrix, x, we compute the sample covariance matrix as the maximum likelihood estimator (MLE) of the population covariance matrix and shrink it towards its diagonal.

cov_shrink_diag(x, gamma = 1)

Arguments

x	data matrix with n observations and p feature vectors
gamma	the shrinkage parameter. Must be between 0 and 1, inclusively. By default, the shrinkage parameter is 1, which simply yields the MLE.

Value

shrunken sample covariance matrix of size \(p \times p\)

Details

Let \(\widehat{\Sigma}\) be the MLE of the covariance matrix \(\Sigma\). Then, we shrink the MLE towards its diagonal by computing $$\widehat{\Sigma}(\gamma) = \gamma \widehat{\Sigma} + (1 - \gamma) \widehat{\Sigma} \circ I_p,$$ where \(\circ\) denotes the Hadamard product and \(\gamma \in [0,1]\).

For \(\gamma < 1\), the resulting shrunken covariance matrix estimator is positive definite, and for \(\gamma = 1\), we simply have the MLE, which can potentially be positive semidefinite (singular).

The estimator given here is based on Section 18.3.1 of the Hastie et al. (2008) text.

References

Hastie, T., Tibshirani, R., and Friedman, J. (2008), "The Elements of Statistical Learning: Data Mining, Inference, and Prediction," 2nd edition. http://web.stanford.edu/~hastie/ElemStatLearn/