We have stated that estimates of variance parameters are biased in ML (and IGLS) and unbiased in REML (and RIGLS). We believe that lack of bias is an important property and therefore REML will be used to analyse most of the examples in this book. Fixed effects estimates are unlikely to differ greatly between ML and REML analyses, but their standard errors will always be biased downwards if the variance parameters are biased (because they are calculated as weighted sums of the variance parameters). This will be most noticeable when the degrees of freedom used to estimate the variance parameters are small. Fixed effects estimates are also subject to additional sources of bias as explained below.
2.2.2 Estimation of fixed effects Maximum likelihood and REML
The fixed effects solution can be obtained by maximising the likelihood by differentiating the log likelihood with respect to a and setting the resulting expression to zero. This leads to a solution which is expressed in terms of the variance parameters:
Rearrangement gives a = (X'V-1X)-1X'V-1y and the variance of a is obtained as var(a) = (X'V-1X)-1X'V-1var(y)V-1X(X'V-1X)-1 = (X'V-1X)-1X'V-1VV-1X(X'V-1X)-1 = (X'V-1X)-1^
This formula is based on the assumption that V is known. Because V is, in fact, estimated it can be shown that there will be some downward bias in var(a). However, this is usually very small and approximate corrections for the bias can be made (see Section 2.4.3). Note that this occurs even when the variance component estimates are themselves unbiased. In Section 2.3 we will see that the Bayesian approach avoids having to make this assumption so that the problem of bias does not arise.
Was this article helpful?