Downward bias of fixed effects standard errors will occur as described in Section 2.4.3, because the covariance parameters are estimated and not known. A 'robust' variance estimate for fixed effects, known as the 'empirical' variance estimator, was mentioned for covariance pattern models in Section 2.4.3. This variance takes into account the observed covariance in the data which, it is claimed, may help alleviate some of the small sample bias. Our own simulation studies have revealed, however, that, for relatively small sample sizes, the use of the empirical variance estimator leads to the wrong size of test - when the null hypothesis is true, an excess of statistically significant differences are found. Paradoxically, we have found that with larger sample sizes the empirical variance estimator performs much better than at smaller sample sizes.
An alternative approach to the downward bias of fixed effects standard errors is to inflate the estimated variance-covariance matrix of the fixed effects, var(a), following the approach described by Kenward and Roger (1997). This is implemented in SAS using the option DDFM=KENWARDROGER (or DDFM = KR) within the MODEL statement. In contrast to our findings on the empirical variance estimator, our simulation studies showed that the Kenward-Roger method performed satisfactorily down to very small sample sizes (five subjects per treatment group) and in the presence of missing values. Our recommendation is that in most circumstances, and particularly with small sample sizes, the Kenward-Roger method should be the one of choice. There is still the problem, of course, of specifying an appropriate covariance pattern. For large sample sizes the methods of Section 6.2.2 can be applied in the knowledge that there will be reasonable power for detecting statistically significant improvements in model fit, with more complicated covariance patterns. Alternatively, a pragmatic approach which would lead to a simple, pre-specified, analysis plan would be to choose a simple covariance pattern, such as compound symmetry, but use the empirical variance estimator. This will ensure that the estimated standard errors of the fixed effects reflect the observed covariance pattern of the data. Although we cannot recommend this approach for small sample sizes, we feel it is a viable alternative with larger studies. For smaller sample sizes there are no ideal solutions. Our preference is to choose a simple, plausible, covariance pattern for the situation (often compound symmetry or Toeplitz) and use the Kenward-Roger method.
The empirical variance estimator and the Kenward-Roger method can be used with both normal and non-normal distributions. Our simulation studies have so far looked only at the normal distribution, and we cannot legitimately infer the extent to which the findings will generalise to non-normal distributions. The empirical variance is calculated in SAS by using the EMPIRICAL option in the PROC MIXED statement. For non-normal data it is calculated by default when the REPEATED statement in PROC GENMOD is used. The introduction of PROC GLIMMIX to SAS offers additional flexibility in the use of empirical variance estimators, though we have no experience in their use and cannot make recommendations. Within the PROC GLIMMIX statement, there are five options of the form EMPIRICAL = <keyword>. The option EMPIRICAL = CLASSICAL produces the usual empirical estimators while the other four choices are different bias-adjusted estimators.
Was this article helpful?