# Bias in fixed and random effects standard errors

Fixed and random effects standard errors are calculated using a formula that is based on a known V (e.g. var(a) = (XV-1X)-1 for fixed effects). When data are balanced the standard errors will not be not biased. However, because V is, in fact, estimated, it is known that in most situations we meet in clinical trials there will be some downward bias in the standard errors. Bias will occur when the data are not balanced across random effects and effects are estimated using information from several error strata. In most situations the bias will be small. It is most likely to be relevant when

• the variance parameters are imprecise;

• the ratio of the variance parameters to the residual variance is small; and

• there is a large degree of imbalance in the data.

However, there is not a simple way to determine how much bias there will be in a given analysis. Results from simulation studies for particular circumstances have been reported in the literature (e.g. Yates, 1940; Kempthorne, 1952; McLean and Sanders, 1988; Nabugoomu and Allen, 1994; Kenward and Roger, 1997). Although information from these studies is not yet comprehensive enough to allow any firm rules to be defined, they indicate that the bias may be 5% or more if the number of random effects categories relating to a variance parameter is less than about 10 and the ratio of the variance parameter to the residual variance is less than one. In these situations a mixed model may not always be advisable unless an adjustment to the standard error is made.

Various adjustments for the bias have been suggested (e.g. Kacker and Harville, 1984; Kenward and Roger, 1997). Kenward and Roger's adjustment is now available in sas, but it occurs, surprisingly, as a degrees of freedom option within the model statement. Use of ddfm=kenwardroger inflates the estimated variance-covariance matrix of the fixed and random effects as described in their paper, following which Satterthwaite-type DF are then computed based on these variances (see Section 2.4.4).

For models fitting covariance patterns in the R matrix (e.g. repeated measures models) an alternative 'robust' variance estimator using the observed correlations between residuals known as the 'empirical' variance estimator (Liang and Zeger, 1986) has been suggested. It is calculated by var(a) = (X'V-1X)-1X'V-1cov(y)V-1X(X'V-1X)-1, where cov(y) can be taken as (y - Xa)(y - Xa)'. This estimator takes into account the observed covariance in the data and it is claimed that this may help alleviate some of the small sample bias. It does, however, cause the fixed effects variances to reflect observed covariances in the data rather than those specified by the covariance pattern modelled, and there is evidence against its use with small samples (see Long and Ervin, 2000). We discuss this further in Section 6.2.4. The empirical variance is calculated in sas by using the empirical option in the proc mixed statement.

Note that when the Bayesian approach is used (Section 2.3) exact standard deviations are obtained directly from the posterior distributions for each parameter and the problem of bias does not arise.

2.4.4 Significance testing