Author: kuekawa

Random effects:

Effects in this context refer to group outcome averages estimed by a regression model.  In regular regression models (OLS, Logistic Regression model), these are often estimated as coefficients of a series of dummy variables representing group units (e.g., school A, school B, .. each coded as 0 or 1).  These are fixed effects if estimated by non HLM model.  If I am part of the school whose test score average is 500 and if it is estimated as fixed effects, that value is determined solely by the information obtained from that school (I used the word solely to emphasize the point, but this may not be exactly correct due to the fact that predictors from the whole data helped derive coefficients that in turn help adjust the average outcome).

HLM does something similar, but after the initial group averages were estimated, they will be adjusted by reliablity of the group estimates (and this process is called Baysian Shrinkage).  They are adjusted such that the estimates are pulled towards the grand mean. Reliably measured group estimate will stay close to the original fixed effect value.  Not so reliable estimates will be pulled towards the grand mean.  The idea is to mix the grand mean and the group mean to prevent not reliant group estimate from being away from the mean.

Reliabitliy of the group average is a function of a) n of subjects in it and b) outcome variance.  I think Intraclass correlation may be part of the algorithim, but I will check.

Not common: If your data is a census data (everyone is in the dataset, like US Census), you do not need to use HLM.  You do not even need statistical testing because every estimate you get is a true estimate.  (Note: my friend wrote me and said he disagrees.  He said even if he had all people's data he wants to do stat testing to compare male and female when the difference is very small.  Again I disagree.)

Common: If your data is a sample and data are hierarchically structured and thus errors are dependent (violation of the independence assumption), you may consider HLM to alleviate the clustering problem.  Examples of hierarchically structured data are:

• Students nested within schools (2-levels)
• Repeated measures nested within subjects who are nested within schools (3-levels)

It is sometimes said that the motivation for the use of HLM must be whether the group units are a random sample of the population.  The argument claims that if, for example, schools in the sample are a convenient sample, one cannot use HLM.  This is not exactly correct.  I state the following using RCT (randomized control trial) or QED (Quasi-experimental design) impact studies as a context.

If the group units are randomly sampled, one can generalize the result of impact analysis to the whole population.  If Intervention program A was found effective (or not effective) and the sample was a random sample of US population, this finding is generalizable.  If impact estimation relied on a convenient sample, one cannot generalize it to the whole population.  If RCT, the random sample vs. convinient sample difference should not affect the internal validity of the impact estimate.

There is a tricky case.  HLM is inappropriate if the group units are, for a lack of better word, distinct groups with apparent identity and as a researcher you are genuinely interested in the pure group estimates.  This is a case when the exact group estimate, derived as fixed effects not as random effects, are of interest.

For example, if group units are US states, HLM is most likely inappropriate.  State specific estimates should be interpreted as such and should not be treated as random effects. Imagine the outcome of interest is income level and the state you live in had the average income of 50,000.  Just because your state had a small number of survey respondents (and thus reliability of the estimate is lower and HLM will pull your average closer to the grand average), you do not want to see your state’s average to be changed to look more like the national average.  Another example would be a study of 20 regional hospitals.  You should be interested in the fixed estimates of hospital outcomes.

When HLM treats schools as the mixed effects, we are treating school units somewhat instrumentally (a bit rude thing to do :>) in order to obtain the best value for the intercept (=grade average of the group specific effects estimated as random effects).  So if you are a school, you may feel HLM is treating you without respect.  HLM will not respect your data if the sample size is small and outcome variance in your school is large.  But HLM is respecting you in a different way.  Your data is unreliable, so let me just adjust it to be more normal, so you won't embarrass yourself.

I have the copy of the first edition.  The second edition is here:

http://koti.kapsi.fi/~jvimpari/MM.pdf