More On Seroquel's BOLDERs

Last month I wrote about some confusion around how treatment benefits--effects sizes--were determined in a recent study of Seroquel in bipolar depression. The study is known as BOLDER II, and was partly the basis for the FDA's approval of the atypical antipsychotic for use in treating depression in bipolar disorder. I sent some questions to the lead study author, Michael Thase, a professor of psychiatry at Pitt.

He got back to me with some answers, which I appreciate greatly and am now just getting around to posting. My original questions come first:

"1) What formula or formulas were used to calculate effect size based on mixed model repeated measures (MMRM) analysis?

2) How were means and standard deviations calculated using MMRM methods?

3) Why was a MMRM model used as opposed to using the conventional model for calculating effect size?

4) Why wasn't the conventional calculation of effect size also included in the writeup?

5) Why wasn't there an explanation for using the MMRM model in the study writeup?"

Thase's answers are:

"1) given the level of cynicism out there right now regarding research conducted by industry, it is important to point out that the MMRM analysis was the planned (a priori) method so that the results could be directly compared to those of Tohen et al. (Archives '03), who studied olanzapine (+/-fluoxetine) in bipolar depression. please keep in mind that olanzapine was the marketplace leader when the BOLDER studies were designed, so benchmarking results against that standard was an important goal.

2) there was no detailed description of MMRM because we didn't think it was necessary. MMRM is rapidly being adopted as the method of choice for clinical trial data sets; most statisticians view MMRM as superior to LOCF for handling the data of subjects who do not complete the treatment protocol. There are several nice write-ups on the merits of this approach, including a brief description in the Tohen et al. paper. I think that the description by Mallinckrodt et al (BMC Psychiatry. 2004;4:26) is one of the best.

3) the method used to calculate means and standard deviations from MMRM is as follows:

Least square mean and standard error estimates were provided by the MMRM methodology for each level of the treatment-by-visit interaction (i.e., for each treatment group at each visit). From the standard error estimates at Week 8, standard deviations were calculated for each treatment group: SD = SE * sqrt[n]. Then an estimate of the pooled SD (SDp) was calculated as:

SDp = (((n1-1)*stddev1^2 + (n2-1)*stddev2^2 + (n3-1)*stddev3^2) / (n1 + n2 + n3 - 3))^0.5

The mean difference at Week 8 was divided by the pooled SD to get the estimated effect size. The estimates at Week 8 are as follows:

300MG: -5.62 (SDp=9.23) => ES=0.61

600MG: -4.95 (SDp=9.23) => ES=0.54

4) we did not report effect sizes using the LOCF method largely because we didn't anticipate that someone would actually want to see them."