Friday, September 4, 2009

OMG!!! That's What They Said! Significant


"Clinical studies of adults with depression showed that adding ABILIFY to an antidepressant helped to significantly improve depressive symptoms compared to adults treated with an antidepressant alone."

Okay, first let me say that this is not "Pick on Abilify Month."  I usually wander the web, (not quite so intentional as surfing), for interesting little tidbits to share with my readers.  But at my last appointment, my doctor gave me a list and told me to do my research and pick one.  So for the last month, I have had a focus.

Abilify has long since been eliminated as the winner of this assignment.  But it is such a good example of so many of my interests, including the use of language (as in this monthly OMG!! feature), marketing and clinical trials, that I can't let it go.  In fact, it gets another post later this month.  Not because I am picking on it, but because, well, stay tuned.

I found the winning quote for the month's OMG contest, "...helped to significantly improve depressive symptoms..." at Abilify.com.  It's the word "significant" that wins the award.  They really should share this award with many contestants, because that's what they all say, "significant."

The passage is found on the page intended for consumers.  So you would think they are speaking in the language that consumers speak.  This is not the case.  "Significant" in this sentence does not mean "significant" in the language that you and I speak.  The authors are referring to clinical trials, where the word "significant" is as significant as "toast," as in, "We are having toast for breakfast."  It is not significant enough to include as a facebook status update.  It is more like a twitter.  Though in FDA Land, it is the magic word, like "Open, Sesame", Sesame meaning big bucks.  So that is significant in the language that you and I do speak.  But I don't have a button for OMG Sesame!

At a University of Berkeley site, you can find the following definition:
Significance, Significance level, Statistical significance:  The significance level of an hypothesis test is the chance that the test erroneously rejects the null hypothesis when the null hypothesis is true.
 
And they wonder why we turn to Wikipedia? -- where it says:
In statistics, a result is called statistically significant if it is unlikely to have occurred by chance.

So here is the deal.  Abilify.com is talking about their clinical trials, where people who were not responding to an SSRI or SNRI, one of the current crop of antidepressants, tried adding Abilify or placebo.  There are many interesting features about how Wyeth conducted these trials, and you will hear about them later.  The point is that they had to demonstrate to the FDA that those who took Abilify along with their antidepressant got better results than those who took the placebo.  If they could demonstrate that, the "effectiveness test," then they are part way toward approval for on-label usage, and a vast expansion of their market share, because there are a lot of us around who don't get better on the current crop of antidepressants, and more of us every day.  They also have to pass the "safety test" -- an issue for another day.

So how much better?  A "significant" amount.  And as I said, that does not mean what most people think it means, as in "I feel significantly better since I added Abilify to my treatment strategy."  Did you think that it did, when they said that "adding ABILIFY to an antidepressant helped to significantly improve depressive symptoms"?  It does not mean that at all.

So what does it mean?  There are several tests that researchers use to measure levels of depression.  One is the Montgomery Asberg Depression Rating Scale (MADRS).  This is a ten item scale that lets an evaluator rank your symptoms on a scale of 0 to 6, 0 meaning no symptoms, 6 meaning whale shit on the bottom of the ocean, to quote crazymeds.  Items include feelings of sadness, appearance of sadness, appetite, sleep, suicidal thoughts, etc.  Theoretically, you could get a total of 60 points, but that would put you out of the reach of clinical trials.  They don't let people that sick into clinical trials. They want to pass their clinical trials. So they go for a crowd that is easier to impress.

There were three clinical trials done for Abilify.  The results were consistent from one to the other.  So I will use just one as an example, the third, published in April, 2009.  172 people took a placebo along with their antidepressant.  They had a range of MADRS scores, and the middle score (the "mean") was 27.1, which is moderately depressed.  177 took Abilify with their antidepressant.  They also had a range of MADRS scores, with the middle score of 26.6, also moderately depressed.  There are a variety of small differences between these two groups.  In each case, those receiving the placebo were a lttle bit sicker, but as far as I know, not "significantly" sicker.  After six weeks, both groups had lower MADRS scores, meaning that both had reduced their depressive symptoms.  That is good news for both groups, from the patient's perspective.

Since the placebo group improved as well as the Abilify group, you could infer that some of the improvement came from the experience of being in a clinical trial itself, or maybe just from the passage of time, because people with depression do get better.  But the $64,000 question for Wyeth is whether there was a "significant" difference between the two.  And the answer -- ding,ding,ding -- is yes.  The placebo group reduced its score by 6.4 points, and the Abilify group by 10.1.  So the difference between the two was 3.7.  In the language of statistics, the probability that this difference of 3.7 points was due to chance is .1% -- one in a thousand.  If you get that score for two clinical trials, that's good enough for the FDA.  And they got it for three.

So isn't that significant?  Yes, if you are a statistician.  If you are a patient, if you are weighing the risks and benefits, then maybe yes, maybe not so much.  In a test with a possible score of 60, the difference between the two groups was less than four, or two questions that were answered just a little differently.  3.7 points on the MADRS scale means going from "looks miserable all of the time" to "appears sad and unhappy most of the time," and from "slightly reduced appetite" to "normal appetite."  Do you consider those two differences to be significant?  Is that what you expected when you read Abilify.com, "adding ABILIFY to an antidepressant helped to significantly improve depressive symptoms"?

I have clinpsych.blogspot.com to thank for helping me find the original research report.  In the next stage of the Abilify story, I will explore why these results, significant or not, did NOT impress the people who actually took the medication. 

No comments:

Post a Comment