Statistical Significance Versus Clinical Importance

Statistical significance tells us that an observed difference is reliable, but whether the difference is large enough to be important cannot be resolved by applying a statistical formula. 

The more important recent issue seems to be a study published in The Lancet which indicates a 9% increased risk of Type 2 diabetes associated with Crestor. As usual the question is whether these risks outweigh the benefits. The Crestor trial was typical of heart disease trials and involved a large population of 18,000 subjects. As the article notes, statistical significance in the reduction of heart attacks in this population does not necessarily translate to clinical significance:

Critics said the claim of cutting heart disease risk in half — repeated in news reports nationwide — may have misled some doctors and consumers because the patients were so healthy that they had little risk to begin with.

The rate of heart attacks, for example, was 0.37 percent, or 68 patients out of 8,901 who took a sugar pill.

Among the Crestor patients it was 0.17 percent, or 31 patients.

That 55 percent relative difference between the two groups translates to only 0.2 percentage points in absolute terms — or 2 people out of 1,000.

Stated another way, 500 people would need to be treated with Crestor for a year to avoid one usually survivable heart attack. Stroke numbers were similar.

“That’s statistically significant but not clinically significant,” said Dr. Steven W. Seiden, a cardiologist in Rockville Centre, N.Y., who is one of many practicing cardiologists closely following the issue. At $3.50 a pill, the cost of prescribing Crestor to 500 people for a year would be $638,000 to prevent one heart attack.

Is it worth it? AstraZeneca has concluded it is.

Others disagree.

“The benefit is vanishingly small,” Dr. Seiden said. “It just turns a lot of healthy people into patients and commits them to a lifetime of medication.”

Statistical Significance Versus Clinical Importance

Trials on Exercise Therapy for Chronic Low Back Pain as Example

Maurits van Tulder, PhD,*†§ Antti Malmivaara, MD, PhD,‡ Jill Hayden, DC,§
and Bart Koes, PhD

Study Design. Critical appraisal of the literature.
Objecives. The objective of this study was to assess if
results of back pain trials are statistically significant and
clinically important.
Summary of Background Data. There seems to be a
discrepancy between conclusions reported by authors
and actual results of randomized controlled trials. Little
attention has been paid to the problem of over-reporting
of conclusions.
Methods. All 43 trials of the Cochrane review on exer-
cise therapy for low back pain were included. Descriptive
analyses were conducted.
Results. Eighteen trials reported positive conclusions
in favor of exercise. Only six of the 43 studies showed
both clinically important and statistically significant differences in favor of the exercise groups on function, and 4 on pain.


It seems that many conclusions of stud-
ies of exercise therapy for chronic low back pain have
been based on

statistical significance of results rather
than on clinical importance and, consequently,

may have been too positive.

Authors of trials should report not only
statistical significance of results but also clinical importance.

Key words: low back pain, randomized controlled trials,
statistical significance, clinical importance. Spine 2007;32:

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.