When thinking about prevalences or treatment success rates, does 3/10 equal a 30 percent prevalence or success rate? The answer, surprisingly, is no. The fraction 3/10 is a proportion and reflects harvesting of data from which one might be tempted to claim a 30 percent success rate.
If we were able to check every human being on the planet and still came up with a 30 percent rate, it would represent a true proportion, but since we cannot, we must accept a sample that may vary from one sample-taking to another. Sampling "error" is what happens when we happen (randomly) to get 3/10 on Monday and 5/10 on Tuesday. Which is right and how confident can we be that any answer is representative of the true success rate?
A Formula
We need a formula that will allow us to say we are 95 percent confident that the real proportion lies somewhere within A and B, the upper limit and lower limit of the values observed.
The formula to use is:
In the above formula, "p" is the observed proportion, "n" is the number of subjects, 1.96 is a coefficient that generates a 95 percent probability and "p*" is the range within which there is a 95 percent chance the true proportion actually lies.
Let's look at our 3/10 rate using this formula:
Thus:
Interpretation
In sampling 10 subjects and finding an index condition in three, the prevalence of that condition is not necessarily 30 percent. It may be as low as 2 percent or as high as 58 percent! How can we improve our utility? Look at "n." The bigger n becomes, the tighter the confidence limits. That is why we usually need reasonably large numbers to reassure ourselves our estimates are realistic.
Another Use
Could a new treatment be placebo? Consider an investigator who audits a new (or old) treatment that has an apparent success rate of 8/10. Is this useful - or could it just be placebo? Let's compute the confidence intervals! If we do, we find that the confidence interval for 8/10, where "n" is 10, is 55 percent to 100 percent.
On the face of it, this looks OK, but recall that we have already calculated the CI for 3/10 (the rate conventionally accepted as the rate of placebo response in pain conditions): 2 percent to 58 percent. Note the two ranges overlap, so 8/10 may just be placebo!
Thanks again for thinkin' with me!
Resources
- Armitage P, Berry G. Statistical Methods in Medical Research, 3rd Edition. Oxford: Blackwell, 1994: pp. 93-125.
- Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology. A Basic Science for Clinical Medicine, 2nd Edition. Boston: Little, Brown. 1991: pp. 175-6.
- Bogduk N. Truth in musculoskeletal medicine: I. Confidence intervals. Aust Musculoskel Med, 1997 Nov:13-16.
Editor's Note: This is the third in a short series of articles by Dr. Charlton focused on different aspects of research as applicable to clinical practice. "Mechanism vs. Outcome: Thinking About the Gap Between Research and Clinical Practice" ran in the Sept. 1 issue; "Thinking About Cohen's Kappa" appeared in the Oct. 1 issue.
Dr. Keith Charlton, a 1975 graduate of National College of Chiropractic, practices in Brisbane, Queensland (Australia). Over the years, he has held membership in various chiropractic associations and been active on a political level, including serving as president of the Queensland Branch of the Chiropractors' Association of Australia, during which time he helped secure legislative recognition for the profession in Queensland. He is a member of the editorial boards of Chiropractic & Osteopathy and the Journal of Chiropractic Humanities, and previously served on the boards of several other peer-reviewed journals including JMPT and the Chiropractic Journal of Australia.