2 Scientific Misconduct, Error or Willful Ignorance?
Printer Friendly Email a Friend PDF RSS Feed
Dynamic Chiropractic – June 30, 2003, Vol. 21, Issue 14

Scientific Misconduct, Error or Willful Ignorance?

By Stephen M. Perle, DC, MS
I like to perform magic. It is a hobby I became interested in as a child, and I still do magic periodically when I lecture. The most famous magician ever (unarguably) was Harry Houdini. In his later years, Houdini set about debunking psychics after they were unable to contact his dearly departed mother, Cecilia. He believed they were all frauds using tricks he was familiar with as a magician, to make people believe they could actually contact the "other side."

A modern magician-turned-skeptical-crusader is James Randi ("The Amazing Randi"), whose magic act has often included a class of magic called "mentalism" or "psychic magic" (giving the illusion of having psychic abilities). I once heard Randi talk about why he, a magician, was in the "business" of debunking psychics, pseudoscience and scientific misconduct. He said scientists were rather credulous, because the dissemination of scientific information is, in part, dependent on the honesty of the scientist. As a result, scientists tend to believe what people write in their papers. They naturally assume the author is honest. In fact, if you read books on critical evaluation of the literature,1-3 you will not find any discussion about how to determine the veracity of what is written, but rather, a method to critique what was done. These books contain information about how to determine if the methodology is appropriate to the aims of the study.

Randi knows, as did Houdini before him, how to make people believe something that does not exist; consequently, he is better than most at finding the fraud. However, sometimes misconduct (purposeful or unintentional) can be found by almost anyone with a little detective work. I am going to give some examples of misconduct, errors or willful ignorance that can be found readily in the chiropractic literature, to give an idea of some of the types of problems that occur. Knowing whether these problems are misconduct (intentional), error (unintentional) or willful ignorance requires knowledge of intent that cannot be determined from reading the paper in which the problem appeared.

A common type of misconduct, or perhaps unintentional error, is the way people cite research. I have seen this frequently in papers my students submit to me. The student reads paper #1, in which he or she finds an interesting fact attributed to paper #2. Rather than citing paper #1, the student cites paper #2 as the reference. I know that this seems reasonable to students, because the fact they are citing ostensibly comes from paper #2. The problem with this is that having never read paper #2, the student cannot certify the fidelity of paper #1's report of what was presented in paper #2. One should only cite references one has actually read. There are two solutions to this problem: the preferred of the two is if the fact paper #2 supposedly presented is critical, one should obtain paper #2. A lesser alternative is to cite paper #1, because ultimately the "fact" is being interpreted by the authors of that paper.

There is a humorous example of this type of problem in the chiropractic literature. Smallie4 wrote about a parody piece that was published in Palmer College's student newspaper, The Beacon, when he was involved with the paper. The joke, written by another student at Palmer, supposedly traces subluxation back to the "Aurignacian period (17,500 B.C.)." The misconduct is that both papers and textbooks about chiropractic have noted the "fact" that the subluxation is this ancient. Given that this "fact" was fabricated, it is obvious that neither of these authors actually read any actual historical research about the origin of subluxation in the Aurignacian period. This is a myth that has persisted because of citing secondary - and in this case, false - sources.

Overstating or misstating the results of a study is misconduct, or critical error, because it has the effect of fundamentally altering the interpretation of research. One example of misstating the results of a study occurred incidental to the publication of the RAND study on the appropriateness of spinal manipulation for low-back pain (LBP). Many in the chiropractic profession said this study validated chiropractic care for LBP. While chiropractors do deliver the majority of spinal manipulation services, the RAND study was not about chiropractic, specifically.

The World Chiropractic Alliance (WCA) appears to have overstated the results of studies it cites in a position paper on the supposed merits of high-volume chiropractic practices.5 It contends that there is ample evidence to show that volume of procedures performed is associated with successful outcome of those procedures. The WCA cites five papers6-10 it says show medical procedures are associated with better outcomes if performed more often. These studies did not find that all procedures investigated had better outcomes when performed more often; some procedures had no association between quality and quantity of procedures performed.7,10 One study did not even look at the effect volume had on quality of care.9

I think it also is a problem when one compares apples in the literature to oranges in real life. When the WCA compares medical "high-volume" to chiropractic high-volume, it is comparing apples and oranges. For Bardach, et al.,6 high-volume was 19 to 70 procedures performed per year. There was a greater variance for Birkmeyer, et al,7 with high-volume varying from "more than 11" to "more than 849" procedures per year, depending on the procedure. Schrag, et al., called "very-high-volume" 166 to 383 procedures per year8 and Canto, et al.,10 termed greater than or equal to 49 procedures per year "high-volume." It is hard to believe that any chiropractor would call 11 to 849 adjustments per year a high-volume practice. I think most would call chiropractic practices that see 100 or more patients per day with a single doctor "high volume."

It also is scientific misconduct or willful ignorance to exclude from a literature review studies that refute the author's hypothesis. Again, WCA's position paper provides a good example. I did a cursory search of the literature and identified three studies11-13 that concluded that improved quality of care comes with more doctor-patient contact time. This specifically contradicts the last sentence in the WCA's position paper: "Criticism of, or opposition to, high-volume chiropractic practices based solely on the number of patients who receive care, without regard to the outcomes of that care, is unfounded and unsupported by any valid research."5 It appears research about medical, not chiropractic, procedures shows the issue is equivocal about the effect of quantity of procedures performed and the quality of those procedures.

Misconduct in science comes in many forms. What has been presented here is the easiest form to find: the misrepresenting of previously published research. One need only trace an author's references and read them. The more malicious form of misconduct - falsifying research data - is both a more serious offense, and more difficult to find.

References

  1. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-Based Medicine: How to Practice and Teach EBM. 2nd ed. New York: Churchill Livingstone; 2000.
  2. Greenhalgh T. How to Read a Paper: The Basics of Evidence-Based Medicine. London: BMJ Publishing Group; 1997.
  3. Hagino C. How to Appraise Research: A Guide for Chiropractic Students and Practitioners. London: Churchill Livingstone; 2003.
  4. Smallie DD. Static and dynamic components of the chiropractic subluxation complex: a literature review. J Manipulative Physiol Ther 1989;12(2):152.
  5. www.worldchiropracticalliance.org/positions/highvolume.htm.
  6. Bardach NS, Zhao S, Gress DR, Lawton MT, Johnston SC. Association between subarachnoid hemorrhage outcomes and number of cases treated at California hospitals. Stroke 2002 Jul;33(7): 1851-6
  7. Birkmeyer JD, Siewers AE, Finlayson EV, et al. Hospital volume and surgical mortality in the United States. N Engl J Med 2002 Apr 11;346(15):1128-37.
  8. Schrag D, Cramer LD, Bach PB, Cohen AM, Warren JL, Begg CB. Influence of hospital procedure volume on outcomes following surgery for colon cancer. JAMA 2000 Dec 20;284(23):3028-35.
  9. Mahle WT, Spray TL, Wernovsky G, Gaynor JW, Clark BJ III. Survival after reconstructive surgery for hypoplastic left heart syndrome: A 15-year experience from a single institution. Circulation 2000 Nov 7;102(19 Suppl 3):III136-41.
  10. Canto JG, Every NR, Magid DJ, Rogers WJ, Malmgren JA, Frederick PD, French WJ, Tiefenbrunn AJ, Misra VK, Kiefe CI, Barron HV. The volume of primary angioplasty procedures and survival after acute myocardial infarction. National Registry of Myocardial Infarction 2 Investigators. N Engl J Med 2000 May 25;342 (21):1573-80.
  11. Howie JG, Porter AM, Heaney DJ, Hopton JL. Long-to-short consultation ratio: a proxy measure of quality of care for general practice. Br J Gen Pract 1991 Feb; 41(343):48-54.
  12. Howie JGR, Heaney, DJ, Maxwell M, Walker JJ, Freeman GK, Rai H. Quality at general practice consultations: cross-sectional survey. BMJ 1999; 319:738-743.
  13. Kaplan SH, Greenfield S, Ware JE. Assessing the effects of physician-patient interactions on the outcome of chronic disease. Med Care 1989;27(suppl 3):110-125 .

Stephen Perle, DC
Bridgeport, Connecticut

Click here for previous articles by Stephen M. Perle, DC, MS.

To report inappropriate ads, click here.