Imagine being a student again, about to take one of the four tests required to become a doctor of chiropractic. You've studied almost nonstop for the past few weeks. You can feel your anxiety level rise as you sit down in front of the computer screen.
After some time, the computers reboot and come back up. You are asked to start over. Frustrated, you begin again. You had answered as many as 70 questions. Now you get to answer them twice.
Then it happens again: computers freeze, then reboot, and students are forced to start over (for the second time). One proctor comments that there is only one technician at the NBCE home office trying to solve the problem, as the test is being administered on a weekend.
A short time later, the screen freeze happens a third time; answers again lost, tempers rising. Finally, after the fourth computer crash, the decision is made to go to paper tests. But with the loss of time, you have to work through your scheduled breaks and even forgo lunch.
Such was the experience of many students when they took their board exams this June. Some were lucky to have completed sections before a crash; others were not. Students on the West Coast were spared the trauma, as they went straight to paper exams.
Not the First Time?
This unfortunate situation is just one of a series of problems that suggest serious deficiencies within the NBCE organization, many the apparent result of complacency and a continued lack of innovation. In fact, in the board's second-quarter progress report, NBCE President Dr. Paul Morin admitted the June test fiasco was not the first:
"In March 2016, we experienced a CBT malfunction that affected some examinees. Those examinees were offered an opportunity to retest in early April to complete their examination process, which resulted in all examinees receiving a fair and valid examination process."
In the same report, Dr. Morin also announced that "Dr. Martin Kollasch, who served as Executive Vice President, is no longer with the organization." Dr. Kollasch was relieved of his position in May, after the problems with the March exam. Whether the timing of Dr. Kollasch's departure was related to the exam fiasco remains unclear.
Symptomatic of Bigger Board Issues to Address?
The computer problems experienced in the March and June exams are not the only challenges faced by the National Board regarding its testing protocols. The NBCE routinely provides test accommodations to students – everything from requests for exceptions for religious convictions to accommodations for disabilities. Prior to the June exam, the NBCE sent copies of all test accommodation schedules to all students who had requested accommodations. This means every student who requested a test accommodation received the accommodation confirmations and schedule of every other student who requested an accommodation. At the very least, this was an embarrassment to these students; at the worst, a violation of their privacy.
Q&A: Student Questions and Board Responses
To make matters worse, the students were left with little understanding of how these issues would ultimately affect them. When contacted by Dynamic Chiropractic, students clearly had many unanswered questions, which we submitted to Dr. Morin for his response. Here are just some of their questions, along with Dr. Morin's responses. (Note: The complete list of questions submitted by students, and Dr. Morin's responses, appear online at www.dynamicchiropractic.com/testing.)
Q: How is it fair that some students (West Coast) did not have to endure the stress of having the servers crash multiple times and then restart on paper?
A: "We are aware that a less than ideal testing experience can cause stress. It is also correct that we alerted the test sites on the West Coast of the decision to transition to paper-and-pencil testing before they began their exams. The NBCE exam interruption policy states that the NBCE may convert to a paper-and-pencil version in the event of a technology malfunction, and this is what occurred during the June administration. The NBCE will offer a free retake to any examinee who experienced an adverse testing event that results in a non-passing score."
Q: Why were we not issued any explanation or apology for the incidents that occurred during the June testing period?
A: "The focus of our current communication is on the affected examinees. Our goal is to assist each individual in evaluating his/her specific situation and addressing needs on a case-by-case basis. At the test sites, administrators also directed examinees to visit a website where the NBCE has been posting information pertaining to the June exam malfunction."
Q: Why was no course of action set in place to ensure a smooth transition from computer-based testing to paper testing for an already proven faulty system?
A: "The NBCE did have a pre-determined contingency plan in place that allowed us to quickly transition to paper and pencil as soon as we verified that the CBT (computer-based testing) exam was not functioning properly. The contingency plan involved all the necessary test site and NBCE headquarters personnel – from the president of our board to the internal IT department in Greeley. At each test site, affected examinees were immediately informed of the CBT malfunction and were given options for continuing at that time with paper and pencil or returning in September for a free retake. NBCE's commitment is to ensure a fair and equitable exam administration regardless of the delivery format."
Q: With the future of many aspiring health care professionals hanging in the balance, why was the NBCE not concerned with offering any type of explanation?
A: "The NBCE communicated the status of CBT in a letter to the profession. In a later communique to the colleges, we informed them that the September and October exams will be in paper-and-pencil format only. We also explained that we will not resume CBT testing until we are confident of success. We are using the events of the June malfunction to evaluate our CBT strategy as a whole and develop alternatives for the future, and we will communicate our future CBT delivery strategies with the profession."
What is immediately obvious in this exchange between the students and the NBCE is the consistent lack of communication to the students. President Morin's statement that "the NBCE communicated the status of CBT in a letter to the profession" is not supported by the facts. While the NBCE may have communicated this information to the colleges, it did not communicate with the students, as the students asking the questions clearly were unaware of the information; nor did the board send the information to the chiropractic media or even place it on the NBCE website. The board's second-quarter progress report is also not available on its website as of press time – or at least extremely challenging to find.
Lingering Questions
The NBCE is one of the wealthiest organizations in the chiropractic profession, with assets of almost $30 million. The board takes in more than $11 million each year from student exams and is usually able to pocket over a million of that after expenses. One would think that with abundant resources, the NBCE could execute computer testing in an era in which the average person can do almost anything on their cellphone. This suggests the challenges facing the NBCE may not be financial, but rather internal – a contention that has been made (and validated) on several occasions over the years.
Lacking any competition, has the NBCE effectively grown fat and lazy, with little effort put into innovation? These recent issues regarding computerized testing have unleashed a flurry of questions by chiropractic leaders:
- Why are prospective DCs still taking and paying for four different exams? Couldn't they be combined into two? Wouldn't this reduce the cost of the exams from $3,340 to around $2,000?
- Why a board exclusively comprised of doctors of chiropractic? Why not reduce the number of DCs and add a leader in the testing industry, an educational specialist, an IT expert, etc., to get the expertise at the decision-making level?
- Will the board prepare the NBCE for tomorrow by hiring an outside expert who has 20-plus years in the testing industry or choose someone who will keep the status quo? Why not bring in the best and the brightest, with tenure in other industries, to serve as new executive vice president?
- Considering the challenges related to online testing, is it time to utilize an independent testing organization, rather than reinvent what the board has already done (with all its obvious limitations)?
- With more than $20 million in cash and investments, why isn't the NBCE investing more in the chiropractic profession? Is that $1 million-per-year surplus being put to its best use?
By all counts, these are the types of issues the NBCE should be addressing. The board's current problems may be mere symptoms of greater ailments. The fate of our future doctors depends on a strong, well-managed, innovative board. This will likely require a greater focus on the future and perhaps even an eventual change in the leadership – which longtime readers will recall is not the first time such a transition has been required.
Editor's Note: Search on our website using the keywords "national board reform" to learn more about previous NBCE organizational issues and resulting board changes.
Dynamic Chiropractic editorial staff members research, investigate and write articles for the publication on an ongoing basis. To contact the Editorial Department or submit an article of your own for consideration, email
.