Do Test Accommodations Accommodate?
One of the most frequently requested accommodations sought by people with learning disabilities is extended time provisions during tests. For example, 97% of the accommodations requested on the MCAT (Medical College Admission Test), for non standard administration, involve extended time. Advocates say this provision "levels the playing field” for people with learning disabilities and attentional problems. They claim that although speed is a factor on such tests, “speededness" is not an important attribute for the practice on medicine.
On the other hand critics, claiming that such a provision changes the nature of the examination and offers an unfair advantage to some medical school applicants, ask ominously, “Would you go to a doctor who can’t read fast enough to pass an entrance examination?” The advocates reply that the flagging of results when tests were taken under “non standard conditions” (i.e. with extended time accommodations) stigmatizes an individual's chances for admission. Thus some people with learning disabilities hesitate to request this accommodation, to which they are entitled.
A recent study of the impact of extended time on MCAT scores (Julian, et al. 2004), conducted by a group of medical educators, sheds some light on a situation that here to for has generated only heated arguments. This well-designed study with a large sample (300,281 MCAT test records), clear objectives, logical statistical procedures, and thoughtful consideration of results, provides a model that may lead to resolution of issues surrounding accommodations in testing. This work may have implications not only for medical school admissions but also for the host of other admissions and licensing examinations for which extended time accommodations are regularly requested and sometimes casually recommended.
MCAT records for 29,788 applicants who took the test under standard conditions and for 2,401 applicants who took the test under flagged (non standard) conditions were examined to delineate similarities and differences between examinees in the standard and non standard administration groups. The following research questions were answered:
How many applicants requested accommodations?
During the years 1994 2000 less than 1% (2,401) of the examinees took MCAT under non-standard time conditions. Reasons for accommodations were LD (55%); AD/HD (17%); both LD and AD/HD (5%); other disabilities, such as vision, hearing, mobility, and chronic illness (23%).
What differences existed between scores flagged and unflagged scores?
Accommodated examinees achieved higher average scores on all sections of MCAT than those who took the test under standard conditions. This difference was small, but it was statistically significant.
What differences were found for examinees who took the test under both standard and non-standard conditions? Doesn’t everyone gain taking any test the second time?
A group of 866 examinees took MCAT first under standard and subsequently under non-standard conditions. Comparison of scores for this group showed a gain of 6.2 points overall, yielding an effect size of 1.4 (effect sizes of .3 or more are considered statistically significant (i.e. not due to chance) and generally considered to indicate practical significance between the two sets of scores). These investigators studied the scores of examinees who took the MCAT twice, both times under standard conditions. The mean gain between the first and second administrations of MCAT under standard conditions yielded an effect size of .4 (which barely meets the criterion for statistical significance). It would seem, with people who do not have learning disabilities and take the test under standard conditions, that the differences between initial and repeated testing are negligible.
What conclusions can be drawn from these findings?
Julian et al. concluded that the higher scores on non standard administration earned by the accommodated examinees could indicate either 1) appropriate compensation or 2) overly generous accommodations. However, because of the large increase in scores for the accommodated group in contrast to the meager gains for the standard group on repeated testing, the importance of accommodations for people with disabilities is supported by the data.
These investigators are currently studying the effects of flagging on admissions decisions by medical schools by comparing the proportions of students with flagged scores to admission rates for students who took MCAT under standard conditions.
Finally, they also point out that, although the power of the MCAT taken under standard conditions to predict success in medical school has been well documented, research is lacking regarding the predictive success of the MCAT taken with accommodations. These investigators are currently engaged in studying the outcomes in medical school and in licensing examinations for applicants who received accommodations prior to medical school admission.
References
Julian, E.R., Ingersoll, D.J., Etienne, P.M., and Hilger, A.E., (2004) The impact of Testing accommodations on MCAT scores: Descriptive results. Academic Medicine, 79, 160-164.
Rosa A Hagin, Ph.D.
|
|
|
