Should the HPAT exam continue to be used as a means of evaluating medical students?

Kevin O’Malley.

University College Dublin

[email protected]

Introduction
The Health Professions Admission Test (HPAT) was introduced in 2009 on the grounds of the 2006 Fottrell report1. Recommendations from the report aimed to diversify the pool of incoming students and afford greater access to medical education by expanding alternative entry routes.

Health Professions Admission Test (HPAT)
The HPAT is a two and a half hour, multiple-choice style examination comprising three sections: logical reasoning and problem solving; interpersonal understanding; and non-verbal reasoning. HPAT scores are combined with Leaving Certificate points with highest combined scores gaining entry to medical school.

The merits of cognitively-oriented aptitude assessment are many, primarily it provides objective evaluation of ability and removes (or at least reduces) the role of rote learning. Moreover, such measures demonstrate reduced sensitivity to socioeconomic variability in candidates compared to the Leaving Certificate.1,2.  Aims to defuse the infamous “points race” have been relatively successful. Admission to medical school is no longer solely reliant on achieving top grades in all subjects. Retrospective analyses have shown that significant numbers of successful applicants would not have secured places at medical school were entry based on Leaving Certificate results alone3. In addition, prospective students are significantly less likely to repeat the Leaving Certificate exam since the introduction of HPAT. Nevertheless, the leaving cert still represents a considerable hurdle for applicants: In spite of the 480 point minimum threshold (90th national percentile), the successful applicants of 2016 all obtained points ranging 535-625 (placing them in excess of the 95th national percentile).

In the cohort of students who have been admitted under the revised system, research has shown a small gender difference: where previously females outperformed their male counterparts the original HPAT exam appeared to slightly favour male candidates.  Adjustments to the weight distribution of each section appear to have addressed this effect. However, candidates from lower socioeconomic or disadvantaged backgrounds were previously underrepresented and continue to remain so4.

Regarding the exam itself, the content and means of assessment are of debatable relevance to medical education and professional life thereafter. In particular, critics question the relevance of the non-verbal reasoning section to performance at medical school or clinical competence. 2 It has also been suggested that this section may be targeted by groups offering preparatory courses as participation in such workshops has demonstrated potential to achieve a higher score5.

One study compared performance of medical students with that of non-consultant doctors (NCHDs) and consultant surgeons in a modified HPAT exam. Contrary to expectation, consultant doctors performed only “marginally better” than undergraduate students whilst NCHD’s attained the lowest scores of the 222 participants5.  This finding led the authors to question what exactly the HPAT is measuring. One would assume the doctor groups would outperform the students, having accrued and consolidated skills in critical thought and problem-solving over many years.

Alternatives?
To argue for the discontinuation of the HPAT is to suggest that a better (feasible) alternative option exists. Academic performance at secondary school has been suggested over the years and has been shown to be a good indicator of performance at medical school. Psychometric testing of emotional intelligence and personality have also been found to be useful, however those traits which predict success in medical school do not necessarily translate to post-graduate performance and competence. There is little evidence supporting the use of personal statements or character references, both of which demonstrate poor validity and reliability. Similarly, interviews are subjective and prone to bias, particularly in such a small country as Ireland2.Graduate entry programs have been introduced in Ireland with notable success however many entrants have been forced abroad to repay colossal debts accrued to complete their education.

Conclusion
Incorporation of the HPAT exam to the selection process appears to be the most objective and fair means of expanding access to medical school, with little if anything to be gained from its eradication. However the question of whether or not it should be continued may be flawed if the question pertains to the suitability and ultimate calibre of doctor it selects. If the aim is to identify students with the mental wherewithal to see a medical degree through to completion then arguably the test is sufficient. If the aim is to identify students who will progress to be well-rounded, competent and compassionate doctors, then the problem becomes a more difficult one. Simply because a test score dictates a person could do medicine, does not necessarily imply that they should.

References

1. Education WGoUM. Medical Education in Ireland: A New Direction (The Fottrell Report). Dublin:, Department of Health; 2006. Report No.: Available at: http:// health.gov.ie/blog/publications/medical-education-in-ireland-a-new-direction.
2. Patterson F, Knight A, Dowell J, Nicholson S, Cousans F, Cleland J. How effective are selection methods in medical education? A systematic review. Medical Education. 2016; 50.
3. O'Flynn S, Fitzgerald T, Mills A. Modelling the impact of old and new mechanisms of entry and selection to medical school in Ireland: who gets in? Irish Journal of Medical Science. 2013; 182(3).
4. O'Flynn S, Mills A, Fitzgerald T. National research group evaluating revised entry mechanisms to medicine: Interim report- School leaver entrants. ; 2012.
5. Quinn A, Corrigan MA, Broderick J, McEntee G, Hill ADK. A comparison of performances of consultant surgeons, NCHDs and medical students in a modified HPAT examination. Irish Medical Journal. 2010; 103(6).