|
|
|
|
|
|
|
|
A Quinn,MA Corrigan,J Broderick,G McEntee,ADK Hill
|
|
|
|
|
A Quinn1, MA Corrigan1, J Broderick2, G McEntee2, ADK Hill1
1Department of Surgery, RCSI, Beaumont Hospital, Beaumont, Dublin 9
2Department of Surgery, Mater Misericordiae University Hospital, Eccles St, Dublin 7
Abstract
Following the implementation of the Fottrell report, entry to medical school in Ireland has undergone significant change. Medical school studentship is now awarded based on a combination of points obtained from the final examination of Irish secondary schools (the leaving certificate) combined with HPAT scores (Health Professions Admissions Test). The HPAT is designed to test a candidate’s knowledge in several different fields including problem solving skills, logical and non verbal reasoning. A sample HPAT was administered to a test group composed of consultant surgeons, non consultant hospital doctors, and medical students. Statistical analysis was performed and no significant difference was found between the performances of the groups. This is surprising as it was expected that groups with greater experience at medical problem solving would have translated to higher scores. This exposes a flaw within the HPAT system and a potential weakness in the process of doctor selection.
Introduction
Entry to Irish medical school is subject to immense competition. Until 2008 studentships have been awarded based on performance in the final examination of Irish secondary schools (the leaving certificate). Points for entry into medical school have traditionally been in the range of 92-97% of maximum achievable points. However, as part of the review of Irish medical schools1,2, it was concluded that selection based solely on academic achievement was not the optimal way of choosing future doctors. Internationally, a variety of assessments are made of a candidate looking to secure a place in medical school. Entry is awarded based on a combination of interview, performance in clinical aptitude testing, and academic honours. In 2009, the HPAT Ireland was introduced, as part of the competition process. The HPAT is purported to test a candidate’s ability to reason, think logically and understand the thought process, emotions and intentions of other people. These are skills which a person acquires over time. Currently HPAT Ireland advises candidates to familiarise themselves with the outline of test questions, and to read widely. It does not advocate attending specific courses designed to maximise HPAT scores as it is believed that this is not a test that a candidate could study for and therefore improve.
If HPAT is a true measure of problem solving skills, combined with the ability to reason and understand human behaviour, and if in turn this reasoning is truly applicable to the practice of medicine, than consultants should outperform medical students. We sought to determine if a group of hospital consultants would perform better than medical students in a HPAT examination.
Methods
A standardised HPAT examination was administered to a group comprised of medical students, surgical NCHDs and consultants. Prior warning was not given to participants and examinations were administered simultaneously under uniform conditions. A total of twelve questions were administered, with one minute per question. Answer papers were graded by a single examiner. Results were inputted in spreadsheet format and subjected to t-table and Chi-squared analysis using Minitab version 13.32.
Results
In total 222 candidates sat our modified HPAT at five separate centres across the country. Centres were located in Beaumont Hospital, Our Lady of Lourdes Drogheda, St Lukes Hospital Kilkenny, Waterford Regional and Graduate Entry Medicine, RCSI in Sandyford Dublin. In total there were 9 consultants, 29 NCHDs, 105 medical students sitting the traditional five year programme, 60 first year graduate entry students (GEP) and 18 graduate entry students (GEP) now in their final year of medical school. The graduate entry students had all previously sat a clinical aptitude test to gain a place on the programme. Students in the five year programme were awarded studentships based on points in leaving cert alone. Results were grouped and analysed by a t test. Raw data is shown in Table 1. There were no statistical differences between the groups. However when looking at the average performances, the graduate entry programme students, irrespective of their year of study, out perform other groups, with scores of 7.31 and 8.31( first year and integrated). The consultant group scored marginally better on average than the five year medical programme. Non consultant hospital doctors
achieved the lowest scores with an average of 6.36.
Discussion
The training of a doctor is a lengthy and expensive process. Funding for training is derived from a number of different sources depending on the medical school. However, irrespective of the college, there is a significant investment on the part of Irish tax payers. Clearly it is of paramount importance that the most suitable candidates are selected for training. Originally, entry into any course at third degree level was dependent on achieving two honours at leaving certificate and the ability to afford tuition fees. With the evolution of the leaving certificate examination, and points being awarded for grades achieved in subjects, the points race emerged. Only top performing students in the leaving cert secured a place to study medicine. The system was transparent, could not be tampered with, and rewarded academic excellence. There is evidence indicating that academic achievement does correlate with performance during undergraduate medical education2. The high points necessary were obtained across a range of subjects which had to include maths, science and language skills. Students performing well in all these areas were clearly diligent, organised and hardworking. These are traits which are necessary as doctors3.
However, they are not the only key skills necessary to be a doctor. One of the criticisms of selection singularly based on leaving certificate points alone, was that it excluded many students who had inherent personality traits which would give added value to their worth as a doctor. Hence the introduction of aptitude tests evolved, and in February 2009, students seeking to gain entry to medical school in Ireland sat the HPAT Ireland. The introduction of the aptitude test has also given rise to changes in the scoring system for points achieved in the leaving certificate. Points are awarded as before up to 550 (from a total of 600). Thereafter every five points accrued under the previous scoring system is now worth 1 mark. So a candidate previously scoring 555 points now gets 551, and a candidate previously scoring 600 points now gets 560. To be eligible for a place in medical school, a candidate must achieve 480 leaving certificate points as a minimum. The HPAT scores are worth a maximum of 300. The leaving certificate and HPAT points are combined to give the score used to assign spaces.
The aptitude tests are designed and administered the Australian Council for Educational Research (ACER), an Australian company specialising in education. ACER have extensive publications relating to many diverse aspect of education particularly relating to primary and secondary school level4. ACER state that tests are designed and developed to rigorous professional standards5. There is no specific information available for students as to what exactly will be tested in the HPAT other than “general skills and abilities developed over the course of your education so far”6. The performances of our groups must be interpreted with caution, as sample groups are not of equal size. No group had prior knowledge that the test was going to be administered on that morning. Notwithstanding these limitations the trends that emerged are interesting.
The GEP students scored the highest. Their background is different to the five year programme. All students underwent competitive selection to secure a place in medical school. GEP students have all achieved a 2.1 honours degree, which can be in any discipline, in addition to securing highest ranking GAMSAT scores. Therefore their higher ranking in our study could be a reflection of the life skills acquired by these students during their previous degrees or their experience in the work place. However, the previous study for aptitude tests, and even the experience of sitting an aptitude paper before cannot be discounted. It is therefore difficult to deduce whether the HPAT will truly measure innate problem solving and humanity skills, or if these are tests that can also be improved upon by practice. There is some literature suggesting that participation on preparatory courses improved scores in the non verbal reasoning component of the equivalent exam in Australia7. Some sample questions are provided. No syllabus or curriculum is defined.
Furthermore, we were somewhat surprised at the scores of the consultants. One of the central tenants of the HPAT, and all similar aptitude tests is that they function to measure problem solving skills and critical thinking. Some concerns have already been raised as to what exactly the HPAT is measuring8. We would have expected that consultants with years of experience accrued in the clinical, research and academic fields would have out performed students. The ability to interpret data along with the more intuitive “wait and see” decisions, combined with the ability to make a decision based on suboptimal knowledge and change as the situation evolves, are critical determinants in the performance of a doctor. These are skills gained with time and experience, and would be akin to the traits purported to be tested by the aptitude tests. If a true measure of these skills, consultants should clearly have scored higher. The scores achieved don’t reflect this. The consultant group tested was composed of consultant surgeons. Additionally, the NCHDs tested were all working on surgical teams. If there is a trend of people with characteristics suitable for a career in surgery performing less well (be at ever so slightly), there is potential for discriminating against a future generation of top flight surgeons, by not allowing them entry to medical school from the start.
Two of the consultants had scores of 6/12, and one had a score of 5. With the weighting of total points emphasising HPAT performance once 550 points have been accrued, it is unlikely these three would have been offered a place in medical school. Clearly in our test situation our profession would have lost two valuable colleagues had this been the requirement 30 years ago. Similarly as a profession can we afford to loose the medical student with 600 points and a low HPAT score who currently is being denied entry? The breath of specialities necessary to provide health care to the population is vast. Different skills are necessary and more highly valued in respective specialities. A global test of suitability for entry to medicine does not allow for the fact, that junior doctors can find their way into the niche of medicine most suitable for their inherent skills and personality.
We recognise that the previous method of competitive entry to medical school did not necessarily select out the population of best potential doctors. The change in the selection criteria is positive, and emphasis on problem solving, critical thinking and interpersonal skills is desirable in choosing our future doctor. The HPAT is a step forward; however it may not necessarily reduce strain on students sitting the leaving certificate examination. It is after all another exam to prepare for and, may indeed prove a distraction in the preparation for the leaving certificate itself. In addition, grind schools for the HPAT have flourished. This is despite the ideology that this was an exam relatively impervious to grind school technique. Certainly our results would reflect that having studied for, and taken the exam previously, higher scores were obtained. In conclusion, we feel that medicine, surgery and all affiliated therapies and techniques are developing at an exponential rate. The public’s perceptions of standards of care are increasing. It is important that the evolution of our selection criteria keeps pace with this. The introduction of the HPAT is a positive step; however it is not without its flaws. We wish to emphasise that undergraduate medicine is a broad basic qualification. The traits that are required to be a pathologist, surgeon, or paediatrician are as varied as the traits required for any other profession, and we would caution that the HPAT, as it stands, does not allow for this diversity.
Correspondence: ADK Hill
Department of Surgery, RSCI, Beaumont Hospital, Beaumont, Dublin 9
Email: adkhill@rcsi.ie
References
1. Medical Education in Ireland: A New Direction, Report of the working group on undergraduate medical education and training (the “Fottrell Report”) 2006
2. Quigley C, Cunningham A. Review of medical schools in Ireland 2007. A report to the public by the medical council.
3. Ferguson E, James D, Madalay L. Factors associated with success in medical school: systematic review of the literature. BMJ 2002; 324:952-957
4. Benbassat B, Baumal J. Uncertainties in selection of applicants for medical school. Adv Health Sci Educ Theory Pract 2007 12:509-21
5. www.acer.edu.au
6. www.hpatireland.acer.edu.au/index.php?option=com_content&view=article&id=50&Itemid=35
7. www.hpat-ireland.acer.edu.au
8. Griffin B, Harding D, Wilson I, Yeomans N. Does practice make perfect? The effect of coaching and retesting on selection tests used for admission to an Australian Medical School. The Medical Journal of Australia
9. Murphy JFA. Irish Medical School Entry Criteria have changed. IMJ 2009 102:176
|
|
|
|
Author's Correspondence
|
|
No Author Comments
|
|
|
Acknowledgement
|
|
No Acknowledgement
|
|
|
Other References
|
|
No Other References
|
|
|
|
|