Admission to medical schools in Sri Lanka : predictive validity of selection criteria

1Faculty of Medicine, University of Kelaniya; 2Faculty of Medicine, University of Ruhuna; 3Faculty of Medicine, University of Peradeniya; 4Faculty of Medicine, University of Jaffna; 5Faculty of Medicine, University of Colombo; 6Faculty of Medical Sciences, University of Sri Jayawardenepura, Sri Lanka. Correspondence: NR de S, e-mail: <nrdes@sltnet.lk. Received 5 July 2005 and accepted 24 September 2005. (Competing interests: none declared). Abstract


Introduction
Finding the right policy for admission to medical school is a balancing act: the selection process needs to be fair to society, by choosing people with the potential Admission to medical schools in Sri Lanka: predictive validity of selection criteria NR de Silva 1 , A Pathmeswaran 1 , N de Silva 2 , JS Edirisinghe 3 , PVR Kumarasiri 3 , SV Parameswaran 4 , R Seneviratne 5 , N Warnasuriya 6 and HJ de Silva 1   (Index words: Medical education, medical school admissions, predictors, selection) to be good doctors; it also needs to be fair to applicants, who are often a diverse group unified only by their common desire to become doctors [1].The policy for admission to medical schools in Sri Lanka has been primarily driven by the need to be fair to applicants because the admission process is fiercely competitive.
The importance of choosing students who have the potential to be good doctors has been repeatedly emphasised [1][2][3][4].Previous academic performance can be a good predictor of achievement in medical training [5], but a recent study in one medical school in Sri Lanka found that the aggregate score obtained at the GCE Advanced Level examination (the sole measure of academic performance used for selection of medical students) was a poor predictor of success, accounting for only 2-5% of the variance in performance in that school [6].The study was, however, confined to a single medical school and it did not include certain factors that could be of importance in predicting outcome.The present study was done to examine the predictive value of the A Level examination results (aggregate score and examination attempt) across the complete range of students admitted to Sri Lankan medical school.

Study sample
The study sample consisted of all students who were admitted to all six medical schools in two consecutive years.The process by which these students were selected has been described previously [6].

Entry point factors
The following data were obtained from the admissions database in the University Grants Commission (UGC): student's gender; aggregate marks obtained at the GCE A Level examination; district of residence; admission category (merit or district/underprivileged district); type of candidate (school or private); and faculty of admission.In the absence of UGC records on whether students had been selected for admission on the first, second, or third attempt at the GCE A Level examination, 'type of candidate' was considered as an indirect indicator, since students are allowed only two attempts at the A Level examination as school candidates and those who wish to make third attempt must do so as private candidates.

Performance in medical school
Outcome measures were restricted to those that could be considered common to all six medical schools.The curricula vary but in general, the basic sciences of Anatomy, Biochemistry and Physiology are taught in the first two years of the MBBS curriculum.All medical schools have a summative examination in the second year, which students must pass in order to proceed to the next level of training.Third and fourth year curricula include clinical training and instruction in the paraclinical or applied science subjects, but the manner in which these subjects are evaluated is quite diverse.Finally, all medical schools concentrate on the clinical subjects of Medicine, Surgery, Obstetrics and Gynaecology, Paediatrics and Psychiatry in the final year.Another common feature is that honours are awarded to students who pass all subjects and obtain an average mark of at least 60% in their first attempt at an examination.Data on results obtained by each student in the first attempt at the first summative, and final examinations in medical school, were obtained from the Dean's office in all six medical schools.Students who fell behind their entry cohort after the first summative examination were removed from the study cohort for the final examinations.Examination results were categorised using four outcomes that were common to all medical schools.They were used as categorical variables, and not as continuous variables, because no single set of marks could fully reflect results.For example, a student may obtain an overall mark of 52% in an examination, but still be referred, even though overall pass marks are set at 52% in all medical schools.This is because students also need to score a minimum mark in one component (45% in the theory or clinical component, for example), to pass the examination.Thus, students were dichotomised according to whether they passed the first summative examination at first attempt or not, and whether they passed the same examination with honours or not.Similarly, students were dichotomised according to whether they passed the final MBBS examination at first attempt or not; or passed the final MBBS examination with honours or not.

Correlation of predictors and outcome measures
Multivariate analysis using logistic regression was carried out using the four outcome measures listed above, in relation to GCE A Level examination aggregate score, gender, year of examination, district of admission, admission category, candidate type, and university, as predictors of performance in medical school.The proportion of variation in the outcome that was accounted for by these predictors was assessed using the Pseudo-R 2 statistic.Data entry was done in MS Excel and analysis in Stata Version 7.0.

Entry point variables and success in medical school
A total of 1740 students were selected for admission to medical schools in the period under study: 892 from the 1994 GCE A Level examination, and 848 from the 1995 examination.Both cohorts had more men than women (59% and 57%, see Table 1 the median aggregate score of the Colombo Medical Faculty was consistently higher than in the other medical schools.This is reflected in the fact that the distribution of 'merit category' students was very different between medical schools, but similar in the two cohorts: whereas 100% of admissions to Colombo in both cohorts were merit quota admissions, they were less than 10% in Sri Jayawardenepura and Kelaniya in both years.Private candidates accounted for just over 20% of admissions in both cohorts.Again, there was some disparity in the distribution of these students in the medical schools: Colombo had the least number in both cohorts (8.7% and 13.0%).
The results of the first summative and final examinations are summarised in Tables 2 and 3.At the first summative examination, an average of 39.4% and 39.9% of students obtained ordinary passes in Cohorts 1 and 2, respectively, while 22.7% and 30.9% of students obtained honours passes.At the finals, an average of 45.9% and 43.5% of students obtained ordinary passes, and 34.1% and 33.1% obtained honours in Cohorts 1 and 2, respectively.However, as Tables 2 and 3 show, there was much variation between medical schools, and even within a medical school, between years.

Predictors of success in medical school
Multivariate analysis using logistic regression also showed that of the entry point factors, the only independent predictors of success in all four outcome measures were, being school candidate, female, and having a higher aggregate score (Table 4).Different models were constructed to assess the extent to which these variables could predict success in medical school.In the model that gave the highest coefficient of determination, candidate type, gender, and aggregate score (after adjusting for year, district of admission and university) together accounted for 20% of the variation A subgroup analysis was also done by applying the model to each medical school separately.When the six medical schools were compared, the correlation was highest in Colombo (where all students were 'merit quota') with regard to obtaining honours in the first summative examination, explaining 13% of the variation.Colombo also ranked highest with regard to obtaining honours in the finals, with aggregate score explaining 7% of the variation.With regard to the R 2 value for passing the first summative examination and the finals, Colombo ranked second.
The odds ratios for school candidates to have a better outcome when compared with private candidates ranged from 1.7 for an ordinary pass in the first summative examination to 4.0 for an honours pass in the same examination.The results also indicate that private candidates need from 19 to 26 marks more in the A Level aggregate score to do at least as well as school candidates in medical school.The odds ratio for female students to have a better outcome when compared with male students, ranged from 1.8 for getting a class in the first summative examination to 2.3 for passing the final examinations.The odds ratios for a higher A Level examination aggregate score to result in better outcome were 1.04 for an ordinary pass in the first summative examination or the finals; 1.06 for honours in the finals; and 1.08 for honours in the first summative examination.The aggregate score thus appears to be a marginally better predictor of obtaining honours rather than an ordinary pass in both the first summative and the final examinations.

Discussion
For each of the four measures of success in medical school, we found only three independent predictors: candidate type, gender and aggregate score.In all four outcome measures, candidate type had the highest odds ratios, suggesting that school candidates are much more likely to have a better outcome than private candidates (who constitute about one fifth of admissions).The difference was such that private candidates needed 19-26 marks more in the A Level aggregate to do at least as well as school candidates.Assuming that school candidates are all in their first or second attempt at the A Level examination, and that private candidates are in their third, this would indicate that examination attempt is a strong predictor of success in medical school.This finding is consistent with previous reports from Peradeniya and Sri Jayawardenepura universities [7].In countries such as the UK, the USA and Australia, applicants to medical school are rarely considered if they have failed to gain admission to medical school at the first attempt, and are hardly ever considered for a third time.
Although we found a correlation between A Level aggregate and success in medical school, these data show that at best, the A Level aggregate score on its own accounts for only a small proportion of the variability in medical school examination outcomes: 1-2% of the variability in obtaining an ordinary pass and 5-7% of the variability in obtaining honours.This is similar to the predictive value of 2-5% found in the previous study from Kelaniya University [6].However, even the highest predictive value in a subgroup (13% of the variation in honours at the first summative examination in Colombo) was very much lower than the 23% found in the metaanalysis carried out by Ferguson and colleagues [5].
The A Level aggregate score is a better predictor of the chances of obtaining honours than an ordinary pass.If striving towards improving the quality of medical graduates means increasing the percentage of students who obtain classes, rather than only increasing the pass rate, then selecting students with higher A Level aggregates than at present should improve undergraduate performance.However, this improvement is likely to be minimal, because the aggregate score predicts so little of the variation in outcome.The admission category per se was not an independent predictor of success, probably because it is determined by the students' aggregate score.District of admission was also not a predictor of success.
The issue of how to improve the process of selecting medical students in the UK has been debated extensively in the recent past [1,2,[8][9][10].There appears to be general agreement that although many medical schools value pre-admission academic scores at the expense of other considerations, future doctors should be selected on wider criteria than scores of academic success [1].

Conclusions
The data presented here suggest that the sole academic criterion currently used for selecting medical students in Sri Lanka is a poor predictor of success in medical school.Significant improvement in the quality of medical graduates through a change in admissions policy will probably require a comprehensive change in the tools used for selection.Until such a change takes place, restricting the number of attempts at the selection examination to two, and increasing the merit quota may bring about limited improvement.

Table 2 . Results of first summative examinations by medical school and student cohort
).The overall median aggregate score was 285; it was very similar in both cohorts: 284 for Cohort 1 and 285 for Cohort 2. However,

Table 4 . Significant independent predictors of success after adjusting for district, year of admission, and university
a Additional marks needed by a private candidate to perform as well as a school candidate b Values within brackets indicate R 2 for model without A Level aggregate score c Values > 0.05 indicate good fit Papers