Selection of medical students in Sri Lanka : time to re-think criteria ?

Objective To examine the validity of selected entry level characteristics in relation to the GCE A/L examination as independent predictors of performance of students in medical school. Methods A retrospective, analytical study was done at the Faculty of Medicine, University of Kelaniya. Student characteristics at entry were described by sex, the average z-score, General English grade and attempt at GCE A/L examination, and average mark obtained at the English placement test on registration to medical school. Average marks at three summative examinations were Selection of medical students in Sri Lanka: time to re-think criteria? S N Hewage1, L S S Salgado2, G M O Fernando1, P L C K Liyanage1, A Pathmeswaran3, N R de Silva1 (Index words: medical students, selection criteria) 1Medical Education Centre, 2Department of Anatomy and 3Department of Public Health, Faculty of Medicine, University of Kelaniya, Sri Lanka. Correspondence: NRdeS, e-mail <nrdesilva@gmail.com>. Received 24 March and revised version accepted 3 July 2010. Competing interests: none declared. used as indicators of performance in medical school. Multivariate analysis using multiple linear regression was carried out using these three outcome measures in relation to four entry point variables as predictors of performance in medical school. Causal path diagrams were constructed using standardised regression coefficients for the whole group and for male and female students separately. Results The A/L z-score, A/L attempt and English placement test marks were all significant predictors of outcome at the First Examination. Of the variables relating to the A/L examination, the attempt had a much higher


Results
The A/L z-score, A/L attempt and English placement test marks were all significant predictors of outcome at the First Examination.Of the variables relating to the A/L examination, the attempt had a much higher

Introduction
The annual selection of new entrants to Sri Lanka's eight state-funded medical schools is centrally administered by the University Grants Commission, with a policy that has remained basically unchanged for several decades.Selection is based on two criteria, one academic and one non-academic.Academic ability is measured by a single z-score, derived from marks obtained in the subjects of Biology, Chemistry and Physics at the General Certificate of Education, Advanced Level (GCE A/L) examination.Students seeking university admission are allowed up to three attempts at the GCE A/L examination, and no distinction is made between those taking the examination for the first, second or third time.The single non-academic criterion is the student's district of residence.At present, 40% of places are awarded on the sole basis of the GCE A/L z-score ('all-island merit'), and the rest is divided among the 25 districts of Sri Lanka ('merit on district basis').Selection of students to fill the places allocated to the districts is also based on the zscore at the GCE A/L examination.
There is intense competition for admission to medical schools in Sri Lanka so it is of vital importance that selection criteria are fair by all applicants.It is equally important, however, that the selection criteria are effective in identifying those applicants who are most likely to become good doctors.Validating selection criteria in this way is a difficult task, since it requires long term follow up and tools to identify 'good doctors'.However, medical school examinations try to evaluate the knowledge, skills and attitudes that a good doctor should possess, and these examination results are relatively easily accessible.Hence examination results have frequently been used as a means of validating criteria for selection of medical students [1].
Previous studies in Sri Lankan medical schools have suggested that the A/L aggregate score is a weak predictor of success in medical school [2], and that other variables such as the attempt at the GCE A/L examination and proficiency in English may also be used as predictors of outcome in medical school examinations [3,4].Since these studies were carried out, the curricula for the GCE A/L examination has changed, and the means of ranking students for selection to university has also changed, from aggregate marks to z-scores.Students also have to take a paper in General English as well as a Common General paper.
The objective of this study was to examine the validity of selected entry level characteristics in relation to the GCE A/L examination as independent predictors of performance of students admitted to the Faculty of Medicine, University of Kelaniya.

Methods
This was a retrospective, analytical study that examined the relationship between several variables relating to student characteristics on admission to medical school (including A/L examination attempt), and their success in medical school as measured by performance at medical school examinations.

Data collection
The following variables were used to describe student characteristics at entry: The following variables were used as indicators of performance in medical school (outcome measures) 1.Average mark obtained on the first attempt at the First Examination for Medical Degrees.This was a simple average, as calculated for the award of classes in the First Examination, which is conducted at the end of five terms of study.
2. Average mark obtained on the first attempt at the Second Examination for Medical Degrees.This was a weighted average, as calculated for the award of classes at the Second Examination.This examination is conducted in two parts, at the end of the 3rd and 4th years of study.
3. Average mark obtained on the first attempt at the Final MBBS examination.This was a simple average of five subjects: Medicine, Surgery, Obstetrics and Gynaecology, Paediatrics and Psychiatry.
4. The A/L z-score used was that sent to the Faculty by the UGC on admission of students.Data on exam attempt and General English grades were based on information provided by students at admission.Data on sex, English placement test Papers score and marks obtained at the First, Second and Final Examinations were obtained from the Dean's office.Data collection and entry has taken place systematically into a database, on a routine basis, from the point of registration in the medical school onwards.
Data collection was restricted to two consecutive batches of students, the first admitted to the Faculty in January 2004 (cohort 1, 168 students) and the second in January 2005 (cohort 2, 164 students).These students were admitted on the basis of the GCE Advanced Level examinations held in April 2003 and April 2004, respectively.

Statistical analysis
Multivariate analysis using multiple linear regression was carried out using the three outcome measures described above, in relation to four entry point variables as predictors of performance in medical school.Data analysis was done on SPSS and Stata.
Causal path diagrams were constructed using the standardised regression coefficients for the whole group and for male and female students separately.The A/L attempt, A/L z-score and the English marks were taken as exogenous variables the marks obtained at the first, second, and final examinations were taken as endogenous variables for the path analysis.The A/L attempt was used as a numeric variable, taking the values 1, 2 and 3 for 1st, 2nd and 3rd attempts.

Ethics
Ethical clearance was obtained from the Ethical Review Committee of the Faculty of Medicine, University of Kelaniya.

Entry characteristics
Of a total of 332 students in the two cohorts, 294 students (88.6%, 153 from cohort 1 and 141 from cohort 2) were included in this study for the First and Second Examinations.Only 282 students were included in the analysis with regard to the Final MBBS examinations because of attrition at the First and Second examinations.Reasons for exclusion were: no z-score (8 foreign students); no data on A/L attempt (6 students); no English placement test marks (13 students, late registrations); no exam marks (11 students).No data were available on performance in General English at the GCE A/L examination for 34 students who were included in the analysis.
Overall, 58.8% of the study population was female.Most students had been selected for admission following the 2nd attempt at the GCE A/L examination: 48.3% from the 2nd attempt, 32% from the 1st attempt, and 19.7% from the 3rd.Among the 260 students for whom A/L General English grades were available, 26 (10%) had failed the paper; 51 (19.6%) had a simple pass; 88 (33.8%) had a C grade; 51 (19.6%) had a B grade; and 44 (16.9%) had an A grade.The mean English placement test mark was 59.1 (SD14.2).There was a strong correlation between the results of the General English paper at the A/L examination, and the English placement test scores (Pearson correlation = 0.701, p<0.001).Since English placement test scores were available for more students than the General English grades, the former was used in the subsequent analyses.Comparison of the entry characteristics of the two student cohorts did not show any statistically significant differences.

Outcome variables
Mean marks in the First, Second and Final Examinations were 54.3 (SD 7.3), 56.5 (SD 7.6) and 58.4 (SD 4.6) respectively.Mean marks obtained at the First and the Final Examinations for Medical Degrees were not significantly different between the two cohorts, but in the Second Examination, cohort 1 had a higher mean than cohort 2 (ANOVA F statistic=10.921,p=0.001).
Table 1 compares the entry characteristics and examination performance of the 121 male and 173 female students included in the study.The mean A/L z-scores were not significantly different between male and female students, but significantly more male students than female were admitted on the results of the 3rd attempt at the GCE A/L examination (27.2% of male students, compared with 14.5% of female students, Pearson chi-square=7.743,d.f.=2, p=0.021).Significantly higher proportions of female students also had A or B grades in General English than male students, while a smaller proportion of females had failed (44.3% vs 22.6%, Pearson chi-square=20.965,p<0.001).Mean scores obtained by female students at the English placement test were also significantly higher as shown in Table 1 (t-statistic=5.333,p<0.001).The mean marks obtained by female students at both First and Second Examinations for medical degrees were significantly higher than those obtained by male students (see Table 1, t-statistics=4.009and 6.416, respectively, p<0.001 for both).

Predictors of performance in medical school examinations
Table 2 shows the mean A/L z-scores, English placement test marks and the mean marks obtained at the First, Second and Final Examinations in medical school, when students are categorized on the basis of the A/L examination attempt on which they were admitted to medical school.A/L z-scores did not differ significantly between students selected from their 1st, 2nd or 3rd attempt at the examination.However, English placement test marks, as well as performance at both First and Second Examinations declined significantly as the number of attempts at the GCE A/L examination increased.

Papers
Results of the overall path analysis (Figure 1) showed that the A/L z-score, A/L attempt and English placement test marks were all significant predictors of outcome at the First Examination.Of the two variables relating to the A/L examination, the attempt had a much higher path coefficient with performance at the First Examination than the A/L z-score (-0.29 and 0.13 respectively).It should be noted however, that the proportion of variation in this outcome variable, explained by the model was only 0.24 (1 -0.872).Although there was high correlation between performance at the First and Second Examinations (0.71), as well as the Second and Final Examination (0.69), A/L zscores were not significant direct predictors of outcome at the Second or Final Examinations.The English placement test marks showed significant positive correlation with both First and Second Examinations (0.28 and 0.20 respectively), but showed a significant negative correlation with the Final Examination marks (-0.11).However, since performance at the First Examination was a significant predictor of performance at the Second, with a path coefficient of 0.71; and the A/L z-score was a significant predictor of performance in the First Examination, the A/L z-score is also a predictor of performance in the Second Examination, with a path coefficient of 0.09 (0.130.71).Similarly the path coefficient for A/L attempt as a predictor of performance in the Second Examination is -0.21 (-0.290.71).
Since there were significant gender-based differences in both entry level characteristics and outcome variables, the path analysis was repeated separately for male and female students (Figures 2 and 3).This showed that the significance of the relationships largely remained the same for both sexes, but the magnitude of the correlation was different.For example, the path coefficient between English and the First Examination was 0.27 and 0.20 for male and female students, respectively.This declined to 0.10 for English and the Second Examination for female students, but remained at 0.26 for male students.Controlling for other variables, students admitted on their 3rd attempt at the GCE A/L examination scored 5.9 marks less than 1st attempt and 3.2 marks less than 2nd attempt students in the First Examination.

Discussion
The results presented here suggest that all four entry level characteristics examined in this study (sex, A/L z-score, attempt at GCE A/L examination and English language proficiency) are significant predictors of performance in medical school for this particular study population.
Gender differences in the performance of medical students (that on average, women tended to perform better than men) have been noted previously, both in Sri Lanka [4,5] and other countries [6,7].It is very likely that the differences between sexes in examination performance are linked to both A/L attempt and English language proficiency.
A greater proportion of male students were admitted after the 3rd attempt at the A/L and our findings show that the attempt at the GCE A/L examination is a significant predictor of performance in medical school, although it is not given any weight in selection of students for university admission.This finding confirms those reported from the Colombo Medical Faculty where a higher percentage of first attempters were found to perform well and obtain classes than the second or third attempters [2].This is despite a considerable difference in the two study populations in terms of entry point characteristics: 57.8% of the Colombo study population was 1st attempters compared with only 32% in this study, and only 7.5% of students in the Colombo study population were after their 3rd attempt, compared with 19.7% in our study population.Similarly, male students comprised 63.5% of the study population in Colombo, compared with only 41.2% in this study population.
Gender differences in English language proficiency were also seen in univariate analysis, both with General English at the A/L examination, and the English placement test conducted in the Faculty.The present analysis also suggests that competency in the English language, as measured by the English placement test scores, is an independent predictor of subsequent performance at medical faculty examinations.Since the English placement test scores correlated well with General English results at the GCE A/L examination, it is likely that General English would also be a significant predictor of performance at medical school examination.The Colombo study, which used the results in English at the GCE Ordinary Level examination as a measure of proficiency in English, did not find it to be a significant predictor.This is probably because the GCE Ordinary Level examination results are not a good measure of the level of English proficiency required in medical school.
At present, in order to be eligible for admission to a course of study in Medicine in a Sri Lankan university, an applicant is only required to have at least 'S' grades in each of the three subjects of Biology, Chemistry and Physics, and to have obtained at least 30% in the Common General Paper.General English was introduced as an A/L subject several years ago, but students are not required to pass this subject (or even sit for the paper), even though Medicine courses are conducted in English in all eight medical faculties.As shown above, several studies including the present, have demonstrated that students who gain admission after their 3rd attempt at the A/L examination fare much worse than those admitted to medical school on their 1st attempt at the A/L examination.
Requiring students who apply for Medicine to have at least an 'S' grade in General English, and restricting the number of attempts to two instead of the present three, will undoubtedly result in the selection of students who are less likely to struggle in medical school, and ultimately make better doctors.

Figure 1 .
Figure 1.Results of path analysis for entire study population (n=294).Numbers against arrows indicate the path coefficients (standardised regression coefficients).e 1 , e 2 and e 3 are the error terms for the respective models.

Figure 3 .
Figure 3. Results of path analysis for female students (n=174).Numbers against arrows indicate the path coefficients (standardised regression coefficients).e 1 , e 2 and e 3 are the error terms for the respective models.

Figure 2 .
Figure 2. Results of path analysis for male students (n=120).Numbers against arrows indicate the path coefficients (standardised regression coefficients).e 1 , e 2 and e 3 are the error terms for the respective models.