University admissions and the Z score

The Island, January 12, 2012, 12:00 pm

Prof. R.O. Thattil

University of Peradeniya

The release of the GCE A/L results has led to many controversial issues regarding the method used to rank the students. The students and parents are in a dilemma whether to accept the results or not. Therefore, I decided to write this article to clarify matters since I was responsible for introducing the Z score as a basis for ranking a decade ago.

History

Admission to universities prior to the year 2000 was based on aggregate marks of 4 subjects obtained at the A/L examination. However, in that year the syllabus was changed and the new system required students to take only 3 subjects. Thus, there were 2 groups taking the A/L examination. One group (old syllabus) took 4 subjects while the other group (new syllabus) took only 3 subjects. The problem then was how to rank students for university admission.


Within a group of students tested for a given subject, the marks received are an indicator of who the better student is. However, the aggregation of marks of different subjects for the purpose of ranking is grossly unfair especially when students sit 2 different sets of subjects.


Distribution of marks is different from subject to subject. It is an exception rather the rule for marks of different subjects to have the same distribution in terms of location (measured by the mean) and dispersion (measured by the standard deviation). Discussions on what was the best way to rank students were held in the year 2000 chaired by the then secretary of Education Prof. R. P. Gunawardana. Many proposals were discussed and finally the proposal submitted by the author for using the Z score method of scaling marks was accepted.

The Z score

The scaling of different variables (some using different units of measurement) has been done using the standardized score Z for many decades. However, this was the first time it was to be used at a national examination. The Z score for a given subject is calculated as


Where X = raw mark of the subject,X = mean of the marks of the subject and S = standard deviation of the marks of the subject.


The Z score thus formed for a given subject is comparable to the Z score of another subject, while raw marks do not posses this attribute. A raw mark of 45 in a subject such as physics is an above average mark while in another subject it may even be below the average or very close to being so. However, a Z score of 1.2 in a given subject is equivalent to a Z score of 1.2 in any other subject. Although mean and variance of raw marks can differ from subject to subject, the Z scores are distributed with a mean = zero and variance = 1 for any subject. Thus, the Z scores of all subjects will have the same location and spread parameters. There is nothing called the perfect index.


Complications are avoided by not assuming any particular distribution for the raw marks. The Z score can be viewed as a scaling technique that makes the location and dispersion parameters for the marks of all subjects the same.

The present problem

A new syllabus was introduced in 2009 for the students sitting the A/L examination for the first time. Thus, there were 2 groups (new and old syllabuses) sitting for the A/L examinations in 2011 (similar to the year 2000!). A committee was appointed to decide on the method of ranking. The author was not a member of this committee. However, the author is aware that they pooled the mean and variances (square of the standard deviations) of a subject from the old and new syllabus. Pooling cannot be done for a subject since the 2 student groups should be considered as 2 populations. The mean and/or the standard deviation of 2 populations will be different and should have been treated separately.

Danger of pooling

Consider the means and variances of a subject for the 2 groups (new and old syllabus)

The standard deviations will necessarily be much lower than the means. The author has taken conservative values for the number of students (n). In some subjects the number can be much higher. I have used the formulas given by the recently appointed committee to obtain the pooled means xp and variances (S2p)


If these values are used to test the hypothesis, whether means of the 2 populations are significantly different or not, we can use the Z test (not to be confused with the Z score) since populations are large. This is a standard test of means of 2 populations.




The Z statistic is much larger than 2, which implies that the population means are significantly different and therefore pooled means cannot be used.


On first glance one may think that the mean differences are not very different. However, even a difference of 1.5 will lead to a significant difference. Now, consider the effect of pooling on the Z scores, for example a student receiving 80 marks in group 1 would have a Z score

if the unpooled mean and variance was used. However, if the pooled values are used the Z score will be equal to




which is higher than the un-pooled Z score. Therefore, students in group 1 will have an unfair advantage over the students in group 2 whose un-pooled mean is lower. In general the Z scores of the group with the higher mean will get elevated while the Z scores of the other group will be lowered. A case of the rich getting richer while the poor get poorer!

Solution

The solution is to consider the 2 groups as 2 different populations and calculate the Z score for each group separately using the (un-pooled) mean and variance for each subject. The average of the Z scores of the 3 subjects can then be taken to obtain the ranks. This solution is simple and elegant.