• Version
• Download 391
• File Size 141.00 KB
• File Count 1
• Create Date July 21, 2018
• Last Updated July 21, 2018

Predicting Final Grade Distribution of Examination Results – Selection of Sample Scripts

The State Examinations Commission is responsible for administering public examinations at the end of lower second-level and upper second-level education in Ireland. Unusually for such examinations, grade boundaries are predetermined and no transformations are applied to raw scores. The raw mark therefore directly determines the grade, and no manipulation of the grade distribution is possible after the raw marks are finalised. For security reasons, examination items are not pre-tested. When scripts are being marked, therefore, early and accurate predictions of the overall distribution are essential, so that any necessary interventions can be made at the raw-mark level, by adjusting marking criteria and revisiting scripts already marked. In order to generate this initial estimate of the final distribution, each examiner marks a sample set of scripts early in the process.

This paper addresses the mechanism used for selecting the sample. It does not deal with the issue of determining an appropriate sample size. It reports empirical confirmation that the mechanism currently recommended to examining teams (systematic sampling) produces better estimates than might be expected from random samples. It also confirms that certain alternative mechanisms previously used by some examining teams will produce biased samples in most circumstances.

Attached Files