• Version
• Download 113
• File Size 626.28 KB
• File Count 1
• Create Date August 2, 2018
• Last Updated August 2, 2018

Indonesian Achievement Scores for TIMSS 2003: Effects of Changing The Score Estimation Model

The Trends in International Mathematics and Science Studies (TIMSS), like other international assessment studies, uses a complex scaling methodology to produce population-orientated scores for participating countries. Based on item response theory (IRT), the plausible value methodology combines test information with contextual variables. This procedure enables estimates to be produced for each student providing at least some achievement or contextual information is available. Some researchers view that the combination of contextual information with achievement data to produce population measures as controversial. It is often argued that providing the assessment information dominates the scaling model, and that the plausible-value estimates are superior to other IRT measures. In this study, Indonesian mathematics data from TIMSS 2003 are used to investigate the importance of assessment data in the student plausible values.The scored mathematics data from TIMSS, published item parameters and commercial IRT software were used to produce maximum likelihood (MLE), Warm’,s maximum likelihood (WML) and Expected A Posteriori (EAP) estimates for each student. The EAP estimates were produced using a number of priors, including uninformative and various normal informative priors. In some cases, the maximum likelihood and Warm’,s maximum likelihood procedures failed to give student estimates under a number of conditions.We describe the model-fit plots of the theoretical item response curves against the reported scores and the WML estimates, report the MLE, WML, and various EAP averages nationally, and delineated the weighted percentage of students with missing MLE estimates for each TIMSS test booklet. In mathematics-focused books, the percentage of students not receiving valid MLE scores was trivial. However, the percentage of students without scores became non-trivial for science-focused books with relatively few mathematics items. Results showed that significant model misfits occurred on Indonesian students with atypical response patterns, especially for the low-scored students with no apparent pseudo-guessing behaviour.

Attached Files