Conference papers

Singapore 32nd annual conference
32nd annual conference

Authors	Title	Paper
Zupanc, Darko MSc	(Mis)use of national assessment results in Slovenia Abstract: National assessment of students' knowledge is frequently carried out in school systems throughout the world and different countries participate in international comparative studies such as TIMSS as well. The elementary reporting on results is a demonstration of the achievement of multi-level benchmarks. This article deals with the distribution of results in mathematics of 13 and 14 year-old students in systems of national assessment in Sweden, England, the USA, Canada, Australia and Slovenia, and aims at comparing these with the results of the international study TIMSS 2003. In national assessment Slovenia shows a pronounced discrepancy: the majority of students are very good at mathematics and the number of those that do not attain basic knowledge is negligible. But is the fact that more than one half of the students in Slovenia achieve excellent or very good grades in mathematics in national assessment good or bad? What is the case in other countries? How do other countries compare in their distribution of grades, or levels of achievement in their systems of national assessment? Do international measurements of knowledge confirm this anomaly revealed by school grades and national assessment in Slovenia? Isn't this embellished demonstration misleading and in reality all about lowering standards of knowledge and inflation of grades? The function and use of national assessment results in the last few years have been questionable. Document type: Congress Presentation ISBN:	View
Adeyegbe, S. O. DR.	2 years of E-registration: Keeping pace with it revolution in testing: the WAEC experience Abstract: The key to the continued relevance of any Examination Body with a stature like that of The West African Examinations Council (WAEC) lies in its responsiveness to the changing needs and aspirations of its clientele. With growth and expansion in the operations of WAEC came the need for automation and Council’s efforts in this direction were enumerated. Operating in a technology - dominated era makes it incumbent on WAEC to keep pace with technological advancement. The focus of this paper is the process change resulting from the deployment of Information Technology (IT) to service delivery by Council. The challenges posed to Council by the shift to e-registration of candidates, the benefits of the process transformation and possible expansion of the scope of IT application to Council’s operations in future were highlighted. Document type: Congress Presentation ISBN:	View
Kow, Yeo Kai Joseph	A Critique of Paper-and-Pencil Secondary One Mathematics Semestral Assessment Papers Abstract: Assessment is an essential part of the teaching and learning process. Since assessment plays such a significant part in the educational process, it is imperative that we examine closely the individual questions that make up the assessment paper. This paper is based on a preliminary investigation into about 30 sets of secondary one semestral examination papers. In this paper only a sample of problematic short-ended and structured / long-answer questions have been highlighted and discussed. From this sample, it could be seen that short-ended and structured / long-answer questions are challenging and demanding to design. The questions were scrutinized based on the following criteria involved in the designing of test/examination questions: (1) Questions with incorrect Mathematical Concept, (2) Questions which test trivial facts, (3) Questions with ambiguous verbal communication, (4) Questions with inconsistent symbols and notations, (5) Questions with misleading diagrams, (5) Questions with impracticality of scenarios, and (6) Questions with imbalanced learning objectives. Document type: Congress Presentation ISBN:	View
Kondo, Amiri Rutayuga, Adolf B.	A SHIFT FROM ASSESSING A SET OF LEARNING CONTENTS TO ASSESSING EACH LEARNING OUTCOME: NACTE’S PERSPECTIVE Abstract: Assessment is the general term used for measuring student’s performance on a course against the set learning outcomes of the particular educational programme. When the mode of assessment is not sound, educational programmes suffer and definitely, students are victimised. Assessment plays an important role in influencing a student’s academic prospects, career opportunities and even job success. The same also helps to provide accurate predictions for future academic success or future professional competence. Under the contemporary world, employers are becoming increasingly concerned with ensuring that their employees have up-to-date competencies and capabilities needed in the world of work. Given the increasing importance attached to assessing a wider range of competencies (i.e. knowledge, skills, understanding and wider attributes), it is clear that relying on traditional methods of assessment, is not regarded as appropriate means of responding to an ever-increasing demand to provide relevant, quality, and effective education. This paper discusses NACTE’s Perspective on the shift from assessing a set of learning contents (topics) to assessing every learning outcome in respective curriculum to students in technical institutions in Tanzania, in order to generate high quality graduates who are able to respond to labour market needs. Document type: Congress Presentation ISBN:	View
Tsheko, G. N.	A TEACHER WITH A VISION AND SCHOOL BASED ASSESSMENT Abstract: School based assessment has a major role to play in the education of our children. Children spend the best part of their young lives with teachers and it is through the teacher assessments that we can better understand their capabilities. To rely solely or to a large extent on the one time examinations as is common practice in most countries is to obscure the vision of a teacher and deny them a voice of creativity. The role of school based assessment in the final grading of students in senior secondary schools in Botswana needs to be given an uplifted face. Teachers in senior secondary schools continue to assess students on a day to day basis but the fruits of their hard labour are easily swallowed by the national examinations which are prepared by an external body. This could be due to lack of confidence in the way teachers assess or that the importance of such assessment is not readily seen. In this paper, results of a study in which views of teachers were solicited about the importance of school based assessment in senior secondary schools in the South Central district of Botswana will be shared. Recommendations on what could be done to enhance visibility and usability of school based assessment are presented. Document type: Congress Presentation ISBN:	View
Okumbe, Matthews Adera	ACCOMMODATING LEARNERS WITH SPECIAL EDUCATIONAL NEEDS IN BOTSWANA: A CHALLENGE TO EDUCATIONAL PSYCHOLOGISTS AND SPECIAL EDUCATORS Abstract: The critical issue for special educators and educational psychologists is the effective accommodation modes for learners with special needs, in an inclusive classroom. Accommodation is done with the view of enhancing the performance of these learners and to help them meet individual needs of acquiring necessary input from lessons irrespective of their diverse backgrounds as they interact and socialize with their nondisabled peers. The purpose of this paper is to examine the role of educational psychologists in the accommodation of special needs learners, particularly in the identification and recommendations that will assist special educators. Secondly, it will also discuss the roles of special educators in facilitating the acquisition of skills of special needs learners that will go a long way in having them included not only in school settings, but also in the world of work. It will throw light on the challenges posed to educational psychologists and special educators in helping these learners in inclusive school setting, in Botswana. Lastly, the paper will outline the best practices to accommodate these learners. Document type: Congress Presentation ISBN:	View
Greatorex, Jackie Suto, W. M. Irenka	An empirical exploration of human judgement in the marking of school examinations Abstract: A major theme of our recent research has been the nature and use of human judgement in the marking of school examinations. In an era of innovation and rapid development, it is important to have an understanding of these psychological processes, which have the potential to impact upon modernisation. In this paper, we present an overview of our studies in this area. Working within a popular cognitive psychological paradigm, we explored examiners’ judgements in a number of marking contexts. Both experienced and newer examiners participated in the research, in which a ‘think aloud’ method was utilised. GCSE and A-level examinations were marked, and both paper-based and computer-based marking formats were investigated. We identified five distinct cognitive marking strategies, which were used in all of the contexts considered. Subsequently, a quantitative analysis of strategy usage in the traditional paper format was conducted. We will conclude this paper with a discussion of the potential implications of this research for future examination marking. Document type: Congress Presentation ISBN:	View
MOOI, Lee Choo Ph.D.	Assessing student learning in computer graphics Abstract: Computer Applications (CPA) is a subject designed to nurture students as effective end-users of computer-related technology and software applications for word processing, multimedia presentation, computer graphics, spreadsheet, web browser, desktop publishing and database management. The subject is available to students from the Normal Technical course in Grades 7 – 10 in Singapore. These students are generally visual learners who are best engaged when there are hands-on learning activities and practical work in the computer laboratory. Assessment tools were designed to assess the practical work. For the computer graphics component, students are assessed on their ability to plan, and then select and use appropriate tools/skills to produce a drawing that is as close as possible to a given drawing. The mastery of skills and concepts is evident when the drawing produced is nearly identical (in terms of likeness and size) to the given drawing. The paper describes a qualitative analysis of students’ work and how this analysis provides insights into the way activities could be designed and used to improve classroom instruction and student learning of the computer graphics component. Document type: Congress Presentation ISBN:	View
McGaw, Barry	Assessment Fit for Purpose Abstract: Assessment is a powerful educational tool. It influences the judgements of students and teachers about what is of most importance in the curriculum. The effects can be positive and can justify expectations of assessment-led reform. The effects can also be negative, for students, teachers, schools and the curriculum. It is important, therefore, for those responsible for assessment, particularly high-stakes assessment, to pay attention to the ‘consequential validity’ of their assessment systems. Assessment cannot be seen only as a technical task. Assessment needs to be fit for purpose. The well-established distinction between formative and summative assessment is helpful in clarifying purpose and informing choice about method. The longstanding distinction between norm-referenced and criterion-referenced assessment is also helpful in clarifying purpose but has become less relevant for choice about method because of the capacity of modern psychometric methods to dissolve the methodological distinction between them. Document type: Congress Presentation ISBN:	View
Kwawukume, Victoria Mrs	ASSESSMENT FOR IMPROVING LEARNING Abstract: Present systems of examining, its rationale, functions and development have engaged the attention of researchers and stakeholders of education in this country. In the English-speaking West African countries constituting the WAEC, the menace of examination leakages has played no mean role in efforts to fashion out a more adequate examining modules that could help enhance teaching and learning in our schools. Other considerations include fairness to students and maintaining the standards already set by the Examining Bodies. It is in the light of the above that this paper is relevant. This paper attempts to discuss the topic Assessment for Improving Learning. Its focus is on examining the role of assessment as a tool for improving learning with particular reference to pre-tertiary education in Ghana. The paper also looks at the Curriculum, Instruction and Assessment and how the three components are linked and their impact on learning. Document type: Congress Presentation ISBN:	View
Yan, Han	Assessment for improving learning Abstract: This paper will present that assessment for improving young learners’ learning. For young learners who are 6-12 years old, the purpose of assessment for below 12 years old is different from those over 12 years old. According to character and cognition of young learners, the purpose of assessment is to build confidence on learning up, gradually be more interested in learning and enhance learning in the future. This paper will analyze the style of assessment for young learners in , including which mode of assessment of young learners is be choose, how to product question paper, how to give awards, how to give feedback to young learners. After assessment, for young learners, the situation of learning which means that ability of learning and attitude of learning will be changed. The assessment of Cambridge Young Learners English (CYLE)will be example of analyze and the resources of analyze come from candidates of CYLE. This paper will put forward ideas on how to improve the function of assessment on learning. Document type: Congress Presentation ISBN:	View
Wyatt-Smith, Claire Professor Bridges, Susan Dr	Assessment for learning: An Australian study in middle schooling Abstract: Assessment for learning has been widely reported as a reform in educational assessment. The recently published OECD study (OECD, 2005) provided several case study investigations of how assessment for learning is variously enacted in diverse policy and system contexts. In this paper we take up the topic of assessment for learning and draw on a recently completed large-scale study of teacher capacity-building in assessment in middle schooling (Years 4 to 9; ages 8 to 14 approx.) to explore characteristics of ‘assessment as critical inquiry’. Specifically, we probe the question, How can we enact a framework of assessment as critical inquiry for improving learning outcomes? The paper proposes a conceptual framing for developing a professional mindset that places assessment at the heart of the pedagogic enterprise. Document type: Congress Presentation ISBN:	View
Chan, Jacqueline Kin-Sang Fok, Ping-Kwan Yu, Flora Wai-Ming Kennedy, Kerry J	ASSESSMENT FOR PRODUCTIVE LEARNING: FORMS OF ASSESSMENT AND THEIR POTENTIAL FOR ENHANCING LEARNING1 Abstract: It has become fashionable in the literature on assessment to highlight the importance of "assessment for learning" and counter pose it against "assessment of learning" as though one form is intrinsically better than another. It can easily be acknowledged that there are different purposes of assessment (for example making decisions about selection and progression, providing feedback that will enable students to improve their learning, helping teachers to improve their instruction). These different purposes might dictate certain forms of assessment, but this should not mean that different forms of assessment need to neglect the learning potential that so many scholars and practitioners have seen in the assessment process. The purpose of this paper is to explore the learning potential of different forms of assessment, irrespective of the purpose for which the assessment is being conducted. From this overview, a holistic "learning model" of assessment will be developed to demonstrate how classrooms, schools and systems might take advantage of learning opportunities provided by all forms of assessment. Document type: Congress Presentation ISBN:	View
Knight, Brian	Assessment for recognition of prior learning in technical and Abstract: Recognition of prior learning (RPL) provides formal recognition for vocational knowledge or skills gained on-the-job or as a result of other informal or unstructured learning experiences. RPL, if granted, can count towards completion of recognised vocational qualifications. RPL provides a range of potential benefits for both learners and providers of technical and vocational educational and training (TVET) but also presents challenges. Assessment for RPL is invariably individualised and one-off rather than the more usual mass-assessment situation. This has implications for both costs and validity, including maintenance of consistent assessment standards. RPL also poses difficulties for the conventional models for publicly funding TVET, which generally assume that learners will typically engage in learning activities for a period of time and be assessed at the end. By contrast, RPL involves an up-front assessment, turning the conventional assessment situation on its head. Since the early 1990s, promotion of RPL has been a major policy objective of the Australian TVET system. Progress has been made and awareness has increased but to date the take-up of RPL has not met expectations. This paper will review the current situation with a particular focus on the issues thrown up by RPL Assessment and recent Australian government policy initiatives designed to address them. Document type: Congress Presentation ISBN:	View
Chan, Gregory Luo, Guanzhong Cheung, Amy K.M	Assessment of Basic Competencies Abstract: In its 2000 report entitled Learning for Life, Learning Through Life, the Hong Kong Education Commission set out detailed proposals for assessing students’ basic competencies in Chinese language, English language and mathematics. This paper reports on the progress in implementing these proposals. Two initiatives were undertaken. One set of initiatives has involved the development of a web-based student assessment system that recently won a silver medal for innovative excellence at le Salon International Des Inventions 2005. The other set of initiatives has involved paper and pencil testing of the whole cohort of students at the end of each key stage of schooling. This testing is conducted throughout the Hong Kong Special Administrative Region, China, and therefore, it is termed the Territory-wide System Assessment. This paper provides an overview of the measurement issues underpinning both assessments, including the calibration of items, equating of tests and the setting of standards. It also presents results of the first two years of implementation and the impact of the policy thus far. Document type: Congress Presentation ISBN:	View
Chan, Jacqueline Kin-Sang Kennedy, Kerry J Yu, Flora Wai-Ming Fok, Ping-Kwan	Assessment Policy in Hong Kong: Implementation Issues for New Forms of Assessment1 Abstract: Large-scale curriculum reform initiated 2001 in Hong Kong has set the stage for change in the school curriculum as well as assessment. In the past, Hong Kong education was characterized by its elitist system with highly selective examinations at different levels of learning. The selective examination oriented policy had its influential impact on the schooling of Hong Kong where teaching and learning were inclined towards examinations. Since 1990, there have been changes in the assessment policy in the school curriculum with more emphasis on formative assessment, e.g. Target Oriented Curriculum. The new forms of assessment have therefore posed a different teaching and learning environment for the schools, teachers and students. By looking at the previous experiences regarding implementation in the local context, this paper will analyze various aspects of implementation of assessment policies adopted by the schools in Hong Kong. Implications will also be drawn to address the possibilities of implementation for the current policy. Document type: Congress Presentation ISBN:	View
Yu, Flora Wai-Ming Kennedy, Kerry J Fok, Ping-Kwan Chan, Jacqueline Kin-Sang	ASSESSMENT REFORM IN BASIC EDUCATION IN HONG KONG: Abstract: In the implementation of a 10-year educational reform since the turn of the century in Hong Kong, assessment has been one of the key areas of change. The importance of changing assessment practices in the process of teaching and learning has been discussed in various government documents since 1990. In the past, assessment practices were inclined to be mainly "assessment of learning" which was more summative in nature. In the reform, schools were encouraged to put more emphasis on "assessment for learning" which was considered more of a formative nature. On approaching the mid point of such reform, significant difference in its degree and scope of development can be seen in schools. In this particular paper, the focus is assessment reform in the primary school sector. The two concepts in assessment are explored by a detailed documentary analysis of assessment policies written on basic education over the last decade. Together with reviewing the current developments on assessment change in primary schools, implications for more effective use of assessment for enhancing teaching and learning are made. Document type: Congress Presentation ISBN:	View
Hill, Peter W. Tak-wing, Wan	Assessment Reforms in Hong Kong Abstract: In many nations around the world, reforms are underway to allow more students to participate in senior secondary education and to better prepare them for life in the modern world. In Hong Kong, after some five years of consultation and discussion, a consensus has been reached regarding a series of reforms that will be implemented in the coming years. The academic structure will be changed to allow all students to complete senior secondary education. The curriculum will be changed so that all students will take four core subjects and two or three elective subjects. Assessment will change to incorporate a component of statistically moderated school-based assessment and to introduce standards-referenced methods of reporting achievement. This paper provides an overview of these reforms. Document type: Congress Presentation ISBN:	View
Sellan, Rajenthiran Chong, Karen Tay, Connie Chong, Karen	ASSESSMENT SHIFTS IN THE SINGAPORE EDUCATION SYSTEM Abstract: In recent years, the Ministry of Education in Singapore has made extensive changes in the areas of curriculum and assessment. These changes are aimed at preparing students to meet the challenges of an increasingly globalised world. The national school curriculum has been revised to promote critical and creative thinking skills, and to nurture problem-solving and independent learning abilities in students. To complement these changes in the curriculum, significant changes have been made in several areas of assessment. This paper discusses the recent assessment shifts in the Singapore education system by focusing on national examinations in the following areas: the greater emphasis on coursework and oral communication, and the changes in item types. Document type: Congress Presentation ISBN:	n/a
Sellan, Rajenthiran Lee, Lup Yuen Wong, Cheow Cher	Assessment Using Mobile Phone - An Exploratory Study Abstract: An assessment task typically comprises a stimulus posed by the assessor with the purpose of eliciting a response from the examinee. Traditionally, both stimulus and response are written on paper. More recently, non-print platforms such as computers have been used in presenting the stimulus and response. The increasing use of mobile communication devices such as mobile phones and PDAs has now prompted studies on the use of these mobile devices in assessment. The use of mobile phones has increased tremendously in the past decade in Singapore and around the world. Hence, it is timely to conduct an exploratory study to gain first-hand knowledge on the use of this medium in assessment. The study aims to determine factors to consider in designing an assessment system using mobile phone and find out if it is technically feasible. A prototype was developed and an internal trial conducted. The paper reports on the findings of the trial, flashes out problems encountered and discusses potential gains and shortfalls of such a system. Document type: Congress Presentation ISBN:	View
Jones, Neil	Asset Languages – a multilingual proficiency framework that supports learning Abstract: Asset Languages is the system being developed by Cambridge Assessment to implement the Languages Ladder, "a new voluntary recognition system to complement existing national qualifications frameworks and the Common European Framework" which is a major element of the UK’s National Languages Strategy. Asset sets out to accredit functional language proficiency within a can do framework. It is comprehensive, including at least 26 languages: those most commonly learned as "modern foreign languages", and those spoken by particular communities in the UK. It targets three contexts (Primary, Secondary, Adult), with skills assessed separately. It offers two assessment strands: external assessment at six major stages, and more informally accredited teacher assessment at 17 finer grades. The challenges of developing this complex framework are not merely technical or logistical: they concern how to design tests and interpretations which enable valid and useful comparison across such widely differing languages and learner groups, and above all how to do this in a way which impacts positively on learning. This leads us to look critically at the framework metaphor in general, and at the Common European Framework in particular; and to propose some conceptual clarification and practical methods for framework construction. Document type: Congress Presentation ISBN:	View
Odor, P. I. Ph.D Kolo, T. N. Ph.D Salim, B. A. Prof	Assuring Quality Through Adherence to Due Process in Test Development Abstract: The paper attempts to define quality of a test in terms of properties that are intrinsic to a test item as well as those that are extrinsic to the test as an instrument. It stresses the need for laying a solid foundation in test development such as capacity building and following due process in the actual test development process as essential steps in assuring a quality test. Document type: Congress Presentation ISBN:	View
Meng, Hoe Wee Tong, Irene	Authentic Assessment through Innovative Project Work in Jurong Secondary School Abstract: At Jurong Secondary School (JSS), Project Work (PW) is conducted for all secondary 2 and 3 students. Besides multi-disciplinary content learning, students' project tasks are crafted so that they carry national education themes to develop in students a sense of pride and belonging to the school and community. For example, in 2005, some students featured achievements of the school, as well as prominent alumni members who made significant contributions to their alma mater and society. Other students featured interesting places in the neighbourhood to create awareness of Taman Jurong community. Some students even focused on the role of Jurong in industrialisation and in the tourism industry, and hence the development of Singapore. This paper details how PW is conducted in JSS to captivate students' interest and motivation. In particular, the paper focuses on how students are assessed using a set of tailored scoring rubrics that give emphasis on the process of doing the project, including information processing, communication and collaboration. Other important outcomes such as innovation and enterprise, complex thinking, and reflective learning, are also achieved through PW. Actual students' work is shared. Document type: Congress Presentation ISBN:	View
Adams, Carolyn	AWARDING IN THE 21ST CENTURY - A VIRTUAL MODEL Abstract: Awarding meetings are essentially unchanged since their inception, with Senior examiners coming together to look at statistical information and candidates’ work. Modern techniques permit different approaches to this process whilst preserving the validity of the process by incorporating the views of subject matter experts. A virtual award meeting will be held using Virtual Classroom technology, with Senior examiners contributing to the meeting remotely. Senior examiners scrutinise scripts remotely and their judgements are collected electronically. The Chair of Examiners and staff establish zones of uncertainty and prepare other materials for the virtual award. The normal Awarding Committee are presented with the collated information and discuss it over a synchronised telephone conference. Scanning of the scripts and remote scrutiny opens up the possibility of including other stakeholders in the qualitative judgments process, such as examiners, teachers and even students. The source of judgements would need to be delineated, but it is possible that standard setting could be made more transparent to stakeholders. More modern approaches offer the opportunity to make the most of the Senior examiners’ expertise and make the process more efficient and robust. Document type: Congress Presentation ISBN:	View
Miller, Kari	Building competence based capability in organisations Abstract: In New Zealand the National Qualifications Framework has been in place for 15 years. During that time the main focus has been on vocational skills but this has been expanded in the last 5 years to encompass the school sector. In general industries have now embraced the competence based system and are facing issues of how to quality assure the assessment process in a workplace context. In this paper I will explore the issues that industries are facing instituting competence based assessment systems into the workplace and some of the innovative approaches being taken to resolve those issues. While larger companies seem to have less problem integrating competence based systems into their workplaces because they are able to devote full or part time resources to the process, small to medium enterprises struggle to make the system work effectively. This paper will explore the issues relating to determining candidates’ requirements for assessment, how evidence is collected and validated in the workplace, the issues related to selecting and training workplace assessors and how the system can be quality assured in a way that interfaces with current workplace systems and processes. Examples of good Document type: Congress Presentation ISBN:	View
Poliah, RR SiShi, SNP	Can Statistical and Qualitative Modes of Moderation Co-exist in a Model for Abstract: School-based assessment constitutes part of the assessment that leads to the issue of the exit certificate, after 12 years of schooling in South Africa. The high stakes nature of this certificate warrants a high level of reliability and validity of the assessment. There has been much debate about the reliability of the school-based assessment component, given the challenges associated with the implementation of school-based assessment in any large system. Currently, the system is mainly dependent on a model of statistical moderation that brings the school-based marks within an acceptable deviation of the examination mark. This paper explores the challenges relating to the quality assurance of school-based assessment in South Africa and presents a model for the quality assurance of school-based assessment that is built on three fundamental pillars, i.e. (a) setting of clearly defined standards; (b) providing appropriate support and guidance; and (c) monitoring and evaluation of the assessment process and the outcome. The setting of standards incorporates input, process and output standards and the support and guidance relates to professional and administrative support required for implementation. In the evaluation of the final outcome, this model argues for the integration of statistical and qualitative data that will ensure that every mark is a reliable indicator of the performance of the learner. This model could be progressively implemented as the new curriculum is phased in, commencing in 2006, and, if successfully implemented, will go a long way in improving public confidence in the reliability of school-based assessment. Document type: Congress Presentation ISBN:	View
Meadows, Michelle Dr	Can we predict who will be a reliable marker? Abstract: The Assessment and Qualifications Alliance, Research and Policy Analysis Department have conducted a large number of studies that have attempted to identify factors which might allow awarding bodies to predict those examiners who are likely to mark most reliably and those who are likely to require additional training or monitoring. Most of the work, however, is not publicly available. This paper provides a review of this research and internationally published studies relevant to predicting marker reliability. The relative importance of the following variables are discussed: examining experience; teaching experience; subject knowledge; senior examiners’ ratings of examiners’ previous performance; and examiner traits such as logical reasoning capacity and personality. Overall the conclusion of the review is that the criteria used by UK awarding bodies to select examiners (subject knowledge and teaching experience) are not empirically supported. The paper ends with the description of work that is currently being undertaken to examine whether psychometric measures of personality can be used to predict the marking reliability of individuals with distinctly different levels of examining experience, teaching experience and subject knowledge. Document type: Congress Presentation ISBN:	View
Alagumalai, Sivakumar .	Can we trust our teachers, their tools and techniques? Abstract: There have been positive directions in formative assessment and school-based assessment. These assessments provide students opportunities to highlight their learning in a developmental and progressive manner in authentic settings. Learners are empowered to indicate, through their work, both the heuristics of the learning and content mastery. Teachers (or raters!) in schools have the arduous task of examining a student’s work (against a list of capabilities statements or through a set of criteria articulated in a rubric) and making objective judgements. Scores are assigned to various activities and tasks, which are then aggregated for a grade and reported to the students and relevant stakeholders.This paper highlights the problems with raw scores, its aggregation and the effect of raters (or teachers) on the scoring and grade assignment processes. It also discusses the challenges associated with rubrics and judgements, grade inflation and comparability. The use of Rasch and multilevel techniques is highlighted. The principles underlying objective measurement are included. Document type: Congress Presentation ISBN:	View
Tax, Henny MsC	CBT development limits test development? Abstract: When planning ICT in a test one calculates advantages and disadvantages. What kind of knowledge, skills and attitudes are possible? At the moment the schools are training their pupils at showing competencies. Knowledge is not enough. A pupil must act in the right way and must know why acting like this with awareness of risks for itself and its attendants. We are developing CBT examinations in the Netherlands integrating knowledge testing and theoretical and practical skill testing. We want to use one ICT system to test each of them separately or both of them at a time. We evolved from paper and pencil tests to overall CBT with the software Citotester. Using a question viewer asks for an item construction bank that allows all kind of questions. At the moment we use Citotester for low stake exams. When using it for the high stake exams, as for example in high school exams, we must tackle problems like how many computers are available in the school at one time and can we make equivalent exams to be used at successive moments. The presentation will show some examples of the use of CBT in a complex situation. Document type: Congress Presentation ISBN:	View
Bukenya, Matthew	COMPARING THE RELIABILITY OF THE CONVEYOR BELT MARKING Abstract: In Uganda, Uganda National Examinations Board (UNEB) conducts Primary Leaving Examinations (PLE), whose results are mainly used to select pupils for post primary education. As such, it is imperative that UNEB maintains high standards of validity and reliability of PLE. Since its inception, in 1980, UNEB has used the Traditional Marking System (TMS), in which one examiner alone marks all the questions on a candidate’s script. There were concerns about the reliability of marks obtained by using TMS, as inconsistencies in marking were sometimes discovered while scrutinizing scripts with suspected malpractice. In the year 2003, UNEB piloted at PLE the use of a marking system, it called the Conveyer Belt System (CBS) of marking. This system confines an examiner to marking a set of questions that have been assigned to him/her and passes the script to the next examiner, who marks another set of questions. Following this, a study was conducted to compare the mark-qualities of CBS and TMS. Specifically the study objectives among others were to: estimate reliability to conveyor-belt marking system (CBS) and the traditional marking system (TMS); determine the deviations in the examiners’ marks from the team leaders’ marks in the CBS and the TMS; list advantages and challenges of the CBS; and recommend some ways of improving management of the CBS. This paper will highlight the key findings from the study. Document type: Congress Presentation ISBN:	View
Robbins, Jonathan H Dr	Connoisseurship, assessments of performance and questions of reliability Abstract: Increasing emphasis is being placed on the importance of personal qualities and attributes such as creativity, inter-personal skills, leadership, communication, or aesthetic awareness. These qualities and attributes are not amenable to current assessment processes that rely on specifications and the production of evidence or on more conventional types of examination. This paper describes applications of connoisseurship and construct referencing in assessing achievements of these or similar qualities through observed performance. Examples are drawn from assessments of performance in music and dance as well as from programmes concerned with emotional and behavioural development, employability and training for teaching and learning support. Questions relating to applications of connoisseurship and the reliability of assessment practices and results are considered. A brief description of methods used to express measures of reliability in aviation training and in precision engineering is given and their application to the reliability of assessments of performance considered. Results obtained over a three year period in the use of a method developed for monitoring assessor performance and the standardisation of results from assessments of performance by an awarding body in the United Kingdom is reported on together with some conclusions about its use and applicability in different settings. Document type: Congress Presentation ISBN:	View
Buck, Sarah Lieutenant	Creating the Military eLearning Culture: Evaluating Assessment Techniques Abstract: The Ministry of Defence, in partnership with BT, has implemented a collection of web-based services that support distributed eLearning across the whole of Defence. The service, the largest of its kind in Europe, is known as the Defence Learning Portal and has a potential user base of 300,000. This revolution in distributed learning drives the requirement for a change in assessment. This paper evaluated a British Army eLearning package, Military Knowledge 2, and its forms of assessment. The package employs on-demand assessment using formative and summative assessment techniques. At present multiple choice and open-ended questions are the main form of eAssessment. In most cases, a question refers to a particular Lesson, but there are also some questions that encompass a whole section. These questions are more complex and attract a higher score. eLearning is new to the British Military and encompasses a cultural shift as well as a technological change. This paper explores the assessment techniques employed and how these can be improved with the emerging technology and its implication for the whole of Defence. The evaluation is also considered against the cultural change required to make Military Knowledge 2 an effective learning and assessment tool in the British Army. Document type: Congress Presentation ISBN:	View
Looney, Anne Klenowski, Val Professor	Curriculum and Assessment for the Knowledge Society: Interrogating Abstract: The concept of the Knowledge Society is one of the most pervasive concepts in the rhetoric of educational reform. The knowledge society is neither a stable nor a well-defined concept. Its boundaries are fluid, but its power is undeniable and ubiquitous. As such it is functioning as discourse within educational reform, and a public discourse within the policy making and framing public. Two cases are of interest. In Queensland, the state government has responded in innovative and challenging ways to national demands for greater consistency in education. The development of the Queensland Assessment Task, with a focus on the computer-based response, will illustrate how a different approach to assessment can help to illuminate complex learning for the knowledge society. This important research into curriculum and assessment, design and development has helped to inform the state’s recent plans to implement a Curriculum and Assessment Reporting Framework in Years 1-10 that includes new-form dimensions of curriculum and assessment. In Ireland, the ‘most globalised nation on earth’, sustaining the recently established ‘celtic tiger’ economy and ‘keeping ahead of the curve’ have put enormous pressures on the traditionally inert school system. Ireland’s current social revolution has seen immigration replace emigration, one third of all births to single parents, and a population enjoying one of the highest per capita disposable incomes. Such change has exerted immense pressures on curriculum and assessment development. The ambitious reform proposals for upper-secondary education for example, recently published by the National Council for Curriculum and Assessment, reflect the considerable influence of the knowledge society discourse. The paper raises questions of whether what is transacting is less the re-form of curriculum than the emergence of new-form curriculum where ‘content’ is displaced by ‘skills’ and ‘knowledge acquisition’ by ‘learning’. In this context, assessment also begins to take on new-form, with a focus on assessment that engages and promotes learning. Document type: Congress Presentation ISBN:	View
Hermans, Peter Mr.	Curriculum change, innovation and innovative examinations: Shifting expectations Abstract: In the last two decades, curriculum content in Dutch education has been re-arranged into a limited number of new programs or profiles. Students are expected to study more independently and are encouraged to take increasing responsibility for their studies. Although none of these reforms led to structural changes in the educational system, consultations with the majority of Dutch secondary school principals and teachers have initiated an ongoing debate about the current system of national examinations. Cito, the Dutch National Institute for Educational Measurement, in charge of the production of the national examinations for general education since 1967, the National Examination Board, the Department of Education and the National Organization of School Principals are working together on an agenda for the future of examinations. The aim is to design a more flexible national examination system that will safeguard the quality of the current system while giving enough room for development of innovative examination strategies. On top of these changes, there is a growing influence of social constructivism in secondary school curricula. This trend has resulted in a number of experimental schools and programs. One important characteristic of these experiments, called the ‘new learning’, is the tendency to organize education in terms of ‘mass customization’, in a way similar to mechanisms we see in the organization of industrial processes. Standardization and modular structure leading to ‘tailor made’ products are important characteristics of mass customization in general and recent research show that these characteristics also apply to this latest education reform. This presentation discusses the dilemmas and perspectives of maintaining the quality standards of a system of national examinations while giving enough room for experimental schools and their curriculum development. Document type: Congress Presentation ISBN:	View
Loock, Coert Prof. Govender, Prem Cesare, Joe	Dealing with Examination and Assessment Irregularities in an Era of Rapid Abstract: The occurrence of examination and assessment irregularities can seriously damage public confidence in the validity and legitimacy of examination and assessment results and should be dealt with as a matter of urgency within the judicial framework established for this purpose. Thus, the National Policy on the conduct of Senior Certificate examinations indicates that, a Senior Certificate may not be issued to a candidate found guilty of an irregularity even though he/she satisfies the requirements for a Senior Certificate., pending the holding of a hearing. The problem that emerges where candidates fail to appear for a hearing as it raises the question whether the department can cancel the results of a candidate without the said candidate having personally presented his/her case. The purpose of this paper is to address this issue in lieu of common law development prior to the advent of the 1996 constitutional dispensation and the way in which the common law rule has been altered by Section 34 of the 1996 Constitution. Document type: Congress Presentation ISBN:	View
Lee, Ai Noi Gong, Wengao Wong, Hwei Ming Koh, Kim Hong	Development of the Singapore Prototype Classroom Assessment Tasks: Abstract: This paper describes the development of the Singapore prototype classroom assessment tasks in English, Mathematics, Science, Chinese Language, Malay Language, and Tamil Language as well as the use of authentic intellectual standards and scoring rubrics for evaluating the quality of assessment tasks and related students’ work. The authors will provide a brief background to and the rationale for the development of standards for assessing the quality of classroom assessment tasks and related students’ work across subject areas at Primary grade levels in Singapore schools. This will be followed by in-depth discussions of the alignment of the prototype assessment tasks with the authentic intellectual standards; the training of teachers in crafting high authentic intellectual assessment tasks; the implementation of the assessment tasks in the day-to-day classroom teaching, learning, and assessment; the construction of the scoring rubrics and exemplars; the moderation procedures for teachers to evaluate the quality of students’ work in response to the authentic intellectual assessment tasks; and the strategies in empowering teachers to build professional learning communities within and between schools. Findings and implications of the prototype classroom assessment tasks on teachers’ pedagogical and assessment practices as well as on student learning and performance will also be discussed. Document type: Congress Presentation ISBN:	View
Winkley, John	Developments in on-screen assessment design in examinations Abstract: A paper which will draw on examples from projects undertaken for a range of UK agencies, including the regulators from each of the 4 nations (QCA, ACCAC, SQA and CCEA), and Awarding Bodies such as Edexcel and the British Computer Society. This work includes the use of: • rich media (exploring how video, audio, animation and imaging affect assessment performance, including for candidates with disabilities), • interactivity and adaptivity (exploring how requiring students to make interactive responses affects achievement and engagement), • advanced computer-marking techniques (work to mark candidates’ prose, mathematical workings, and process as well as output), • item banking complex items to allow "when ready" assessment, and comparability issues with more traditional assessments. The paper also discusses how "when-ready" e-assessment is blurring the traditionally clear boundary between summative and formative assessment, and the opportunities open to qualification providers to reshape their assessment offerings to act as learning resources. Document type: Congress Presentation ISBN:	View
Thomas, Sally Peng, Wen Jung	Eight year trends in English students attitudes to school Abstract: This paper provides an initial overview of the findings from a study of student attitudes to school over a period of eight years within and around one English local education authority. However, due to the exploratory nature of the analysis the results should be treated cautiously at this stage. The LEA project was set up in 1992 aims to provide an innovative system of secondary school evaluation and self-evaluation via the feedback of student performance, attitude and other data. The feedback information provided by the project is intended to inform the improvement processes of state funded schools within and around the LEA region and is not intended for external accountability purposes. Student attitude data, collected annually from 1996 to 2003 (excluding 1998), was analysed using multilevel modelling and factor analysis techniques to identify any significant trends over time in four separate student attitude dimensions. The findings indicate that student views have changed in relation to some key aspects of school life over the time period examined. Document type: Congress Presentation ISBN:	View
Cumming, Joy J Professor Dickson, Elizabeth A.	Equity in assessment in Australia: The legal perspective Abstract: Fair practice and equity in assessment can be viewed from many perspectives. In this paper, we examine how courts view equity in assessment. The paper will focus on findings from Australia, with some insights from the USA and the United Kingdom. The cases considered will address ‘fair’ assessment of all students for high stakes or certification purposes, introducing the concept of ‘opportunity to learn’ as well as issues in appropriate accommodation in assessment practices for students with disabilities or special learning needs. Document type: Congress Presentation ISBN:	View
Ajagun, Grace A Obioma, Godswill	ESTABLISHING NEW ASSESSMENT STANDARDS IN THE CONTEXT OF CURRICULUM CHANGE Abstract: Nigeria adopted a National Economic Empowerment and Development Strategy (NEEDS) in response to the global reforms in the social and economic context. The critical elements of NEEDS are value re-orientation, poverty reduction, wealth generation, job creation and using education to empower the people in order to achieve all these. In the education context, Nigeria has embarked in far reaching school reforms. Particularly for the pre-tertiary level, a 9 – year free basic education has been adopted with corresponding re-structuring of the subject offerings and ongoing reforms in the school curriculum. The selective external examination for admission from the 6-year primary school into the 3-year junior secondary school has been scraped. A Basic Education Certificate will be awarded at the end of the 9-year continuous schooling. The education contents are being tailored to ensure acquisition of life-long strategic work skills and appropriate value orientation while ensuring a strong support for functional academic competencies. These curriculum reform measures have thrown new challenges for setting new assessment standards. This paper proposes a strategic framework for establishing these standards. Document type: Congress Presentation ISBN:	View
Maughan, Sarah Monteath, Peter Morley, Jo Dr Sharp, Stephen	Extending Examination Board Activities: an International Review of Innovative Abstract: Examination boards globally are increasingly engaged in a wide range of teaching and learning activities as a complement to their traditional role of setting examinations. This is certainly true of University of Cambridge International Examinations (CIE). In order to establish the extent to which this is indicative of a global trend CIE has conducted a survey of examination boards around the world to establish: �� the types of teaching and learning support most commonly provided by boards, �� how much the use of technology is facilitating this process, �� the rationale underpinning developments in this area, �� key benefits which examination boards achieve via this diversification. This paper describes the research findings, and then draws conclusions on the changing role of examination boards. It goes on to discuss the way examination boards make a positive impact on the standards of teaching and learning in schools, and on the development of education more widely. Document type: Congress Presentation ISBN:	View
Kew, Cheah Piang	Finding the Maximum Likelihood Estimate of a Parameter in an Abstract: A crucial step in applying item response theory to practical testing problems is the estimation of ability and item parameters in the chosen item response model. It is usually not possible to estimate the parameters analytically. In the situation where there is only one parameter, two fast iterative numerical procedures for finding the maximum likelihood estimate of the parameter are presented in this paper. A numerical example is given to illustrate their use. Scope for further research is then discussed. Document type: Congress Presentation ISBN:	View
Watts, Andrew	FOSTERING COMMUNITIES OF PRACTICE IN EXAMINING Abstract: Examiners and assessors who work in teams to judge the quality of students’ work in examinations, or of trainees’ performance in assessments of competence, are frequently described as working in ‘communities of practice’. Following Wenger (Communities of Practice: Learning, Meaning and Identity. 1998), this concept is used to describe the way examiners acquire their craft and maintain their competence in it. This paper discusses some of the literature about the place of communities of practice in examining, and seeks to clarify the rationale for them. It does this in the light of the significant changes taking place because of the introduction of new technologies to examining. The concept of communities of practice has often been put forward as a description of the strategies and procedures which lead to reliable marking. The use of e-technology could support such an aim. The paper argues that, at the same time, the necessity of fostering communities of practice to provide a context for valid assessments, can also be supported by new technologies. Document type: Congress Presentation ISBN:	View
Green, Sylvia	From Principles to Practice:Making Assessment for Learning work in the classroom Abstract: The concept of assessment for learning, also known as formative assessment, is generating a great deal of educational debate in the UK. It is a frequently used phrase but what does it mean and what are its implications for teachers and learners? In this presentation the underlying principles of assessment for learning will be presented and strategies for good practice will be outlined. An innovative, interactive system, developed by Cambridge Assessment and Harcourt Assessment, will be described. This assessment for learning system offers screen-based assessments, diagnostic reporting, targeting and planning for teachers and students (aged 11 –14) in Mathematics, English and Science. The system demonstrates the way in which assessment for learning can be operationalised with support for teachers and learners. Document type: Congress Presentation ISBN:	View
Hudson, Graham	Giving candidates a fairer deal in examinations and tests through electronic marking Abstract: The presentation will consider how electronic marking brings benefits to candidates taking high-stakes tests and examinations where marking accuracy, reliability and timeliness are paramount. Areas to be covered will include how marking quality is monitored and maintained through multiple marking methods, ensuring anonymity, impartiality and avoidance of bias during marking and how detailed data can provide for improvements in both test design and student learning. Some discussion of the challenges of deploying an electronic marking system across more than one country will be included, covering data security, resilience and performance matters. Document type: Congress Presentation ISBN:	View
TAN, Aik Ling Towndrow, Phillip A.	Giving students a voice in science practical assessments Abstract: This paper examines Science Practical Assessment (SPA) in the Singaporean classroom. In contrast to teacher-centric task setting and evaluation, this paper reports findings from a study where a class of students were involved in their own assessments mediated by digital video. Students were recorded during practical work and were then asked to review and edit the footage. Next, they evaluated their own and their classmates’ practical skills. These evaluations, scaffolded with a template and facilitated by the teacher, aimed to give the students a voice in presenting what they thought made ‘good’ science practical skills and practices in the laboratory. They also served as a platform for peer learning and provided a means for the students to be involved in discussing science and science practical skills. Results of this study reveal that students’ awareness of acceptable laboratory practices is enhanced through this innovative method of evaluating science practical work. Document type: Congress Presentation ISBN:	View
Sevilla, Jesus E. Jr. Tan, Kathryn M.	IDENTIFYING PATTERNS OF SKILLS ACQUISITION IN ELEMENTARY Abstract: This study explores the use of test performance to identify patterns of skills acquisition that differentiate good and poor performers in elementary mathematics. Good and poor performers in mathematics were identified through their cumulative raw score on six achievement tests in grade 1 to grade 6 mathematics. These tests were administered consecutively for six school years to an intact cohort group of 1,347 pupils towards the end of each school year. Discrimination and difficulty indices of all test items were computed to identify “critical” skills that highly discriminate the good performers from the low performers. The connection between the critical as well as non-critical skills in doing fractions and the patterns of acquiring these skills from one grade level to the next were then described and illustrated. The results showed that majority of the critical items are more difficult than the non-critical items. The pattern of difficulty and discrimination indices of items on fractions indicated that both good and poor performers acquire the ability to identify fraction concepts from illustrations and perform addition and subtraction on similar fractions. Both groups, however, need to extend their conceptual understanding of fractions. The competencies of good and poor performers diverge at the point when they are required to compare fractions and execute basic operations on dissimilar and/or mixed form fractions. The results also showed how proficiency may be demonstrated with procedural knowledge without necessarily implying a good grasp of underlying concepts. Document type: Congress Presentation ISBN:	View
Cheong, Ng Tong Wai, Noelle Ho Sok Yim, LWK	INNOVATIVE USE OF TECHNOLOGY IN GRADING Abstract: In the past, all grading meetings for the Singapore national examinations between the Singapore Examinations and Assessment Board (SEAB) and Cambridge Assessment were held in Cambridge. These meetings were held to determine the grade thresholds of subjects for various examinations. In 2004, SEAB and Cambridge Assessment embarked on an innovative approach to conduct grading meetings through the use of video-conferencing technology. Although this may seem to be a simple approach to the use of Information Technology (IT), the successful implementation of such meetings required much preparation and the redesign of grading procedures. This paper shares the experience of the use of video-conferencing technology for grading meetings for the 2005 GCE N-Level examination. It also discusses the success factors, benefits, issues encountered and some feedback from related personnel regarding the use of video-conferencing technology for grading meetings. Document type: Congress Presentation ISBN:	View
Petridou, Alexandra Williams, Julian	Instability of Person Misfit and Ability Estimates Subject Abstract: Unexpected response patterns on tests and their problematic interpretation has led to an intense research activity (i) to investigate the sources of such responses and (ii) to model statistics that claimed to detect person misfit in an examinee’s response pattern. The rationale behind this effort was the claim that the test scores of these examinees with unexpected response patterns may fail to provide a useful and valid measure of their ability. In this study we have followed-up ‘misfitting’ examinees in a Mathematics test and during interviews we have asked them to work out items again on which they have provided unexpected responses. Pupils’ response patterns were then changed based on their responses during interviews and fit analyses were re-run in order to obtain new estimates of their abilities and of the fit statistics. When old and new estimates were compared using a paired ttest, there was an overall increase in ability estimates and an overall reduction in pupils’ Infit and Outfit values. However only the change in the Infit values was statistically significant. By examining the change in ability estimates for each individual pupil we have found cases where the two measures were seriously discordant, raising questions about the validity of test-measurement for these pupils. Document type: Congress Presentation ISBN:	View
Fok, Ping-Kwan Kennedy, Kerry J Chan, Jacqueline Kin-Sang Yu, Flora Wai-Ming	Integrating assessment of learning and assessment for learning in Hong Kong public Abstract: Hong Kong is famous for its examination-dominated culture, which heavily relies on the public examinations. So ingrained has it become that the whole society is sensitive to any change in such an assessment mechanism. In recent years, the Hong Kong Examinations and Assessment Authority has used both external and school-based assessment as part of high stakes end of schooling assessment. Recent reforms have increased this reliance on school based assessment. The objectivity of external assessment is easily accepted by the society and the addition of school-based assessment components is often supported by tertiary institutions. Yet the practice of including school-based assessment results in addition to public examinations has been challenged by practitioners such as teachers in secondary schools as well as by the public. This paper focuses on understanding the views of teachers concerning school based assessment since such views are likely to add pressure to its implementation. In particular it will examine why teachers and the public appear to have more faith in external assessment rather than the professional judgment of teachers. Document type: Congress Presentation ISBN:	View
Whitehouse, Gillian	Intelligent Pens, Paper and Ink Abstract: PaperWorks is an EU project concerned with providing distinctive ways of interleaving paper documents with digital materials. The project focuses on developing a core technology for interlinking established content in paper and electronic domains. This is demonstrated through a robust application where a non-obtrusive pattern on the paper allows users to interrelate content with associated digital information. PaperWorks involves innovative developments in the production of novel substrates, inks, reading devices and the integration of software and communication resources as well as requiring an adaptable information architecture. PaperWorks draws on innovative research to develop support for authoring information and associated links. Edexcel / Pearson Education have now been working with this project group for 18 months and are developing a method of linking these technologies to both summative and formative assessment processes. The project also involves an analysis of the development and capture of creative and problem solving processes. Edexcel/ Pearson also have a keen interest in developing the links between assessment and learning and through this project are able to demonstrate a variety of potential opportunities that the technology can provide to foster and nurture these links. Document type: Congress Presentation ISBN:	View
Wu, Amery D. Li, Zhen Siok Leng, Ng Zumbo, Bruno D.	Investigating and Comparing the Item Parameter Drift in the Mathematics Anchor/Trend Abstract: This paper studied the item parameter drift (IPD), uniform and non-uniform, of the trend items in the Trends in International Mathematics and Science Study (TIMSS). TIMSS has maintained and used an item pool from which items have been repeatedly administered since 1995. A problem with long-term usage of test items is that over time the items might perform differently as a result of change in measurement precision and/or difficulty level. This change is referred to item parameter drift. In this paper, the 23 trend items of Grade-8 mathematics test administered across three TIMSS cycles were examined using logistic regression analysis to see whether IPD was present. In addition, the IPD results were compared between Singapore and the United States. The results showed that, for both countries, neither uniform nor non-uniform IPD had occurred. Document type: Congress Presentation ISBN:	View
Yong, Seah-Tay Hui Mrs	Investigating the Intellectual Quality of Abstract: A unique feature of the Integrated Programme at NYGH is Student-Initiated Assessments (SIA). These are alternative assessments which generally involve authentic tasks that provide students with opportunities to integrate and apply classroom learning to the real world. This paper will present the findings of an investigation into the intellectual quality of the SIA tasks set for the upper secondary Maths, Science and Language Arts classes using the scoring rubrics developed by Centre for Research in Pedagogy and Practice (CRPP). It will also discuss the implications of these findings within NYGH and beyond. Document type: Congress Presentation ISBN:	View
Bragg, Joanna Papasolomontos, Christina Boyle, Bill Dr.	Is there equity in assessment?A longitudinal study into the relationship between disadvantage and secondary Abstract: Since 1996 the authors have collected the annual school national curriculum provision and change data for the government in England as part of a funded longitudinal research project. These data are used as the basis for a range of national monitoring reports and test data analysis feedback to schools. This paper focuses on the issue of the relationship between disadvantage and underachievement in secondary schools in England. Since 1998, England has tested pupils aged 11 and 14 in English, mathematics and science subjects alone. This has created a hierarchical system of subjects and has caused teachers to focus much teaching time on the ‘testable’ curriculum sub-domains of those subjects – to the detriment of the principles of broad and balanced curriculum menus. Since 1996 the authors have collected and databased subject teaching time allocations from a national sample of schools and these data from the years 2004 and 2005 are utilised in this paper. The authors match the responding sample of schools to the Indices of Multiple Deprivation, then categorise their national test outputs against the nationally expected levels in the sub-sample groups of ‘disadvantaged’ (more than 20% free school meal eligibility) and ‘control’ (under 20% free school meal). The teaching time allocation data for the years 2004 and 2005 then enable the authors to determine whether the school/subject department has strategised more/less teaching time to address subjects in which the school has under-achieved or whether there is an issue of compliance or non-connectedness in these ‘disadvantaged’ schools. Document type: Congress Presentation ISBN:	View
Keeley, Tony	Learning for life Abstract: The Scottish Executive Education Department has been actively promoting the inclusion of vocational skills in the curriculum. The Scottish Qualifications Authority (SQA) has responded to this by producing a new suite of National Courses. The first group of these new courses, which are being delivered this current session, cover the following areas: Construction, Financial Services; Early Education and Childcare; Sport and Recreation. These new courses have a strong focus on the following elements: Employability skills and attitudes; Vocationally specific skills; Learning through practical experiences; Learning through review and reflection. A range of innovative assessment techniques to reflect these elements has been developed. The unique feature of these courses is that the assessment is aimed at producing evidence, not only of the vocationally specific skills, but of generic employability skills together with evidence of involvement in reviewing and reflecting as an integral part of the learning process. The courses are assessed by teachers/lecturers at local level and moderated at national level by SQA. To assist centres in the assessment of these courses, the SQA has also developed an assessment package for each course to exemplify the standard of evidence required. Document type: Congress Presentation ISBN:	View
Pool, Peter	Losing your inhibitions: possible effects on assessment of dynamic, interactive computer items Abstract: This paper examines some of the issues that arise in writing mathematics assessment questions for presentation on a computer screen. It questions the assumption that the computer screen functions simply as a replica of a sheet of paper. It seeks to argue that the medium of presentation and expression is in fact an inherent part of the discourse of mathematics and that any move of assessment from one medium to another changes that discourse. There will be new possibilities for cognitive activity and the loss of others. In this way the assessment medium redefines the subject. Document type: Congress Presentation ISBN:	View
Shirley, Yeung Sze-Yin	Meaningful Homework? A critical reflection of the new homework policy in Hong Kong Abstract: Over the years numerous studies have been conducted in an attempt to determine the effect of homework on students’ learning (Thomas, 1992). In practice, there are much problems and misconceptions with the implementation of homework. In Hong Kong, this issue worsens existing educational problems (Education Department, & Home School Cooperation Committee, 1994). With the launching of curriculum reform in 2001, the education bureau in Hong Kong recommends that there be a change in homework policy and practices(Curriculum Development Committee, 2001). Schools should put more emphasis on "meaningful homework" as an integral part of the school curriculum. In the past, curriculum changes in Hong Kong often encounter unsuccessful experiences because of poor policy dissemination and implementation and mismatching teachers’ conceptions, etc (Yeung, 2004). Studies alike demonstrate that any educational change needs an authentic "paradigm shift" in perspectives, philosophy and pedagogy (Kuhn, 1970). The paper intends to look into the underpinning theory and thereafter the underlying agenda embedded with this homework policy. The researcher has adopted research method including documentary analysis and simple survey. The researcher conducts and compares content analysis of two kinds of documents -- Official homework guideline as well as various samples of school homework. A simple survey of teachers’ conception about the new homework policy has been used as kinds of triangulation. The researcher wishes that findings from the study may have empirical and practical significance to future development of homework practices and policy in schools. Document type: Congress Presentation ISBN:	View
Kocsis, Mihály Mátrai, Zsuzsa	Monitoring Cognitive and Mental Abilities of Students Abstract: There are two factors that affected the Hungarian higher education system significantly. First: instead of entrance examination the results of leaving examination became the main selection criteria for getting to universities and colleges; second: concerning the higher education system Hungary also has to be accommodated to the new training structure of European Union since the 2006/2007 academic year. Because the double-function new Hungarian leaving examination is subject-oriented just as before, we wanted to map the cognitive and mental abilities of students in a more direct way. It was also important just because one part of the Hungarian accreditation procedure made compulsory the monitoring students’ development. For this reason in the 2005/2006 academic year we administered a test battery which consisted from three sections: verbal, quantitative and mental ones. (Apart from this we also administered a background questionnaire, naturally.) We did not use a representative sample, instead the first grade of 10 higher educational institutes were involved in the country which was about 60% of the whole population, respectively certain commonly selected majors. The main findings of our survey will be presented in the conference paper pointing out the most interesting phenomena. Document type: Congress Presentation ISBN:	View
Hadi, Fawziyah A. M. Ph.D. Al_Omar, Bader Ph.D.	Multilevel Analysis Approach for Determining 8th Grade Mathematics Achievement in the State of Kuwait Abstract: The purpose of this study was to determine the magnitude of the effects of students’ level variables and school level variables in predicting the 8th Grade students’ achievement in Mathematics in the State of Kuwait by using Hierarchal Linear model (HLM) strategy. A sample of 865 eight-graders and their parents has participated in this study. Furthermore a stratified sample of 37 schools together with their principals and mathematics teachers were involved in the study. The assessment battery of mathematics test, students’, teachers’, and schools’ questionnaires were used for the purpose of collecting data. The findings of this study revealed that the variation within schools is higher than the variation between schools, which means that the student-level variables such as prior achievement and academic self-concept in predicting mathematics achievement are more important than school-level variables. Document type: Congress Presentation ISBN:	View
Balázsi, Ildikó	National Assessment of Basic Competencies in Hungary Abstract: In 2001 the Hungarian Ministry of Education initiated a large-scale study to assess school effectiveness in preparing students to the requirements of the labor market and lifelong learning. The National Assessment of Basic Competencies (National ABC) measures the abilities of students in reading and mathematical literacy in grades 6, 8 and 10 in every school. It also involves a short Student Background Questionnaire and a School Questionnaire to map the background characteristics of students and schools. The results are reported to schools and authorities on the basis of a sample of twenty students per school and these reports help schools to get an objective description of their achievements. My paper and presentation will focus on the key properties of the study. To familiarize the reader with the National ABC I am going to shortly describe the framework of the cognitive domains and the defined proficiency scales, the methodological properties and the summarized basic results of the assessment. Document type: Congress Presentation ISBN:	View
Cheng, Leong See	On varying the difficulty of test items Abstract: This paper discusses the practice of varying the difficulty of test items in educational measurement. An item difficulty framework comprising concepts such as content difficulty, stimulus difficulty, task difficulty and expected response difficulty is introduced. The paper concludes with some suggestions on how to vary the difficulty of test items. Document type: Congress Presentation ISBN:	View
Haggie, David	Onscreen marking: introduction strategy and examiner response Abstract: Many awarding organisations consider that significant benefits may derive from the introduction of technical systems and services to support assessment processes after an examination has been sat. The benefits claimed for e-marking systems include: enhanced process control; improved access to wider examiner expertise; enhanced communications and support across the examiner team; richer data on examiner, candidate, item, component and paper performance; reduced time to result; raised marking quality/consistency; reduced administrative error. However, without close matching of the technical system and the assessment process, and without careful introduction to engage appropriate stakeholders benefits remain theoretical. This paper begins by describing the objectives of a project carried out by a leading UK awarding body and RM plc to introduce a comprehensive e-marking system suitable for varied response sources and operating from standardisation to awarding. It describes the approach adopted to solution design, to ensure it was assessment led, setting out the key features of the solution generated together with lessons learnt. It reports the approach taken to system introduction across a series of 8 examination sessions, ensuring both that a new system could be tested and trialled while also protecting key examination sessions. Finally it gives an overview of the results of research by the awarding body into examiner attitudes to the system following this approach. Document type: Congress Presentation ISBN:	View
Mestry, R	Portfolio Assessment as a means of evaluating the professional development of school managers Abstract: The main reasons cited for the poor matric results and declining culture of teaching and learning in many South African schools are poorly qualified principals and their lack of leadership. Principals and other school managers, who wish to improve their qualifications and consequently improve their practice, may register for a two-year course in Advance Certificate in Education for Principalship (ACE) at specific institutions of higher learning. In order to progress from the first year of study to the second year, students are assessed on a number of assignments and problem-solving activities completed in each of the coursework modules. Students are also assessed on a project plan developed in collaboration with all role-players in their schools. In the second year, the student is assessed on a number of assignments and problem-solving activities in the different modules and also on a portfolio. The portfolio is the collection of evidence that the student will compile in order to make claims of competence in the implementation of the project undertaken. The lecturers assess the completed portfolios and external site-based evaluators are appointed to verify the evidence provided by the students. This paper will focus on portfolio assessments (which include the project plan and commentary) as a means of evaluating school managers studying for a course in Principalship. Document type: Congress Presentation ISBN:	View
Wing, Chan Kam	Portfolio Assessment of Cooperative Learning Groups in Small Classes Abstract: The emphasis of learning to learn in curriculum reform has signaled to teachers to adopt student-centered strategies of teaching and different modes of assessment. Cooperative learning, as an innovative instructional strategy, and portfolio assessment, as an alternative assessment, are increasingly being used in Hong Kong classrooms. The change from listening to teachers teach and answering examination questions to taking the initiative to learn and demonstrating competence with self-selected evidence is a great challenge to the students. This paper first discusses the various constraints and difficulties of using cooperative learning and portfolio assessment that Hong Kong teachers and students are encountering, such as the large class size, using social skills in group discussion, writing reflective statements, choosing evidence for inclusion in portfolios, grading portfolios and conducting self and peer assessment. Then examples are drawn from schools of small classes to investigate how the teachers attempt to tackle the problems and difficulties in implementing cooperative learning and portfolio assessment in their schools. The paper concludes with a discussion of the implications for successful use of portfolio assessment. Document type: Congress Presentation ISBN:	View
Chun, Lo Yiu Dr	Practice and Challenges of School-based Formative Assessment Abstract: Formative assessment is regarded as a means to provide teachers and students with information about the strengths and weaknesses of student learning with the ultimate goal of making responsive changes in teaching and learning. In Hong Kong, formative assessment has recently been promoted in schools. Through analyzing a case study, this paper aims to explore: (1) the key strategies employed by the school in assessing and improving students’ performance, (2) the effects of these strategies in student learning and (3) the challenges the teachers have encountered in practising formative assessment. The intention is to provide a living example for those teachers and policy makers who favour the use of formative assessment as the major approach to improving students’ learning. Document type: Congress Presentation ISBN:	View
Carless, David Associate Professor	Pre-emptive formative assessment Abstract: This paper seeks to unpack some dimensions of formative assessment not yet fully articulated in the existing literature. It introduces the term, pre-emptive formative assessment to denote teacher actions which attempt to clarify student understandings before misconceptions have resulted in ineffective learning outcomes and/or loss of marks in assignments or examinations. It is suggested that this dimension is common in practice but its principles and practice have not yet been fully analysed. The rationale for pre-emptive formative assessment stems from key issues in the provision of useful feedback, namely timeliness and the opportunity for students to act. An example of pre-emptive formative assessment is described and some key issues in its implementation discussed. The paper concludes by arguing that pre-emptive formative assessment is worthy of wider attention, and outlines some directions of further exploration and ongoing data collection. Document type: Congress Presentation ISBN:	View
Bryer, Karen	Pre-University Project Work in Singapore: An Alternative Mode of Assessment in Singapore Abstract: In 2003, Pre-University Project Work (PW) was implemented as a national examination for all students in Singapore’s Junior Colleges (JC) and Centralised Institute (CI). PW is an alternative mode of school-based assessment that requires students to work together in small groups on a project. Assessment of PW comprises: a Written Report presenting the findings of the project on which students are assessed as a group, an Oral Presentation, on which students are assessed on a group and individual basis, and a Project File documenting processes that students have been engaged and for which they are awarded an individual mark. PW is compulsory for all JC/CI students and is one of the admission criteria for local universities. This paper aims to update conference on Singapore’s experiences in implementing this mode of assessment and will explore some of the issues that have arisen that are also pertinent to school-based assessment in general. Document type: Congress Presentation ISBN:	View
McManus, Hugh	Predicting Final Grade Distribution of Examination Results – Selection of Sample Scripts Abstract: The State Examinations Commission is responsible for administering public examinations at the end of lower second-level and upper second-level education in Ireland. Unusually for such examinations, grade boundaries are predetermined and no transformations are applied to raw scores. The raw mark therefore directly determines the grade, and no manipulation of the grade distribution is possible after the raw marks are finalised. For security reasons, examination items are not pre-tested. When scripts are being marked, therefore, early and accurate predictions of the overall distribution are essential, so that any necessary interventions can be made at the raw-mark level, by adjusting marking criteria and revisiting scripts already marked. In order to generate this initial estimate of the final distribution, each examiner marks a sample set of scripts early in the process. This paper addresses the mechanism used for selecting the sample. It does not deal with the issue of determining an appropriate sample size. It reports empirical confirmation that the mechanism currently recommended to examining teams (systematic sampling) produces better estimates than might be expected from random samples. It also confirms that certain alternative mechanisms previously used by some examining teams will produce biased samples in most circumstances. Document type: Congress Presentation ISBN:	View
Izard, John	Quality assurance: Asking the right questions. Abstract: Examinations and evaluations of performance used in universities and schools are usually high-stakes assessments. Those responsible for such assessments usually try to ensure that the tasks set as part of such assessments are of high quality, that the tasks represent a balance of relevant content, and that, as far as is possible, each candidate has a fair chance to respond to the tasks and to receive due credit for the responses. Improving the quality of assessments is taken very seriously. IAEA, ACEAB and development agencies such as World Bank, Asian Development Bank and UNESCO have invested considerable resources in conferences, training workshops, consultancies and publications relating to quality assessment. But unless one asks the right questions, those making assessment decisions may, in fact, be threatening the validity and quality of the assessments. In this paper I propose to show how some decisions about assessment strategies threaten the quality of those assessments, using analyses of actual examinations data. Practical advice on how to avoid these threats will be provided in the paper Document type: Congress Presentation ISBN:	View
Raikes, Nicholas Bell, John F. Bramley, Tom Claessen, Mark J. A.	Quality control of marking:Some models and simulations Abstract: As markers trade their pens for computers, new opportunities for monitoring and controlling marking quality are created. Item-level data may be collected and analysed throughout marking. The results can be used to alert marking supervisors to possible quality issues earlier than is currently possible, enabling investigations and interventions to be made in a more timely and efficient way. Such a quality control system requires a mathematical model that is robust enough to provide useful information with initially relatively sparse data, yet simple enough to be easily understood, easily implemented in software and computationally efficient – this last is important given the very large numbers of candidates assessed by Cambridge Assessment and the need for rapid analysis during marking. In the present paper we describe the models we have considered and give the results of an investigation into their utility using simulated data. Document type: Congress Presentation ISBN:	View
Maxwell, Graham	Quality management of school-based assessments:Moderation of teacher judgments Abstract: School-based assessment offers many benefits over external tests and examinations. These benefits include attention to a greater range of important learning outcomes, greater integration of formative feedback for improvement, and generating a performance profile over time rather than on a single occasion. These fit with current understandings of human learning and with anticipated future demands on people and economies. A critical issue is how to establish confidence in school-based assessments. Greater confidence, and therefore stronger quality management, is needed for higher stakes assessment. A key component of successful school-based assessment is teacher expertise. Two kinds of expertise are involved: obtaining good information on student learning (using good assessment procedures); and making good judgments (applying relevant performance standards). In-service teacher education programs are essential and exemplars of good practice can be useful. Self-monitoring (quality assurance) processes can be useful. However, for high-stakes assessment, some form of external quality control (moderation processes) is essential. This paper discusses various options for moderation processes and their advantages and disadvantages. Reference is made to Queensland’s three decades of experience with school-based assessment in secondary schools. Document type: Congress Presentation ISBN:	View
Ling, Toh Pei Chew, Tan Soon Ng Kok Vei, Christopher	RE-DESIGNING LOWER SECONDARY DESIGN & TECHNOLOGY CURRICULUM AND ASSESSMENT… A MODULAR APPROACH @ BEATTY Abstract: In line with the move towards "Teach Less, Learn More" (TLLM), Beatty Secondary School shares her experience on how the Lower Secondary Design & Technology (D&T) and Technical Studies curriculum and modes of assessment were re-designed. It explores the notion of ‘Less’ is ‘More’ and also ‘Assessment for learning’ vs ‘Assessment of learning"’. Often, too much focus has been given on how much (i.e. quantity) students learn instead of how well (i.e. quality) they learn. There is also a tendency to ‘teach to the test’. This inevitably leads to an over-emphasis on assessment and students’ myopic focus on learning. To address this, Beatty Secondary Design & Technology Department has re-designed its curriculum by adopting a modular approach to create more ‘white space’ to embrace a more holistic approach in learning. She has also adopted an alternative mode of assessment which enables teaching and learning to take place beyond classroom walls. It entails research, social interaction, presentation, evaluation of skills, etc. which are essential for lifelong learning. It also provides timely feedback for more effective teaching and learning. Students eventually become confident learners and take ownership of their learning. Document type: Congress Presentation ISBN:	View
Long, Clive	Realising the potential of school based assessment Abstract: Teachers and learners, quite understandably, respond to circumstances that affect them. As a consequence teachers teach to the test and learners are motivated to study for these events, especially if it is a high stakes examination. However it is, today, for many reasons, widely recognized that tests and examinations need to be complemented by alternative ways of assessing what learners know and can do. School Based Assessment (SBA) offers an opportunity to introduce these alternative ways. However to achieve this purpose, SBA must be, and must be seen to be, of major relevance to the outcome of the high stakes assessment. On the other hand examination boards and quality assurance bodies cannot allow SBA to affect the examination result significantly unless they can be confident that it is valid, fair and reliable. This paper looks at steps that have been taken by the IEB, an examination board that serves mostly well resourced schools across South Africa, to make SBA relevant and significant. These steps cover aspects like changing teacher perceptions, broadening the types of tasks used in SBA to ensure that assessment complements examinations and tests, moderation processes and the management of these processes using the internet. Examples are taken from the subject Physical Science. Document type: Congress Presentation ISBN:	View
Ridgway, Jim Professor McCusker, Sean Dr Nicholson, James	Reasoning With Evidence – Development of a Scale Abstract: Reasoning from multivariate evidence is pervasive in political speeches and in the media, but is largely absent in UK schools. Here, we present evidence that students can reason from complex evidence if they are supported appropriately with good computer interfaces. A study will be described where 195 students aged 12 to 15 years were presented with computer-based tasks that require reasoning with multivariate data, together with paper-based tasks from a well established scale of statistical literacy. All the tasks fitted well onto a single Rasch scale; computer tasks were cognitively more complex, but were ranked as being only slightly more difficult than paper tasks on the Rasch scale. Several distinct levels of reasoning are evident in student responses, associated with comprehending; manipulating; and drawing conclusions. Competence ranges from working with single values, one step computation, and elementary reasoning, through to fluency using a variety of representations, fluency with number, and in synthesising evidence and communicating results clearly. We will show examples of computer-based tasks, student work, the Rasch scale, and will describe the development of a short undimensional scale. Document type: Congress Presentation ISBN:	View
Angeles, Ma. Sampang, A. Moseros, Jason	Redesigning the CEM Mathematics Diagnostic Tests as Developmental Assessment Instruments Abstract: Current efforts of the Center for Educational Measurement to redesign its diagnostic tests in mathematics across six grade levels are guided by the developmental approach to assessment adopted by the Australian Council for Educational Research. This approach generates progress maps which place a learner’s skills and knowledge along a typical sequence of development as the learner moves within one grade level and on to the next level. Progress is measured in terms of degree of mastery of content and subsequent attainment of higher levels of performance. This is based on the notion that one’s competence in an area of learning improves over time. The progress map described in this paper is drawn from a synthesis of logical connections among the contents and skills found in the various learning areas covered by the CEM mathematics diagnostic tests. The map is the result of a series of consultations between CEM test developers and subject area experts. The competencies measured by the tests are specified by a national core curriculum – the 2002 Basic Education Curriculum – with the inclusion of some topics not part of the core but found to be commonly taken up by a surveyed sample of private schools. This paper also explores the impact of progress maps on (1) measuring a learner’s growth, and (2) aligning assessments to development across the curriculum. Document type: Congress Presentation ISBN:	View
Dandan, Guan Shiye, Ma Xin, Liu Jie, Yu	Relationship between Satisfaction with Major, Academic Performance and Congruence Abstract: Congruence is measured as a comparison between one’s score on an interest inventory and one’s chosen major in this study. 589 college students come from 11 majors of 3 universities in Beijing are measured by Vocational Interest Inventory of China (VIIC) developed by National Education Examinations Authority (NEEA). Data of satisfaction with major and academic performance are also collected. We explore the relationship between satisfaction with major and congruence index, as well as academic performance and congruence index. The results are as following: 1. Congruence between interest and major has positive influence on satisfaction with major. High congruent students have more satisfaction with major than others. 2. Congruence between interest and major also has positive influence on academic performance. High congruent students get better score than others. And 3. Satisfaction with major has partial mediation on the relationship between congruence and academic performance. Congruence between interest and major is very important, results we got will benefit for individual’s career development and school counseling. Document type: Congress Presentation ISBN:	View
Acana, Sylvia	REPORTING RESULTS OF NATIONAL ASSESSMENT:UGANDA EXPERIENCE Abstract: Uganda National Examinations Board (UNEB) has been conducting national assessment, referred to as the National Assessment of Progress in Education (NAPE), since 1996. The assessment is done annually in grades three and six in the subject areas of Literacy and Numeracy. The information emanating from the assessment is supposed to serve the following purposes, among others: Generate accurate information on what pupils know and can do in various curricular areas, provide guidelines for improving instruction and learning, evaluate the effectiveness of inputs and processes that are employed in the educational activities and provide guidelines on variables that affect achievement. However, the information is not yet being optimally used by education policy makers and practioners. This presentation outlines the methods which UNEB has adopted to report the results in order to make it user-friendly for the various key players: from the teacher in the classroom to the policy maker in the Education Ministry. Furthermore, explanation is made of how such information can be disseminated in a cost-effective manner so as to reach as many persons as possible within a short time. Document type: Congress Presentation ISBN:	View
Bowen-Clewley, Liz Farley, Melita Rowe, Ron Russell, Lauri	So what does “being qualified” really mean? A critical perspective on a growing trend of “credentialism” and its relevance in workplaces in 21st century Abstract: With the growing understanding and acceptance of the concepts surrounding informal and workplace learning, the meaning of the term “qualified” is becoming unclear. This is particularly relevant in complex work environments, where individual performance/competence may be less important than team, group or organisational performance/competence. “Qualified” can encompass individually or in combination, credentials, experience, performance, reputation and personal/team attributes/qualities and may be focused more on complexity of operations than individual knowledge and skills.This paper explores old and new understandings around what it means to be qualified, reviews the growing trend toward “credentialism” and challenges conventional views and assessment methodologies and suggests more meaningful strategies for workplaces in 21st century. Document type: Congress Presentation ISBN:	View
Matters, Gabrielle Dr	Statistical Moderation and Social Moderation around Australia Abstract: With the increased emphasis on teacher judgment in high-stakes assessment has come the role of social moderation in ensuring that student performances of equivalent standard are recognised as such before subject results can be recorded on a certificate with comparability guaranteed. With the widespread use of an index of overall achievement as the basis for offering university places has come the role of statistical moderation in ensuring that the results of different assessments (different subjects, different sites) are on a common scale before being combined to produce a tertiary entrance rank. Social moderation and statistical moderation are two of a variety of techniques for linking results of one assessment to those of another (Linn, 1993). And there are variations within each of these techniques in practice. Australia has eight different systems for senior curriculum, assessment and certification. The federal government has recently proposed the introduction of a single Australian Certificate of Education. The process of exploring a way forward has included an analysis of similarities and differences in current arrangements − including ways of validating teacher judgments and ‘scaling’ subject-group results. This paper outlines the theoretical underpinnings of statistical moderation and social moderation, and describes applications of these − form and purpose − in various Australian states/territories. Document type: Congress Presentation ISBN:	View
Atkins, Susan O’Connor, Gayl	Taking learning objects in new directions: models for assessment. Abstract: The Le@rning Federation (TLF) project employs emerging technologies to produce online curriculum content to encourage student learning and support teachers in Australian and New Zealand schools. TLF has adopted the learning object model for its online interactive content. Learning objects are ‘chunks’ of digital material - for example graphics, text, audio, animation, interactive tools - specifically designed to engage and motivate student learning. Several assessment models are proposed to demonstrate how these chunks can be re-purposed to specifically assess student learning. The models have the common purpose of assessing targeted teaching and learning outcomes: students are afforded opportunities to develop and demonstrate these learning outcomes as they interact with the learning object. Capturing and assessing the learning that has taken place as the student interacts with the learning object is the principle that underpins each assessment model proposed. It is by capturing students’ responses as they interact with the learning objects that insights into student learning can be gained. By establishing what the learner knows – and is able to do – through engaging with a learning object, the next appropriate learning task can be identified: to either extend learning, or to target areas requiring additional support for students in their learning. Document type: Congress Presentation ISBN:	View
Mercurio, Antonio Dr	Teacher-based Assessment at the Upper Secondary Level of Education ? an Exploration of the Interaction between ‘Teachers as Assessors’ and ‘Teachers as Moderators’ Abstract: Upper secondary schooling is characterised by high‐stakes assessment  systems used to certify students’ achievements at the conclusion of this period of education. In Australia, as in many other countries, whether the assessment system consists entirely of school‐based assessment components,externally marked assessment components, or a combination of school‐based and externally marked assessment components, the judgments about students’ achievements are typically made by teachers. This paper examines the teacher‐based assessment system as it has evolved over the past 30 years in an Australian statutory assessment authority withlegislative responsibilities for assessment and certification of the achievements of upper secondary students. It maps the variety of methods that are used at present to moderate school‐based and external assessment components. Importantly, it explores the nature of the relationships between two pivotal roles performed by the teacher: assessor (of the standard of work of his or her own students) and moderator (of the standard of work of other teachers’ students). It interrogates these relationships in relation to three goals: being fairer to students; building the capacities of teachers to understand assessment processes and thereby improve learning; and strengthening public confidence in the fairness of the judgments. The paper suggests that a teacher‐based assessment system can be a solid basis for achieving these goals. Document type: Congress Presentation ISBN:	View
McCurry, Doug Mackenzie, Margaret	Teachers Making Contextualised, Group Judgements of Generic Skills and Dispositions Abstract: This paper outlines a project of the Victorian Curriculum and Assessment Authority, in collaboration with the Australian Council for Education Research that has been running since 2001. Different teachers of a student make independent assessments of the generic skills and dispositions of that student and these separate judgements are subsequently brought together into a single overall or whole-school report. The process is supported by specially developed computer software to synthesise the judgements of different teachers. The judgements made are generic and cross-curricular rather than subject-based. More than 10 000 students in more than 100 schools have been assessed with the procedures developed in the project since 2001. The trials have shown that participating teachers are able to make global, impression judgements of generic skills and dispositions, and that the assessment process developed by the project is cost effective. The trials also showed a significant degree of agreement between different teachers about the individual student. The degree of agreement between teachers from across the curriculum made the production of an overall, collective or whole-school report both meaningful and feasible. Participating schools have used this generic skills assessment and reporting model for a variety of purposes. Document type: Congress Presentation ISBN:	View
Philips, David Dr	Teachers’ and Students’ Experiences of Standards-Based Assessment in New Zealand Abstract: The implementation in New Zealand of a standards-based assessment system for secondary qualifications since 2002 has changed the focus of assessment in the last three years of secondary schooling. Students are now assessed against both internally assessed (school-based) and externally assessed (national examinations) standards. The standards, which are registered on the National Qualifications Framework, have a credit value, and attainment of sufficient credits leads to the award of a certificate, generally the National Certificate of Educational Achievement (NCEA). These are at different levels (1, 2 and 3) comparable to each year (11, 12 and 13) of schooling. This paper examines Year 13 teachers’ and students’ experiences of internal and external assessment associated with the NCEA based on feedback from a range of studies, using information from interviews and surveys. The issues identified by the teachers and students are evaluated within a framework reflecting the key principles underpinning standards-based assessment in order to show how standards-based assessment is impacting on teachers’ classroom practices and students’ attitudes towards gaining national qualifications. Implications for national assessment systems using a standards-based approach and for the development of secondary school qualifications are discussed. Document type: Congress Presentation ISBN:	View
Berry, Rita Dr	Teachers’ assessment practices for classroom diversity Abstract: Classrooms nowadays are exemplified by student diversity. Students may differ in, for example, motivation, learning style, aptitude, cultural background, social economic states, and past learning experience. As students’ learning needs are becoming increasingly diverse, a new challenge is to look for ways that address their needs. Assessment is one useful way to help teachers deal with classroom diversity. Teachers can use assessment strategies to identify different learning needs and to improve teaching and learning. This subsequently leads to a consideration of two critical issues with the first one being how teachers can plausibly identify classroom diversity and the second being how useful are the assessment strategies they use to cater for the identified diversity. This paper presents and discusses the findings of an investigation into the assessment strategies that teachers used to identify classroom diversity and the assessment strategies teachers used to deal with the identified diversity. About one hundred Hong Kong teachers were interviewed, revealing that the teachers used a number of assessment strategies to assist them in understanding students’ diverse needs and that the teachers seemed to be more interested in identifying those students who needed help academically rather than in their needs in the social and affective domains. Document type: Congress Presentation ISBN:	View
Bennett, Randy Elliot	Technology and Writing Assessment: Lessons Learned from the Abstract: In this paper, I review the results of two research studies conducted for the US National Assessment of Educational Progress (NAEP). The studies concern doing writing assessment on computer. The first study examined the comparability of computer and paper tests of writing. Among other things, this study looked at differences in performance between nationally representative groups taking the same writing test in those two delivery modes. The second study examined the meaningfulness of scoring student responses automatically, including the degree to which writing experts and automated routines agreed in weighting the various components of student writing to create test scores. Document type: Congress Presentation ISBN:	View
Vernooy-Gerritsen, Marjan PhD	The CITO Examination System Abstract: Dutch schools all are online, which opens possibilities for a fully web based assessment system. But what is online? Cito has to deal with a wide variety in bandwidths, intranet systems, firewalls, and proxy servers. Furthermore, schools use a mix of Operation Systems, Media Players, and Browser applications. Under these circumstances Cito has to deliver multimedia examinations with video fragments and sound tracks simultaneously to 200.000 candidates. How can this be done? Cito managed to develop an Assessment system for a range of Microsoft Operation Systems (W95 to XP) for both stand-alone situations and local area networks. The Cito Examination System does not need third party applications for browsing or multimedia presentation. Distribution of multimedia test packages and data return of test results can be handled by installing and returning files or by automated synchronization techniques with the central Cito Data Server. The test content is readable only during test taking by a candidate. Test packages and test data are encrypted, and transmission of files over the Internet is secured by the Secure Sockets Layer (SSL) protocol. Document type: Congress Presentation ISBN:	View
Navon, David Cohen, Yoav Dr.	The Computer as a Silent Partner in Essay Sc Abstract: Psychometric measurement based on subjective judgments of performance quality (e.g., essay ratings) is, typically, not very reliable. The subjective judgments are often integrated into a single score by means of the following scoring model: Initially, two independent judgments are conducted; then, if the absolute difference between them is not too large, their mean is used as the score. Otherwise, an additional judgment is conducted, and the score is determined by mean of the third judgment and whichever of the original two is closest to it. Whenever the two judgments are sampled from the same distribution, their mean is an unbiased estimate of the true score. However, quite surprisingly, substituting any one of the judgments according to the scoring model described above would result in increased error variance. In some domains, such as the rating of short essays, it is possible to attain a high level of agreement between a human judgment and a mechanical judgment (Automatic Essay Scoring – AES) based on fairly simple considerations. Though it is not common practice to rely absolutely on AES, the aforesaid high level of agreement suggests that a model employing the difference between a mechanically generated score and a score generated by a human judge is worth considering. Accordingly, we propose that the following model be put into practice: In the initial phase, two judgments are obtained, one human and the other mechanical. It follows from the logic described above that a large difference between the two scores indicates the likelihood that the human-generated score is fairly far from the true score. Since, in some situations, the validity of correcting for this by averaging that score with the mechanically-generated one is disputable, the recruiting of another human judge is called for. The overall cost of judgment would be substantially reduced by reducing the considerable rate of scores generated by human judges that have to be corrected in this manner. The study explores the benefits of using this model. The current study, which is based on simulated essays and scores, explores the error of measurement associated with various scoring rules. Document type: Congress Presentation ISBN:	View
Mikulas, Cathy Edelblut, Paul	The Effects of Inclusion of Native Speakers' Writing Samples on the Domain Scoring Accuracy of Automated Essay Scoring of Writing Submitted by Taiwanese English Language Learners Abstract: While the scoring accuracy of automated scoring of essays written in English has been established, more research is needed with regards to domain scoring for English Language Learners. This paper presents findings regarding the effects of training set composition on the domain (Focus and Meaning, Content and Development, Organization, Language Use and Style, and Mechanics and Conventions) scoring accuracy of essays submitted by Taiwanese students scored by an automated essay scoring system. Typically, each scoring model created is based on a set of previously scored essays. This study compares the accuracy of scoring the same set of essays written by Taiwanese students using two different models: one model using blended native and ELL essays and one using a set of entirely ELL essays. While both models yielded adjacent agreement rates from 98 to 100 percent across the domains, there were differences at the exact agreement level. Exact agreements for the model developed using the ELL training set ranged from 50 to 64 percent, while the blended training set resulted in exact agreements ranging from 66 to 76 percent. Pearson correlations for the two models were very similar (.83 to .89 for the first and .84 to .90 for the second). This study supports the use of a blended training set. Document type: Congress Presentation ISBN:	View
Kaur, Berinderjeet Ercikan, Kadriye Koh, Kim Hong	The Impact of Item Formats on Singaporean Students’ Performance in the Trends of International Mathematics and Science Study Abstract: In many of the large-scale international assessments (e.g., TIMSS, PISA), both multiple-choice and constructed-response item formats are used to assess student achievement. In terms of measuring learning outcomes, it is widely accepted that multiple-choice items are limited to measuring factual knowledge and simple recall skills. On the contrary, constructed-response items are known as more effective tools for assessing deep understanding of content knowledge and higher-order thinking skills. However, some researchers have shown that multiple-choice and constructed-response items measured the same basic trait or proficiency. Based on the released TIMSS 2003 reports, Singaporean students were among the top performers in both mathematics and science at the 4th and 8th grade levels. But yet little is known about the effects of item formats on the Singaporean students’ performance. Are the multiple-choice and constructed-response items measuring the same cognitive and knowledge domains? This study will report the results of the construct comparability of the multiple-choice and constructed-response items in the TIMSS 2003 mathematics achievement test, Singaporean Grade 8 population. The impact of the item formats on student performance will be discussed. 1 Document type: Congress Presentation ISBN:	View
Okumbe, Matthews Adera Tsheko, G. N.	THE NEED FOR CURRICULUM MODIFICATIONS FOR SPECIAL NEEDS LEARNERS IN BOTSWANA Abstract: Curriculum modifications can be put into practice for different purposes ranging from altered content knowledge, conceptual difficulty, educational goals, to instructional methods and assessment. Nevertheless, curriculum modification still remains an ambiguous concept especially in relation to the concept of special needs learners, special educators, and to some extent curriculum implementers. This paper will endeavour to discuss the concept of curriculum modifications in order to understand its nature and potentials. It will also analyze some categorizations of curriculum modifications as suggested by King-Sears (2001), which include, accommodation, adaptation, parallel curriculum outcomes, overlapping curricula on a continuum and to see to what extent these modifications can be applied efficiently and effectively to meet the needs of special needs learners in Botswana. The paper will hence make some recommendations regarding the best curriculum modification(s) that can be used by special educators to harness the potentials of special needs learners. Changes in the curriculum mandate changes in assessments based on the curriculum. This paper will therefore also discuss the accompanying changes in assessment to enhance a quality education for learners with special needs. Document type: Congress Presentation ISBN:	View
Visser, Alvin Hanslo, Monique	The Predictive Validity of an University Admissions Test: A Survival Analysis Approach Abstract: This paper investigates methodological issues associated with predictive studies related to selection and access to Higher Education. These issues are discussed in the context of tests designed and administered by the Alternative Admissions Research Project (AARP) at the University of Cape Town. One of the aims of the project is to design tests that broaden access to talented previously disadvantaged students who are not easily identified through the High School Senior Certificate examination system. The tests attempt to provide a mechanism for selection based upon whether the writers have the potential to succeed in the University environment. The paper comments on the difficulties and limitations of various approaches to predictive studies The paper argues that a methodology grounded in the survival analysis approach holds particular promise for predictive studies and that it can provide a useful insight into the complex processes of student persistence, retention and attrition rates. Document type: Congress Presentation ISBN:	View
Harris, Karen Pitman, John Professor	The QAT (Queensland Assessment Task): Inventive authentic assessment designed to engage early adolescents of all ability levels. Abstract: The QAT is the first attempt at common statewide assessment (apart from basic skills testing) in the compulsory years of schooling in Queensland since the ‘70s.The 2005 QAT for Year 9 students was made up of three standardised assessment tasks in different assessment modes, namely: Task 1 – interactive, computer-based Task 2 – constructed response, paper-based Task 3 – performance-based. It was designed to be intellectually challenging and have connections to the wide world. The QAT assessed generic skills and dispositions in the area of Transforming ideas and/or information with an emphasis on Processing. To a lesser extent it also assessed Knowledge (facts, concepts and procedures) from the curriculum areas of SOSE (Studies of Society & the Environment) and The Arts.Authenticity was achieved by supplementing conventional testing with an interactive computer-based approach and a performance task — with the suite to be administered’ over extended time and close to classroom conditions, to challenge a traditional negative concept and aura of ‘the test’. It was a foray into tasking.The computer-based task required students to interact with the computer at all levels: comprehensive stimulus and support materials, access to questions, and provision of responses. The performance-based component of the QAT was not administered as its viability had been established from recent allied research in Queensland. Student results showed a high level of reliability and provided intriguing data on subgroup differences. The paper presentation will include illustrations of the student experience of the computer-based mode of assessment. Document type: Congress Presentation ISBN:	View
Salim, B. A. Prof	The Role of Questioning Prompts in Conversation: A Study of Pupils’ Performance in a Secondary School EL Oral Assessment. Abstract: Pupils’ performance in the English Language oral examination is never an isolated outcome. The examiner’s role would in some ways, affect how well a student performs. A detailed analysis of the discourse characteristics of the oral interactions could help to highlight this relationship. This study considers the effect of examiners’ prompts in the conversation session of the examination. Based on conversational analyses of 17 full-length oral interactions, it is argued that assessments of students’ performance are reflected in the extent of promptings by the examiners and that the variation in results may provide a useful understanding in validating proficiency and providing a parameter for examiners’ conduct. In general, the findings of this study suggest that the examiner plays an important role in students’ display of oral proficiency. Apart from the questionings, factors like linguistics, pedagogical and social, interact in influencing the performance of test takers. Document type: Congress Presentation ISBN:	View
Boyle, Andrew	The validity of an innovative on-screen assessment Abstract: This presentation will describe work done on the new Key Stage 3 ICT (Information & Communications Technology) test in England. This new test is intended to measure the ICT capabilities of all pupils at the end of the lower secondary phase of education. It is a highly innovative on-screen test; to be delivered, for the first time in 2008, across all secondary schools in England. Specifically, this presentation will describe work done in 2005 to determine whether the test was likely to be valid – given that 2005 was a pilot year. The presentation will describe how validity has been defined as a multi-faceted, yet unitary, construct, and how multiple sets of data (or varying types and addressing varying purposes) were collected to provide evidence for evaluation of the test’s validity. The presentation will then go on to describe interpretative approaches taken to arrive at a fair evaluation of the 2005 pilot test’s validity; and conclude by describing lessons learned, and suggesting how validation and evaluation can be strengthened in future years. Document type: Congress Presentation ISBN:	View
Tan, Kelvin Heng Kiat	Understanding student self-assessment in terms of learning, grading and empowerment Abstract: It is generally agreed that there has been a shift in the emphasis in higher education from focusing on what teachers do to an emphasis on the centrality of the student as a learner. This has resulted in a stronger emphasis on the involvement of students in their programs of study in higher education. In these contexts, student self-assessment is a form of alternative assessment that reflects the more visible role that students have in the assessment process. Whilst the general importance of self-assessment in higher education has been embraced, the definition and practice of self-assessment is subject to diverse and at times incompatible discourses. This paper provides insights on understanding the position of self-assessment in higher education by identifying its distinct identity in terms of learning, grading and empowerment. It is argued that learning, grading and empowerment poses different and at times opposing purposes for self-assessment. Enhancements are suggested for enhancing self-assessment in each of these purposes. The paper concludes with recommendations for understanding and using self-assessment in ways that may fulfil the different purposes of learning, assessment and empowerment at the same time. Document type: Congress Presentation ISBN:	View
Zhang, BaoHui Wong, Lung Hsiang Chew, Lee Chin Jacobson, Michael J. Looi, Chee Kit	Using Computer-based Modelling for Primary Science Learning and Assessment Abstract: Computer-based modeling is not just a means for students to learn important scientific knowledge and skills, but also a technique to assess student understandings of science. A software tool called Model-It allows young students to create their own models so that their learning becomes more interactive and engaged. However, there is a mismatch between how students learn and how they are assessed if conventional paper-administered tests are used. This paper argues for alternative assessments to be better aligned with curriculum and instruction. Forty 4th grade students in a local Singapore school participated in a science inquiry activity that involved learning with modeling as an alternative assessment. The students individually created models of food webs to illustrate their understanding of energy flows and photosynthesis. A scoring rubric based on four criteria ("focus and structure", "accuracy", "completeness" and "functionality") was used to evaluate the models, with the modeling scores being compared to student scores of the school’s paper-based assessments of science learning. In addition, 18 students were interviewed about their understanding of models and modeling. The data is currently being analyzed and the findings of this study and potential implications for educational assessments will be presented in this paper. (200 words) Document type: Congress Presentation ISBN:	View
Roads, Martyn	Using Technology to Enhance Formative Assessment Abstract: paper describing the issues raised in the development of formative assessment tools for the UK DfES “Skills for Life” programme. The tools are an essential part of ensuring that learners on Literacy, Numeracy and ESOL programmes are placed on programmes from which they gain maximum benefit. Technology has enabled the development of easily administered tools. They use sophisticated algorithms to track users’ progress and ensure that teachers get a comprehensive and reliable set of data on the performance of the learner. This helps them develop a tailor-made learning programme. Using technology has been cost effective, enabling the project team to develop tools for general use, the workplace and for specific employment sectors such as health and retailing. The paper deals with the challenges of developing tools that are engaging whilst reliable and on making the technology accessible to a wide range of users. The writers have been associated with the development of eassessment over a number of years. Document type: Congress Presentation ISBN:	View
Abbaszade, M Huseynov, R	Using the results of the entrance examinations to the universities during the examinations to the master degree Abstract: Since 1997 the universities of Azerbaijan passed on to a new two stage education system for our country, bachelor and master stages. The admission procedure to master programs for that time was not well-developed from scientific side and had many obstacles which prevented talented youths for the entrance. And only from April of 2005 when State Students Admission Commission was entrusted to carry on the entrance examinations to the master programs, all these obstacles were eliminated and most of the students got a chance to apply to the master programs. It was decided to get nearer to the standards of examinations like as GMAT and others. But for this year it was decided to use the transition model using the test exam in the first stage and the written exam in the second stage for those who passed the test exams successfully. Before the beginning of exams all information about the candidates applying to the master programs were analyzed and an additional surplus value was determined to knowledge of each candidate. The first exam showed us how different the marks gained in the universities are from the real knowledge of the students. The analysis of results of exams to the master programs allowed to give a number of valuable recommendations for the improvement of activity of the universities. Document type: Congress Presentation ISBN:	View
Elwood, Jannette	Views of assessment, learning and mind: exploring the links and the implications for emerging trends and perspectives in assessment Abstract: This paper considers the fundamental relationship between learning and assessment and also introduces views of mind as a further key concept in understanding assessment practice. These concepts are building blocks for fully understanding emerging trends and practice in assessment. For example, the Assessment for Learning movement that promotes different forms of assessment practice tends to focus on the individual student, their learning and how this can be developed through particular assessment practices. This movement has a particular view of learning, the learner and of assessment but the associated view of mind still sees the student as separately analysable after the learning has taken place. This view of practice has been considered a ‘paradigm shift’ in how we think about assessment practice. I will consider that the ‘paradigm shift’ actually occurs elsewhere in theoretical positions that consider socio-cultural views of the learner with associated non-local views of mind. From this position we view students’ achievements and performance on assessment items and tasks as entangled and in relationship; a student’s ability can only be understood in relation to the teacher, the student and the measurement instrument. Thus we cannot explain learning or achievement only describe them in cultural, social and historical context. Document type: Congress Presentation ISBN:	View
Noijons, José	What does it mean for a test to be at a particular level? Abstract: Increasingly, language tests in Europe are being linked to the Common European Framework of Reference (CEFR) developed by the Council of Europe. The framework distinguishes levels of language competence in terms of descriptors or can-do statements. There have now been a number of studies in which testing agencies claim that their tests are at a particular CEF-level. In many cases, this has as yet been done through content analysis. Such analyses may successfully show that tests typically cover descriptors that are mentioned in the CEFR at a particular level. However, these studies do not always indicate what score the candidate needs to reach to be given a particular CEFR level. It is one thing to pass a test, it may be another thing to pass the test at a particular CEFR level. At Cito, the Dutch National Institute of Educational Measurement, research has been carried out on the levels of existing foreign language examinations produced by Cito. An important phase in this research has been standard-setting. It has been found that for most examinations it is possible to say: within this range of scores the test taker is at this CEFR level and within another score range on the same examination, the test taker is at a higher CEFR level. However, the existing pass/fail score (the sufficient/insufficient score) has no relationship with these levels and may be anywhere in the possible score range of the examination. Document type: Congress Presentation ISBN:	View
Gleave, David	What’s holding things up? Abstract: It is more than five years since the Web-Based Education Commission to the President and Congress made the pithy and wickedly accurate observation: "Too often today’s tests measure yesterday’s skills with yesterday’s testing technologies". How much has really changed in those five years? Education systems are driven by the demands of public examinations, both national and international, which control key rights of passage and bestow qualifications. Such examinations often carry the duty of maintaining cherished historical "standards" and this can make them quite conservative in their response to change. Web-based technologies have the power to completely transform educational assessment, not only making it much fairer and many times more valid and reliable, but turning it into an integral part of the learning process with very real benefits for every student. The great educational advances promised by the Internet will only be realized when public examinations adapt their methods and procedures to take full advantage of the opportunities offered by new technology. A good starting point might be to increase the weighting given to continuous assessment, which web-based systems can handle particularly well, and apply best practice here where it will have the most immediate effect. Document type: Congress Presentation ISBN:	View
Knight, Mark	“Where are we going? - An Update on Workplace Language Benchmarking in China and the Region” Abstract: In recent years there have been several projects in China and the region that have helped organizations, both government and commercial, decide on language benchmarks for their staff and management. Prior to these projects, surprisingly little work had been done in this area and so, through trial-and-error, a basic approach has been developed. Very often Human Resource staff are unaware of the complexities of researching language level requirements for their employees. In particular, there is a real need to explain implications of setting benchmarks from political, financial and language learning perspectives. As organizations continue to think regionally, so must language testing and benchmarking. Links have been forged in a few countries but more needs to be done if we are to fully meet the needs of expanding businesses. There are also a number of areas that need further research. This paper aims to describe some of the work that has been done so far, describe the processes that have been developed and to identify areas that need to be looked at more closely. Document type: Congress Presentation ISBN:	View

• Home
• About Us
• Conferences & papers
• Documents
• News
• Member-finder
• Job Postings
• Links
• FAQ
• Sponsors

Login