Designing an Intelligent Question Design Information System (IQDIS)

Designing an Intelligent Question Design Information System (IQDIS)

[featured_image]
  • Version
  • Download 64
  • File Size 258.02 KB
  • File Count 1
  • Create Date August 2, 2018
  • Last Updated August 2, 2018

Designing an Intelligent Question Design Information System (IQDIS)

The primary objective of the research project described in this paper was to improve and simplify the assessment design process by building an intelligent information system, “,IQDIS”, (Intelligent Question Design Information System) for generating assessments. The key features of the system include rationalisation of assessment tasks based on both a modified cognitive taxonomy approach and a built-in mechanism for assessment quality assurance that includes an analysis of the allocation of marks. This paper discusses the main results obtained in the first project phases: determining the conceptual platform and developing a quantitative approach to quality assurance of assessments. It also describes the main functions of the new information system. To determine the conceptual foundation, we considered a layered approach to the design of questioning strategies based on a hierarchical model for the cognitive domain, commonly known as ‘,Bloom’,s taxonomy’,. We have further developed the cognitive model, introducing a third dimension to the taxonomy. This modification allows for reflection on the relative difficulty of questions pertaining to different cognitive levels, and also allows for quantitative evaluation of an assessment.We have also developed an algorithm for evaluating the distribution of marks for assessment tasks. This algorithm enables analysis of trends in mark allocation in respect of different cognitive categories and evaluation of the impact of each category on assessment design. This algorithm has underpinned significant progress towards quantification of the quality assurance of assessment question sets. ,The suggested approach enables assessment designers to perform accurate quantitative comparisons and evaluations of assessments that include such key elements as the coverage of learning objectives, addressing different levels of the cognitive process, the variety of question types, and ensuring fairness of mark allocation.

Attached Files

FileAction
paper_3017278f.pdfDownload