Proposal Title
Examining Exams: What Makes For A Good Question?
Session Type
Presentation
Room
PAB 106
Start Date
10-7-2013 1:15 PM
Keywords
first year chemistry, item response theory
Primary Threads
Evaluation of Learning
Abstract
Perhaps the most important task confronting educators is the accurate assessment of student learning. All efforts at reform or innovation must ultimately address the question “Has student learning improved?”. In addition, the movement towards individualized learning must be able to answer the question “Has THIS student’s learning improved?” These questions will require that as educators, we develop the tools and the expertise to critically evaluate our assessment instruments. Large first-year science classes permit the application of learning measurement theories that are at the leading edge of cognitive and educational science. At the University of Guelph, first year Chemistry has approximately 2400 students who complete several multiple choice exams throughout the year in principally two courses. When a student’s grade is calculated at the end of the year, what should that grade mean? A careful study of exam questions using modern learning measurement theories begins to inform the process of exam creation and interpretation. Classical Test Theory (CTT) is commonly used to analyze the performance of both students and exam questions, but this older theory has several well-known weaknesses. Item Response Theory (IRT) is better suited to effectively characterizing the performance of both exam questions and students. I will present data on the analysis of exam questions used at Guelph over the past several years to understand the features that lead to better measures of student learning. Future pedagogical changes will be more readily accessed with these measurement tools.
Examining Exams: What Makes For A Good Question?
PAB 106
Perhaps the most important task confronting educators is the accurate assessment of student learning. All efforts at reform or innovation must ultimately address the question “Has student learning improved?”. In addition, the movement towards individualized learning must be able to answer the question “Has THIS student’s learning improved?” These questions will require that as educators, we develop the tools and the expertise to critically evaluate our assessment instruments. Large first-year science classes permit the application of learning measurement theories that are at the leading edge of cognitive and educational science. At the University of Guelph, first year Chemistry has approximately 2400 students who complete several multiple choice exams throughout the year in principally two courses. When a student’s grade is calculated at the end of the year, what should that grade mean? A careful study of exam questions using modern learning measurement theories begins to inform the process of exam creation and interpretation. Classical Test Theory (CTT) is commonly used to analyze the performance of both students and exam questions, but this older theory has several well-known weaknesses. Item Response Theory (IRT) is better suited to effectively characterizing the performance of both exam questions and students. I will present data on the analysis of exam questions used at Guelph over the past several years to understand the features that lead to better measures of student learning. Future pedagogical changes will be more readily accessed with these measurement tools.