Item Analysis With Exam Soft: Do you know how your exam questions are performing?

Published:
July 2022

Exam data provides students with important feedback, but exam creators can also use post-exam data to improve test questions and exams as a whole. The word “psychometrics” can be daunting to some educators, but ExamSoft’s item analysis report makes psychometrics easy. We provide an explanation of how your questions are performing and how to make them even stronger to assess your students more accurately.

Item Difficulty Index (p-value): P-value shows what percentage of exam-takers got this question correct. What counts as a good p-value depends on the intention of the question. This stat is your first indication of how your question performed but shouldn’t be the only piece of data you use to determine validity.

Discrimination Index: This index shows the difference in item performance between top and bottom performing students. If lower performing students are getting this question correct more often than your top performing students, there could be a problem with the item. A good index of discrimination can vary, based on other stats from this particular item.

Upper/Lower 27 Percent: Of the students who scored in the top or bottom 27 percent of exam takers,  what percentage of those specific subgroups got the question correct. This can be useful in determining what distractors drew students away from the correct answer.

Point Biserial: Indicates whether getting the question correct correlates positively or negatively with performing well on the exam as a whole. Unlike the discrimination index, point biserial incorporates performance of all students instead of the two specific subgroups.

Kuder-Richardson Formula 20 (KR-20): Indicates the reliability of the assessment, essentially showing whether an assessment was poorly constructed or a valid measure of knowledge. The degree to which an assessment is free from random error, yielding the same result across multiple administrations the same cohort. An ideal KR-20 score is close to one, on a scale of zero to one.

Procedure to View the Legacy Portal Item Analysis
(https://community.examsoft.com/s/toparticles#scoring)

1. Select the Assessments menu.
2. Select the Assessment for which you would like to run the report.
3. Select Reporting/Scoring, and then select Item Analysis.
4. Select the question types, categories, etc., then select View Report.
5. Use the toolbar to navigate and to export or print the report.

Please visit https://examsoft.com/resources/item-analysis-with-examsoft/ for this and additional exam reporting information.