Journal article
Using retest data to evaluate and improve effort-moderated scoring
2020
Journal of Educational Measurement,https://doi.org/10.1111/jedm.12275
Abstract
There has been a growing research interest in the identification and management of disengaged test taking, which poses a validity threat that is particularly prevalent with low‐stakes tests. This study investigated effort‐moderated (E‐M) scoring, in which item responses classified as rapid guesses are识别和排除在得分。使用achievement test data composed of test takers who were quickly retested and showed differential degrees of disengagement, three basic findings emerged. First, standard E‐M scoring accounted for roughly one‐third of the score distortion due to differential disengagement. Second, a modified E‐M scoring method that used more liberal time thresholds performed better—accounting for two‐thirds or more of the distortion. Finally, the inability of E‐M scoring to account for all of the score distortion suggests the additional presence of nonrapid item responses that reflect less‐than‐full engagement by some test takers.
See MoreThis article was published outside of NWEA. The full text can be found at the link above.
Associated Research
Related Topics
An investigation of examinee test-taking effort on a large-scale assessment
Most previous research involving the study of response times has been conducted using locally developed instruments. The purpose of the current study was to examine the amount of rapid-guessing behavior within a commercially available, low-stakes instrument.
Topics:Measurement & scaling,School & test engagement,Student growth & accountability policies
These studies are conducted based on assumptions under regular conditions for fixed test forms, such as no missing responses and normal distribution of unidimensional ability for a population.
Topics:Measurement & scaling,Computer adaptive testing,Item response theory
This study, using real data, provides empirical evidence of construct and invariance construct of MAP scales across grades at different academic calendars for 10 different states.
Topics:Measurement & scaling,Test design
The current investigative study uses a multiple-indicator, latent-growth modelling (MLGM) approach to examine the longitudinal achievement construct and its invariance for MAP Growth.
The potential of adaptive assessment
In this article, the authors explain how CAT provides a more precise, accurate picture of the achievement levels of both low-achieving and high-achieving students by adjusting questions as the testing goes along. The immediate, informative test results enable teachers to differentiate instruction to meet individual students’ current academic needs.
Topics:Innovations in reporting & assessment,Measurement & scaling,Student growth & accountability policies
The utility of adaptive testing in addressing the problem of unmotivated examinees
This integrative review examines the motivational benefits of computerized adaptive tests (CATs), and demonstrates that they can have important advantages over conventional tests in both identifying instances when examinees are exhibiting low effort, and effectively addressing the validity threat posed by unmotivated examinees.
Topics:Measurement & scaling,Innovations in reporting & assessment,School & test engagement
