Fig. 2

Position effects on item difficulty. GeDI items appear on the X axis and mean item measures for each GeDI item are plotted on the Y axis. Symbols denote scenario position (1, 2, 3, or 4) in a counterbalanced rotation sequence of scenarios and accompanying item suites (see “Methods” section for description). Error bars represent two standard errors of measurement about each mean item measure. As an example, item 4 showed little variation in item measure regardless of presentation order, and was also easier overall than item 13. In contrast, item 13 showed a slightly larger variation in item measure by position. Overall, no substantial differences were found between item difficulty and item position when controlling for the number of tests