Skip to main content

Table 3 Summary of validity and reliability evidence for the GeDI

From: Testing validity inferences for Genetic Drift Inventory scores using Rasch modeling and item order analyses

Validity/reliability evidence type and descriptiona CTT framework (Price et al. 2014) Rasch framework (present study)
Construct validity
 Instrument appropriately represents the specified knowledge domain
Textbook analysis, expert survey, student interviews, review of student work and literature review for misconceptions Rasch model fit, Rasch dimensionality analysis, item fit, person reliability
Substantive validity
 Participants use the thought processes that were anticipated for each item
Student interviews (None)
Internal structure validity
 Items capture a single construct
Cronbach’s alpha Rasch dimensionality test, person and item reliability
External structure validity:
 Scores are appropriately associated (positively or negatively) with an independent measure
(None) (None)
Generalization validity
 Score inferences hold true in different administrative contexts
Five campuses over two geographic areas (Southeast/Midwest) New population (Northeast)
Consequential validity
 Considers positive or negative consequences of score use
Not applicable Not applicable
Reliability
 Reproducibility of scores
Test–retest Item and person reliabilities
Item properties
 Individual item performance characteristics
Difficulty, discrimination Item measures, item fit statistics, Wright map
Item order effects
 Possible item interactions and associated sequence biases
(None) ANOVA of Rasch-scaled scores from forms rotating item-suite order
  1. a Based on Campbell and Nehm (2013); Messick (1995); Nitko and Brookhart (2010)