Skip to main content

Table 1 Types of test validity evidence

From: Concept inventories as a resource for teaching evolution

Evidence based on… Overview Example
… test content Checking the match between the assessment’s content and what it claims to measure Getting the assessment reviewed by multiple content experts to ensure even and thorough coverage of a concept
… response processes Making sure that the test takers’ responses align with the underlying concepts being measured Think-aloud interviews with students where they explain the reasoning behind each answer they chose on the test
… internal structure Analyzing how the assessment questions relate to each other and map onto the underlying concepts being measured Using a statistical modeling approach like factor analysis to analyze how responses to different questions correlate
… relations with other variables Relating test scores to external variables. The external variable could be the results of a similar test (convergent evidence), a test expected to be different (discriminant evidence), or non-test outcomes like students’ future course enrollment decisions Comparing the assessment to a previously established assessment of similar concepts, expecting to find a positive correlation in student scores
… consequences of testing Evaluating the soundness of using the assessment in practice, including potential impacts of the testing itself and appropriateness of interventions made based on student results For an assessment being used to group students into leveled sections, gathering evidence on the learning outcomes of this intervention