Evidence type | Finding |
---|---|
Evidence based on test content | |
Logical analysis | Experts agreed the items were representative of the literature review and the target construct |
Evidence based on internal structure | |
Item fit | Acceptable |
Dimensionality | Three dimensions (personal conflict, family conflict, community conflict) supported |
Precision | Acceptable |
Reliability | Acceptable |
Measurement invariance | Present. Pre-post comparisons of SECM measures would be appropriate |
Evidence based on relationships with other variables | |
Convergence with measure of similar construct | Family conflict was significantly correlated with a modified IOS item. Evidence of convergence needed for personal and community conflict |
Evidence based on response processes | |
Respondent cognition related to test | Partially addressed (i.e., “community”). Evidence is also needed to evaluate respondents’ interpretations of items not studied (e.g., using methods such as “think-aloud” interviews) |
Validity generalization | |
Validity studies in different geographic, institutional, and demographic contexts | Not addressed. Evidence needed to evaluate if the inferences made from the instrument generalize to other contexts |
Evidence of consequences | |
Outcomes of instrument use | Not addressed. Consequences (e.g., positive, negative) of instrument implementation for respondents and associated educational programs and systems |