Table 8 Summary of GAENE studies on undergraduate students and recommendations for future work
Smith et al. | Metzger et al. | This study | Recommendation | |
|---|---|---|---|---|
Sample characteristic | ||||
Sample size | 155 High school; 516 undergraduate | 105 Undergraduate | 770 Undergraduate | None |
Geographic region | Across U.S. | Midwestern U.S. | Northeastern U.S. | More studies in other geographic regions |
Demographics | – | 80% female; 37% URM | 57% female; 22% URM | More studies reporting and analyzing demographics |
Mean GAENE scores | By-item (5 max): 3.78 i.e. 3.02/4 | By-person (65 max): 51.7 [post] i.e. 41.36/52 | By-item (4 max): 3.25 [pre], 3.41 [post] By-person (52 max): 42.22 [pre], 44.3 [post] | Report by-item and by-person scores for ease of comparison across studies |
Analysis | ||||
Dimensionality (Internal structure validity) | One-dimensional | One-dimensional | One-Dimensional | None |
Item fit | Items 2–14 acceptable at single time-point | Items 2–14 acceptable in post-survey; Pre-survey not present | Items 2–14 acceptable in pre-survey; Item 13 poor fit in post-survey | Investigate pre- post- dynamics of item 13, esp. in course with NOS unit. Remove or modify item |
Rating scale functioning (RSF) of items | GAENE 2.1: Unclear how per-item RSF assessed. 7 and 13 had noise | GAENE 2.1: Per-item RSF not assessed | GAENE 1.0: RSF poor for items 7, 9, and 13 | Investigate RSF for each item. Remove or modify items 7, 9, and 13 |
Overall reliabilities | Acceptable Person: 0.91 Item: 0.99 | Acceptable Person: 0.93 Item: 0.86 | Acceptable Person: 0.86–0.88 Item: 0.89–0.9 | None |
Wright map | Large gap at high end | – | Large gap at high end | Add more difficult items |
Impact of instruction | – | – | Pre < post | More pre- post- data in large samples |
Impact of gender and race | – | URM < Non-URM F = M No effect size reported | URM < White Asian < White F < M η2G = 0.02–0.05 | More studies with gender and race, and effect sizes for each |
Impact of degree plan | – | – | Bio STEM = Non-Bio-STEM = Non-STEM | More studies of degree plan to corroborate findings |
Variance explained (R2) by all modeled variables | – | Post-course R2 ~ 11% | Pre-course R2 ~ 9% | More reporting of variance explained by student variables |
External structure validity | – | High correlation with both dimensions of the MATE | High correlation with MATE facts; moderate correlation with MATE credibility | Correlate GAENE with other acceptance instruments like the I-SEA and MATE |