Table 1 Steps followed to ensure validity
Steps ensuring the validity in the latent pattern content analysis* | Steps ensuring the validity in our research |
|---|---|
Develop a coding scheme that guides coders in the analysis of content. If the scheme is faithful to the theory in its orienting coders to the focal concepts, it is regarded as a valid coding scheme | Our coding scheme was pre-FACE which was developed based on the UECF. As described above UECF covers the major evolution ideas (see Appendix A) and has also been used by Asghar et al. (2015). Therefore, using this as a basis enhances the validity of our coding scheme |
Coders have to recognise patterns in the text | Coders had to recognise patterns in the curriculum = the presence of the concepts of the pre-FACE in the curriculum under examination |
Assess the decisions made by coders against some standard (norm). If the codes match the standard for correct decision making, then the coding is regarded as producing valid data. We look at the pattern of agreement that shows at least 80% of the coders making the same coding. This is a high degree of agreement, and this sets a fairly consistent norm. It means that in our analysis the codes were effective in assessing what it was intended to assess (validity) and this would be a widely held judgment (reliability) | Coders (experts with diverse profiles and expertise in the field of biology and education –some are experts in evolutionary biology, science education and science communication and some are elementary/secondary biology teachers or elementary school/biology teachers’ trainers), some working independently and some not, provided the coding. The independent coding of the data ensured that all meaning units would be identified and that none was left outside, that is, all learning goals referring to evolution are included. Codes provided by the coders were compared and the interraters’ (intercoders’) agreement assessed by using Krippendorff’s alpha coefficient (Krippendorff 2011). Acceptable results mean a widely held judgment: anyone who would read the same extract of the curriculum would be led to the same results regarding which evolution concept was covered |