Automated content analysis as a tool to compare content in sexual selection research with examples of sexual selection in evolutionary biology textbooks: implications for teaching the nature of science

Abstract

Background

 We used college-level evolution textbooks to examine the presentation of sexual selection research—a field with ongoing debates related to sex, sexuality and gender identity. Many classic sexual selection concepts have been criticized for androcentrism and other forms of gender-sex bias, specifically for de-emphasizing the female role in reproductive behaviors and over-reliance on gender-sex binaries. These classic concepts are fundamentally captured in the idea that animal reproductive-related behaviors can be grouped in sex roles (e.g. competitive males and selective females). Recently developed alternative concepts provide a more nuanced understanding of the flexibility of sexual and reproductive-related behaviors, stemming in part from growing attention to a broader range of female behavior. To assess whether students are receiving content reflecting these insights, we measured the congruence between textbook content and the scientific literature, using insects as a case study because of the importance of this group in the development of sexual selection theory, its prevalence in current sexual selection research, and the number of insect examples included in textbooks. We first coded textbook content for sexual selection concepts. We used automated content analysis to analyze a database of citations, keywords and abstracts in sexual selection research published between 1990 and 2014, inclusive of the period covered by the textbooks.

Results

The textbooks and research literatures prioritized the same taxa (e.g., fruit flies) and sex roles as embodied in classic sexual selection theory. Both the research literature and some textbooks acknowledge androcentrism and other forms of gender-sex bias in classic sexual selection paradigms, especially competitive male and selective female sex roles. Yet, while the research literature included alternative models, textbooks neglected these alternatives, even when researchers had studied both classic and alternative views in the same insect.

Conclusions

 We recommend using this kind of analysis of textbook content to engage students in a conversation around the social factors that impact knowledge construction, a key part of the epistemological understanding they need for a robust grasp of the Nature of Science and of evolutionary theory.

Background

Attempts to promote students’ understanding of the socio-scientific nature of knowledge construction, while maintaining their trust in the endeavor of science, are often framed within the context of the Nature of Science (NOS). Gender-sex and race are powerful societal, cultural, historical and biological phenomena. They are understood within complex knowledge frameworks that are challenging to capture in scientific knowledge systems. This is because those systems are often reliant on reductionist, binary-categorical, and essentialist models, which originated within racist, sexist and heteronormative frameworks (Longino 2013; Schiebinger 2004). To address this history, NOS integrates an understanding of how knowledge is shaped within simultaneously social (having to do with the interactions among scientists and within research communities) and rational (having to do with how scientists and research communities engage with their object of study) contexts. This social/rational context includes the scientific discipline and its theories and methodologies, as well as its members’ and research communities’ place within the larger society, and its attendant histories. This is manifest in the following three principles: (1) “Scientific knowledge is open to revision in light of new evidence (e.g. Scientific argumentation is a mode of logical discourse used to clarify the strength of relationships between ideas and evidence that may result in revision of an explanation);” (2) “Science is a way of knowing (e.g. Scientists’ backgrounds, theoretical commitments, and fields of endeavor influence the nature of their findings);” and (3) “Science is a human endeavor (e.g. Science knowledge has a history that includes the refinement of, and changes to, theories, ideas, and beliefs over time)” (National Research Council 2013). An understanding of NOS is a key ingredient in student acceptance of evolution. Specifically, students have higher acceptance of evolution when they appreciate the diversity of scientific methodologies and the nature of theory building and testing, even when controlling for interest and background in science (Lombrozo et al. 2008).

Understanding these NOS principles provides a foundation to challenge how science—combined with racism, sexism, heteronormativity and homophobia—maintains power differentials along presumed lines of difference. For example, eugenics—now deemed racist and sexist, among many other problems—was the mainstream and dominant research paradigm during the birth of modern evolutionary science. Scientists working within this framework were following scientific principles as they understood them, most often grounded in a positivist framework emphasizing reductionism and control. This served to maintain the status of the dominant groups, even though not all scientists at this time had this as their explicit goal (Gould 1996; Graves 2019; Subramaniam 2014).

Thus, teaching students about ongoing efforts to use evolutionary theory and other science either to justify or to challenge racial and other stratifications in society requires more than pointing out bias and misapplication of scientific methodologies– it must also incorporate how scientific knowledge production is intertwined with histories of racialized and gendered difference. This is especially important given the translation of scientific knowledge about human racial, sex and gender difference to the public, including biology students. For example, both interactionist and reductionist studies of hormones, sexuality and aggression have explanatory power and receive significant attention in the research community. Yet, the reductionist studies, implying that biology determines difference, have gained more coverage by the media, as well as some textbooks (Ray King et al. 2021). This supports an oversimplified societal narrative about hormones (biology) determining behavior that is not aligned with current scientific research (Longino 2013).

Although acknowledgement of the problematic history of evolutionary biology is becoming increasingly mainstream, strategies to move forward are lacking. In their absence, there has been increasing pushback and efforts to eliminate critical thinking about these issues, in large part either by banning the teaching of content that represents the current scientific consensus—especially in the area of gender-sex—or by curtailing critical frameworks that question systemic oppressions, eg critical race theory, gender studies and other critical frameworks (Rufo 2023a, 2023b; Wallis-Wells 2021). The pushback against critical analyses of racism and sexism rests (1) on shifting the focus to individual identity and (2) using presumed negative impacts on these individuals, especially those from socially dominant groups, to rally support for these bans (Rufo 2023c, 2023d; Wallis-Wells 2021). Thus, engaging in knowledge construction, or epistemological, frameworks that move beyond individual experience is critical.

Critical Contextual Empiricism (CCE) addresses this by framing knowledge as a communal rather than an individual pursuit (Longino 2002). Thus, NOS benefits from guideposts, like CCE, for navigating the social/rational processes that are included in the NOS principles, such as discourse, backgrounds, theoretical commitments, fields and histories. One CCE tenet is the argument that scientific research practices are strongest when scientific research communities are composed of more diverse groups—as long as those groups establish equitable frameworks to share and critique knowledge (Longino 2002). Underlying this approach is the understanding that rather than being about the identity of the individuals doing science, what is significant is their positionality, i.e. where those individuals reside in a complex matrix of identity categories and whether those with membership in these identity categories can access the power of knowledge production (Collins 2019). The objectivity associated with science has been privileged and historically assigned to those whose identities claim the most social, economic, and political power, leading to research outcomes supporting this division of power (Haraway 1988; Harding 1986).

CCE, coupled with the NOS principles, makes visible for students the ways in which knowledge is constructed by providing concrete examples of how scientific knowledge responds to critique. One way to capture this is to consider textbooks as a site of knowledge production, given that (1) the success of textbooks rests in their adoption by the community, and (2) they play a key role in introducing new members of the community to disciplinary norms (Bazzul 2014). Here we present a case study on sexual selection research on insects, which investigates how textbook content aligns with changes in research related to gender-sex, an area with changing paradigms drawn in part from larger societal and scientific discourses.

Textbooks as the Site of NOS engagement

Biology, as a research field, has begun addressing racism, sexism and heteronormativity in two ways—by attending to plasticity, variation and context when studying organisms and by acknowledging the socially constructed nature of race, gender-sex and sexuality as knowledge systems (Ah-King 2022; Eliot 2010; Fausto-Sterling 2012; Hyde et al. 2019; Lett et al. 2022; Montañez 2017; Roughgarden 2013; Zambrana and Williams 2022). Researchers have also begun to scrutinize how science textbooks address and can impact social issues related to race, gender-sex, and sexuality and gender identity (Vojíř and Rusek 2019).

Unfortunately, many changes in research paradigms to address racism, sexism and heteronormativity are not being transferred to the textbooks, where, outside of brief acknowledgements of past problems, textbooks often follow a strategy of avoidance (Bazzul and Sykes 2011; Bickford 2022; Donovan 2015). Although most include disclaimers about biology being destiny and allude to the fact that science does not provide a framework for ethical decision-making (a part of NOS), textbooks largely fail to present information to help students robustly think about race and gender-sex from a biological perspective. For example, content analyses focused on gender and sexuality found that scientific textbooks contained heteronormative assumptions (Ah-King 2013b; Bazzul and Sykes 2011; Bickford 2022; Røthing 2017), gender-biased language and assumptions (Ah-King 2013b), and gender-biased or sexist imagery (Elgar 2004; Good and Woodzicka 2010; Parker et al. 2017; Rosa and Gomes da Silva 2020; Spaulding and Fuselier 2023; Fuselier et al. 2018). In the case of race, although books are careful to challenge the idea that race is a biological construct and include evolutionary information to the contrary, they fail to challenge racism, often supported by pseudoscience, directly (Bickford 2022; Donovan 2015). For example, through a content analysis of 153 biology books (86 textbooks, 44 curricular supplements, and 23 trade books), Bickford (2022) found that although these books covered evolutionary content accurately, they did not present scientific evidence that would refute white supremacy or cis-heteronormativity. For example, Bickford (2022) found that the books often presented the lack of the validity of race as a biological construct but failed to attend to its significant role as a societal construct or to the use of science to justify racialized oppression (eg. eugenics). Overall, students lack exposure to the historical debates within biology that have led to changes in how researchers conceive of race, sex and gender as constructs in their work (reviewed in Donovan and Nehm 2020).

This selective or missing coverage can lead to an increase in student assumptions around biological essentialism associated with race, gender, sexuality, and gender identity. A failure to challenge social constructs of race, often grounded in pseudoscience, leads to increases in racism—even when students are then provided information intended to interrogate racialized disparities. Several studies have suggested that when biology textbooks give examples of outcomes such as diseases that are more common in one race than in others—as an attempt to address health disparities—students may develop or strengthen a belief in racial essentialism and extrapolate into other areas with racial disparities, including educational attainment (reviewed in Donovan 2015). To address this, Willinsky (2020) provides an overview of mixed messages about race—critiquing the falsity of race as a biological variable, while separately presenting content that uses racial groupings as a variable—in high school biology textbooks that, he argues, also reflects the how race as a concept appears in current research on race within biology. He argues that educators should integrate a historical understanding of biology’s contributions to racialized research, especially eugenics, and use the contradictory messaging present within textbooks to demonstrate the complexities of conducting research on systemic racism and racialized outcomes in health and other biological fields (Willinsky 2020).

Similar findings hold for beliefs about gender and sex difference. Donovan et al. (2019a) investigated the impact when 8th-10th grade students read selections from biology textbooks on the students’ belief in a neurogenetic basis for sex differences in humans and interest in science. They compared a passage refuting neurogenetic sex differences with two passages endorsing neurogenetic differences—one in humans and one in plants. Students self-identifying as girls who read the endorsing passages, whether in plants or humans, were more likely to believe in sex differences grounded in neurogenetics; girls in these treatments also indicated less interest in science. A further examination of student writing after reading the passages indicated that students tended to use both sex and gender language in all treatments, with some evidence that they were distinguishing between the concepts to refute essentialism in the refutational text treatment (Stuhlsatz et al. 2020). Recognizing that biology textbooks also conflate biological sex differences with gendered social outcomes, the authors recommend an approach emphasizing the complex histories of science research on both sex and gender, accompanied by training for teachers on how to address this content with their students. Our study aims to provide such a resource in the case of sexual selection.

Sexual selection and changing paradigms

One area in which scientific research and other scholarly work have begun to address at least some gender-biased assumptions is sexual selection research (Ah-King 2022; Ah-King and Ahnesjö, 2013). In our previous work, we found that although some evolutionary biology textbooks acknowledge the critique of gender bias in scientific research, their presentation of sexual selection research in text, and especially in images, retains an emphasis on the work that has been critiqued for said gender-sex bias (Fuselier et al. 2016, 2018). This also occurs in animal behavior textbooks, which devote more space to sexual selection (Spaulding and Fuselier 2023).

Although sexual selection is typically covered in evolution courses, little research has been done to ascertain how it is taught and how students understand it (Ziadie and Andrews 2018). Sexual selection research originated as a study of extreme differences between males and females, e.g. strong sexual size dimorphisms or other traits that occur or are highly exaggerated in only one sex, such as the classic example of the peacock’s ornamental tail. The classic view of sexual selection emphasizes stable binary sex roles with males competing, either by fighting with other males or by displaying to females who may choose the males as mates based on their displays or dominance over other males. The roles may be reversed, with female competition for mates, given changes in the environment, such as restricted nesting sites, resulting in more female animals ready to mate than have access to resources needed for mating —but this phenomena was seen to support the existence of the binary itself (Ah-King and Ahnesjö, 2013; Trivers 1976).

Feminist critiques of androcentric bias in sexual selection theory began soon after its publication (Blackwell 1875; Hamlin 2015), and work critiquing androcentric bias and offering solutions has been ongoing in the field ever since (reviewed in Jackson 2001a, b, 2014). After the 2000’s, the frequency of such research in mainstream animal behavior and evolution journals has increased (reviewed in Fuselier et al. 2016). The field has been critiqued most often for importing assumptions about human sex roles into the study of non-human organisms (e.g. Hrdy 1986). Additional ongoing areas of concern include acknowledging the context-specific nature of sexual behavior and mating patterns (Gowaty 2013; Kokko and Johnstone 2002), though the extent of the challenge to traditional notions of sexual selection is a subject for debate (see for example the exchange between Ah-King 2013a; Kokko et al. 2013). Researchers in sexual selection have acknowledged the lack of studies of female organisms (Clutton-Brock 2009) and have highlighted not only sexual selection on females but also several alternative behaviors that expand the classic understanding of sexual selection, such as male mate choice, female ornaments, male parental care, female-female competition and flexible sex roles (reviewed in Fuselier et al. 2016).

College-level evolutionary biology textbooks present primarily classic sexual selection binary sex-role theory, although some textbooks do present some examples of alternatives to classic roles, most commonly extra-pair copulations and polyandry—situations in which female animals mate with multiple males (Fuselier et al. 2016). Yet, the images included in the textbooks display a more conservative representation of classically understood sex roles than the content covered in the writing (Fuselier et al. 2018). It is unclear how the content presented in textbooks reflects the scientific literature. One challenge to research in this area is the difficulty of synthesizing the vast amounts of information available in the literature for comparison with the textbooks, a necessity for making recommendations for how to modify content or examining how the instructor frames what the books do—or, more importantly, fail to do. Here we explore the efficacy of automated content analysis (ACA) as a tool to assess the alignment of textbook content with the scientific literature.

Automated content analysis (ACA) essentially turns text into data, using sets of algorithms to construct models that allow researchers to determine the concepts on which authors focus, as well as the relationships among those concepts. ACA has been used recently to assess and identify trends and shifts in ecology and evolutionary biology (Nunez-Mir et al 2016; McCallen et al. 2019). Essentially, ACA programs based on machine-learning (ML) identify words or word combinations that are commonly associated with one another in text by determining how frequently they co-occur in small blocks of text (3–4 lines) versus how frequently they occur elsewhere. Leximancer does not use a training set like other artificial intelligence programs might; more information about algorithms used in the program is reviewed in Smith and Humphreys (2006). Through machine learning, ACA identifies and quantifies the associations of terms to develop a thesaurus and create “concepts” and groups of concepts related to the same theme. The frequency of and relationships among concepts and themes can be calculated, assessed, and visualized. The power of this type of analysis is the large amount of literature (or text) that can be assessed in a relatively short time. ACA is thus an excellent tool for comparing the content of textbooks to the topics emphasized within the literature on a given subject. It can reveal how researchers address particular topics both currently and over time, as well as gaps or lags in textbooks’ coverage of a field.

We used insects as a proof of concept for the ML-based ACA technique because our prior research demonstrated that a wider range of sexual selection roles was presented in this taxon than in any other group used in the textbooks (Fuselier et al. 2016). After completing analysis of the peer-reviewed articles, we then compared all the concepts that were studied in insects to the concepts that textbooks used these insects to exemplify. We also examined whether the insects used to represent specific behaviors in textbooks reflected the insect taxa in which these behaviors were most studied in the peer-reviewed articles. We addressed the following specific research questions:

• 1) What sexual selection behaviors are studied in insect taxa in peer-reviewed literature?

• 2) Do the insect taxa described in textbook discussions of sexual selection match the insect taxa studied in peer-reviewed articles in the sexual selection literature?

• 3) How does the range of sexual selection behaviors covered in textbooks compare to the range of behaviors discussed in peer-reviewed articles?

Methods

Textbooks

We used four recent evolutionary biology textbooks (Table 1) published between 2012 and 2013 that in 2016 represented over 95% of the market share of college-level evolution textbooks in the United States. The textbooks were the same used in our prior research (Fuselier et al. 2016, 2018).

Table 1 Textbook publishers, editions and years

We created an inventory of all insects used as examples in textbook sections devoted to sexual selection topics. The examples were classified as fitting into one of two understandings of sexual selection: classic (e.g., male-male competition, female choice) or expanded (e.g., competition among females, reproductive constraints among males, or mate choice as a mutual process).

Literature search and dataset

To construct a literature database, we used the Zoological Record collection within Web of Science (Clarivate Analytics) to identify proceedings, peer-reviewed journal articles, books and book chapters focused on sexual selection in insects. We focused on the Zoological Record because this database is the oldest database focused on animal science and is known for its focus on zoology and animal biology. It covers international journals on behavior, with an emphasis on knowledge pertinent to the study of non-human animals in the wild; it thus contains the literature most relevant to our study (Zoological Record​ on Web of Science 2024). Its organization by taxonomy also mirrors our study’s emphasis on taxonomic differences, and thus its structure was particularly amenable to the ways that we needed to sort the literature to answer our research questions. We limited our search to the years 1990–2014, dates for which we were able to access abstracts for the papers. This period marks a significant time frame for a renewal of interest in sexual selection, and an associated feminist critique of androcentric bias. Given that the latest publication date of our selected books was 2013, it also included the literature most likely to be covered in the books and thus ensured that the records were those most pertinent to our research questions.

We constructed our search using Boolean operators, identifying papers with topics including both ‘sexual selection’ and ‘insect’ or its variants (e.g., insects, Insecta). After reviewing the literature, we realized that this search also included research in which the insect was not the focus of the study, e.g., studies on sexual selection in flowers mediated by insect pollinators, and studies of the impacts of sexual selection on bird traits in which the traits were signaling resistance to an insect parasite. To remove these studies, we added a supertaxon search term to search separately for papers in which the supertaxon was or was not Insecta. Most studies identified by this revised search were those with the supertaxon Insecta, and all of these (n = 1581) focused on sexual selection in insects. In a smaller set of studies (n = 105), the supertaxon was not Insecta. We reviewed these manually and removed 52 publications that did not focus on sexual selection in insects. The remaining 53 papers, which did cover sexual selection in insects, were often reviews or comparative studies in which sexual selection in an insect was being compared to sexual selection in another taxon, e.g., studies comparing nuptial gifts in spiders (Arachnida) versus crickets (Insecta). These papers were included in our final dataset of 1634 papers.

We then imported the full records (including full citations, abstracts, automatic tags, and other metadata) into a database. We manually reviewed the 1634 records to sort them into our final taxonomic groupings. This resulted in nine groups, which included seven insect orders, the genus Drosophila (fruit flies), and an ‘other’ group that included all taxa that were the focus of fewer than 20 studies each. We separated Drosophila from its parent taxon Diptera (flies) because of the large number of studies on Drosophila; there were more studies on Drosophila than on any other group (Table 2). We then exported these to Microsoft Excel^© for automated content analysis.

Table 2 Number of papers for each insect taxon

Automated content analysis

We analyzed spreadsheets containing article titles, abstracts and manual search terms for the nine groups of insect taxa using Leximancer, a machine-learning-based program for automated content analysis (Leximancer 2019). To identify the most commonly studied topics in sexual selection in insects among the 1634 papers, we used an “overall” analysis of concepts in which we allowed the program to find concepts and build a thesaurus from automatically generated terms. For a second, “profiled” analysis we added “user-defined concepts” specifically related to alternatives to classic sex roles such as polyandry, mutual mate choice, alliances, etc. To verify that user-defined concepts aligned with the meaning in the text, an investigator checked the meaning in the text with the excerpts identified by the program. For example, using the compound term “female + competition” when searching for papers that addressed competition among females for mates, text excerpts that contained the two words in a sentence but did not refer to female competition were excluded (e.g., “competition experiments…showed males mated with more females”). We modified the compound concepts (e.g., “female + competition + NOT male”) and re-ran analyses until we minimized the occurrence of inaccurate matches with the text. We used measures (produced by Leximancer^®) of the frequency and strength of association to identify what topics were most commonly studied among which taxa; we used prominence values to quantify the relationship between taxa and topic. Prominence is a combination of strength and frequency within a taxon, and prominence values > 1 indicate that the association happens more often than expected by chance.

Results

• 1) What sexual selection behaviors are studied in insect taxa in peer-reviewed literature?

Overall analysis

The overall analysis identified 64 commonly occurring concepts (see Table 3 and Appendix A). The concept ‘male’ was the most commonly encountered concept in the dataset, and thus was more common than ‘female.’ Examination of the concepts most frequently co-occurring with the five top concepts revealed that research on sexual selection in insects has emphasized males over females and focused on post-copulatory selection, communication (e.g., calling), and biometrics, among other topics. All taxa had a high frequency of association with the concepts, meaning that given the taxon, we were highly likely to find papers that included the concept. But, given the concept, the strength of association with a particular taxon was low, indicating that all the commonly encountered concepts were studied in all taxa. Interestingly, the concept ‘female’ occurred most often in association with the concept ‘re-mating’ and, secondly, ‘choice.’ Re-mating was used in studies of conflict, which was one of the top associations with the term ‘sexual,’ indicating that there is a wealth of literature on sexual conflict and that it includes an examination of females re-mating, which is one of the expanded views of sexual selection because it emphasizes multiple mating by females.

Table 3 The top 10 most commonly occurring terms, and terms with which they co-occurred, in an overall analysis of literature about sexual selection in insects

Profiled analysis

In this analysis we removed very general concepts, (e.g., male, female, sexual, evolution, behavior, reproduction, and variation) that were studied in all taxa and included 16 user-defined concepts that emphasized alternatives to classic sexual selection. Removing general concepts provided the opportunity to examine more closely which insects were used to study expanded sexual selection. For example, the sheer number of studies on speciation in fruit flies impeded the program’s ability to detect associations of fruit flies with non-traditional concepts (e.g., condition-dependent mate choice).

Four of the nine taxonomic groups were strongly and frequently associated with particular expanded concepts (Table 4): beetles, fruit flies, butterflies/moths, and flies. Beetles and fruit flies were frequently associated with concepts related to sperm competition and conflict (sperm competition, male costs, sperm storage, conflict, polyandry, and multiple female mating). Fruit flies, beetles and crickets were associated with condition-dependent mate choice, male mate choice and female aggression. Finally, butterflies/moths were associated with female signals, mainly pheromones, and flies were associated with conflict.

Table 4 Prominence of association of expanded sexual selection concepts with taxa

Overall, expanded concepts were studied in many insect taxa, and all expanded concepts appeared prominently in two or more taxonomic groups. On average, for each concept (e.g., “female ornaments”) there were three taxa with significant prominence values. The most infrequently studied expanded concept was female reproductive success, which was only prominently associated with beetles and butterflies/moths. Beetles and fruit flies were central to the studies of expanded concepts of sexual selection. Although studies using fruit flies made up the largest proportion of papers we identified for our dataset, more expanded concepts (n = 10) were significantly prominent in beetles than in fruit flies (n = 8).

Comparison to textbooks

• 2) Do the insect taxa described in textbook discussions of sexual selection match the insect taxa studied in peer-reviewed articles in the sexual selection literature?

Overall, fruit flies, beetles and crickets/grasshoppers were the most commonly studied groups in the scientific literature (Table 5). All flies (Diptera) including fruit flies accounted for 31% of the experimental science studies. This matches well with the proportions of examples used across all textbooks combined for flies, which was also 31%. However, when we looked at individual textbooks, the proportion of examples that used fruit flies or flies ranged from 16 to 50%, with one textbook (Pearson, 33%) matching the distribution of taxa in the literature but the others with far greater or lower representation than expected based on the literature.

• 3) How does the range of sexual selection behaviors covered in textbooks compare to the range of behaviors discussed in peer-reviewed articles?

Table 5 The proportion of insect examples for each taxon that appear in the textbooks compared to the proportion of the literature on that taxonomic group

The profiled analysis showed that most of the alternatives to traditional sex roles were covered in two taxonomic groups—fruit flies and beetles. Therefore, if the textbooks are covering these alternatives, we would expect to see at least one of these taxa discussed in all textbooks. At least one of the two taxa did appear in all books: fruit flies appeared in all four textbooks, and beetles appeared in three of the four. However, we found that although beetles, fruit flies and flies were strongly and frequently associated with expanded examples in the literature, they were used primarily for classic examples in the textbooks. In the literature, butterflies and moths exemplified expanded sexual selection, specifically focused on female chemical signals; the books did not attend to these taxa or this topic. What did textbooks use fruit flies and beetles to exemplify? Fruit flies exemplified both classic concepts and one expanded sexual selection concept (sexual conflict) in all books. However, beetles were used only to exemplify classic sexual selection. Thus, although studies of expanded concepts in beetles are available in the literature, they are not typically used to exemplify these concepts in the textbooks.

A similar mismatch is found among the grasshoppers/crickets. Grasshoppers/crickets were often used to study expanded concepts in the literature and also occurred in all four textbooks (Tables 4, 5). However, the textbooks used them to exemplify mainly classic sex roles. Female-female interactions, signals, and aggression were prominent concepts among grasshoppers and crickets in the literature. Yet in textbooks, the expanded roles received only brief coverage—one, scent marking of males by females, was only listed in a table rather than as a detailed example in the text of the chapter. Another text used a cricket as an example of a flexible sex role, but this appeared only in the end-of-chapter questions.

Discussion

The significance of our findings, in comparison to most current literature on textbooks, is that we have examined how textbooks track trends in the sexual selection research literature, responding to critiques of gendered and androcentric bias dating back to Darwin’s original writings about sexual selection (Hamlin 2015; Jackson 2001a, b, 2014). Although we previously found that some textbooks acknowledge the importance of the critique of gendered and androcentric bias in their discussion of sexual selection research (Fuselier et al. 2016), their selected images reinforce a traditional view of classic sexual selection theory (Fuselier et al. 2018). In this study we find that they also do not engage with its implications when they present the content of sexual selection to their student audience.

Our work concerns the decision-making processes that affect the presentation of knowledge, using the textbooks as a case study and CCE as a framework. Key to this approach is our main finding that in general, the textbooks do not provide a thorough representation of how research in the field of evolution, specifically in sexual selection, has shifted. Our analysis of 1634 unique research papers on sexual selection in insect taxa revealed that although most studies produced work that aligned with the classic paradigm, there were many examples that expanded upon this paradigm; polyandry and other concepts related to female multiple matings were common, as was male mate choice. Additionally, relative to the textbooks, the peer-reviewed research literature reported a greater number of alternatives to classic sex roles occurring in more and different taxa.

Several insect taxa that were included in the textbooks have been used to study alternative concepts; however, instead of reflecting this diversity, the textbooks used those taxa to illustrate classic concepts of sexual selection and excluded the expanded concepts. Thus, we see more attention being paid to alternatives to classic concepts in research articles than in textbooks. One reason for this discrepancy might be due to the taxa that are used to exemplify the concepts. We found some support for this idea in that some taxa in which the alternatives were most frequently studied were not included in textbooks. But this is not the full story because even when textbook authors included taxa that were most strongly associated with alternative concepts, they still focused on the classic concepts instead of addressing the alternatives. This indicates that textbooks maintain a bias toward classic concepts over those that expand the understanding of sexual selection beyond stereotypical sex roles. For example, in the research literature on insect sexual selection, female remating is a common concept, and ‘remating’ has an association with ‘female’ that is even stronger than the association of ‘female’ with ‘choice.’ However, well-studied charismatic insects that would illustrate the benefits of mating multiply for females are not included in textbooks. One example is the honeybee (Apis mellifera), a species in which a queen mates with twelve males on average (Tarpy et al. 2004); experimental data showed that queens with more than one mate are more attractive to workers, which may give queens longer tenure and thus higher success (Richard et al. 2007).

This is significant in the context of research indicating that reading passages in textbooks that reinforce biological bases of difference, whether about humans or not, can lead to more student endorsement of a biological basis behind racial and gendered stratification in society (Donovan et al. 2019b; Stuhlsatz et al. 2020). Thus, there is a critical need to expose students to the kinds of examples about variation in sexual behavior that we found in our review of the research literature on insects in sexual selection, whether through examples provided in the textbooks or in supplementary material to the textbook provided by the instructors. The provision of supplementary materials also offers the chance to engage directly with NOS principles, using the textbooks themselves as the place where scientific knowledge is being constructed. Our work is significant because our case study provides an example instructors can use to address this gap within the framework provided by CCE.

Recommendations for evolution education

Our recommendations align with those made by (Willinsky 2020). He found mixed messages both challenging and supporting genetic essentialism in a review of textbook content related to genetics and race. As a teaching strategy, he suggests that instructors directly discuss the variation in how textbooks discuss race and genetics, using this to exemplify the complexity of studying racialized biological outcomes within the historical racist context of science. We concur with his suggestion and position our work as a method to allow instructors to engage more critically with textbook content by exploring with students the social/rational process of scientific work—which necessitates a deeper dive into the formation of the research literature than is present in many textbook summations of scientific content. Our study provides strategies to strengthen the epistemological understandings that students need to ground a robust conception of NOS, by considering the communal, rather than individual, nature of knowledge construction (CCE) in the area of sex and gender difference—an area in which students, indeed all of us, are being bombarded with controversial information.

Students with a more robust understanding of the NOS, especially around the complexities of theory building and testing, understand that knowledge production involves gray areas of nuance and context (Cho et al. 2011). To use our work to encourage students to do this, an instructor could ask students to reflect on their views of textbooks. Rather than seeing them as all-knowing repositories that cannot be questioned, such a conversation would encourage what Bazzul (2014) describes as a reflexive process whereby students engage in ownership of the content of their fields by questioning and considering the nuances of information received. The point of this exercise is not to reinforce a simplistic understanding of the history of racism and sexism in science as a case of bias now corrected, but to have the students use the textbook as a place to think about how information is selected and shaped.

In this instance, our study would provide a strategy to consider knowledge production at the level of the community, with the community at play being the group of evolutionary biology texts, rather than any one individual book. The textbooks that we examined collectively provided coverage of insects that was more representative of the scientific literature than any individual book did. Although there was no single book whose examples of evolution in insects matched the diversity of insect taxa found in the literature, when the books were combined, their coverage came much closer to that diversity.

The use of multiple texts and resources (instead of reliance on one textbook as an authoritative source) has been used in several fields to improve students’ understanding. For example, in history, multiple texts have been used to guide college students to understand the importance of the availability of source material, which can for example be used to indicate which groups have been deemed worth preserving in the historical record and the accompanying writing of history; however, the researchers note that students require training to understand this, given that high school classes present history as a collection of facts to be memorized (Hynd 1999). In political science, researchers have identified a hidden curriculum within introductory textbooks that centers institutions and those who have the most power within them (mostly white men), and de-emphasizes or ignores the political contributions of those who have had to fight for equity by segregating coverage of movements for gender, sexual, and racial/ethnic equity into sections linked only to diversity and thus reinforcing the notion that those issues are outside the mainstream (Atchison 2017; Cassese and Bos 2013); the use of original source material and/or diverse sources from the field’s research literature could ameliorate this bias. Within mathematics, there has been a shift in the conception of how teachers use textbooks, with a new emphasis on teachers’ pedagogical design capacity or the ability of teachers to make decisions about how to use, adapt or add to content provided in textbooks grounded in their understanding of how to help their students learn (Matić, 2019). Overall, across a broad range of fields, there is growing recognition that students do not simply receive knowledge from textbooks, teachers or any other source; rather students integrate what they learn with their own frameworks, prior knowledge and goals. Projects that expose for students how textbook authors make choices in their presentation of topics thus offer a way to engage with student sense-making processes and enhance learning (Sikorski and Hammer 2017). Comparison across textbooks—making visible their differences as well and what they share—provides a strategy to address this.

Our finding that collectively the books did a better job than any one book in coverage of the field is key here. Instructors could share with their students how their specific class textbook covers topics in contrast to other books. This could lead to conversations about the selection of what to include and not to include and what mediates those decisions, including the authors’ positionalities—not just their identity put a multitude of associated factors based on how they move through the social world—of those doing the research or writing the books—an issue identified as critical to the construction of science by feminist scholars (reviewed in Intemann 2010).

Key to this conversation would be including how some of the textbooks’ authors offer overviews of the critique of androcentrism in their fields, framed by noting how those historically excluded from the research community—in the case of gender bias, normally women—corrected this bias by attending to the behavior of female animals (Fuselier et al. 2016). Although the discussion indicates the authors saw the value of the critique, it fails to account for the continuing emphasis on classic sexual selection theory, with its androcentric focus and gender binaries. This parallels the split presentation that other researchers found within textbooks—with mixed messaging about race, sex and gender—deconstructing bias in one place, while sharing examples that reinforce it in another passage (Bickford 2022; Donovan 2015; Willinsky 2020). A CCE framework opens the door for a nuanced conversation with students for the reasons behind this finding.

Bringing attention to the increased attention to female behaviors in the context of a discussion of historical and contemporary critiques of sexual selection models for androcentrism would provide a concrete example of the NOS principle that “Scientific knowledge is open to revision in light of new evidence.” This could be accomplished in part by making small shifts in the framing of some concepts and by augmenting textbook examples with examples from different taxa, such as more coleopterans, to represent a wider variety of concepts. In the research literature, Coleoptera and Drosophila were closely associated with concepts related to sperm competition and conflict (sperm competition, male costs, sperm storage, conflict, polyandry, multiple female mating), which require multiple matings among females. Reframing the presentation of sperm competition in textbooks to emphasize multiple mating by females—and the often-positive fitness consequences for females of multiple matings –would put textbooks in closer alignment with the research in this field. Having open discussions with students on the implications of centering sperm competition versus multiple mating or remating by females offers a chance to engage with the NOS principle that ‘Science is a way of knowing’ by having a discussion about the impacts of language choice on who is perceived as having or lacking agency in scientific research.

Further, some taxa used to exemplify classic sex roles, could also be used to show alternatives. A good example would be an orthopteran such as a katydid species that has flexible, condition-dependent sex roles. Although crickets, which are also orthopterans, were used in all textbooks, they were leveraged primarily to support classic sex roles. Again, a small change—adopting examples of orthopteran flexible sex roles in the main body of the chapter—would better align the books with the experimental science. In fact, research on multiple mating by females in orthopterans began in the nineties (Tregenza and Wedell 1998). In addition, a class discussion about the reasons why textbooks continue to center classic sex roles could engage students with the NOS principle that “Science is a human endeavor” and is thus subject to the decisions made by humans in terms of what to emphasize, de-emphasize or not to discuss.

Using ACA to track the progress of fields and how they are synthesized in textbooks

For researchers interested in extending this approach to other topics within and beyond sexual selection, we found that ACA is a promising tool for exploring how textbooks reflect the research being done in a particular field, especially which the field is undergoing change in how it approaches key concepts. Our work builds on prior attempts to assess textbook quality by comparing textbooks to the coverage of disciplinary research. For example, Bierema et al. (2017) used a combination of manual and automated content analysis to identify main topics covered in animal behavior textbooks. For automated analysis, these authors used a program that found terms in text. The difference between this and Leximancer is that Leximancer “learns” from the text and creates a thesaurus of related terms for a particular code. The investigator can then cull the inappropriate terms and ultimately “train” the program to match content with context. This is instructive because it permits researchers to see the relationships among terms and the “composition” of those terms, and then use measures of conditional probability and network analysis to quantify and visualize relationships.

The analysis by Bierema et al. (2017) determined the proportion of research articles’ abstracts that included four different central ideas in the field of animal behavior. That study used the frequency of occurrence of central ideas in this selection of journal articles, and then compared this to journal impact factor to estimate impact in the field. When they compared these results to textbooks, they found that the textbooks overall matched the literature from 28 journals in that there were similar patterns of proportions across the main topics covered. Using ACA allowed us to conduct a more detailed analysis that provided insights into the relationships among concepts. Also, our research question about taxa was specific enough that we could limit the dataset by taxa rather than by journal; this permitted a broader survey of many journals as opposed to choosing only a selection based on readership or other metrics. Instead of assessing which broad disciplinary topics are covered, we emphasize a focal area within evolutionary biology: sexual selection and the evolution of sex roles and reproductive behavior. This level of detail and nuance was significant for our topic because of our focus on a topic that arose from a critique of mainstream research. Further studies which were outside of the classic view of sexual selection appeared in taxon specific or subfield oriented journals decades before studies were published in mainstream journals (Jackson 2001a, b, 2014). Thus, our approach to using ACA is, therefore, appropriate when looking for emergent trends that may counter dominant narratives.

A cautionary note

There are important cautions to bear in mind for those wishing to apply the method of auto-content analysis. One of the biggest challenges is the optimization of search terms to ensure an accurate match between the concept-of-interest and the context in which it is used in the publication. For example, in this study, "multiple female matings" was used more often than “polyandry,” and thus the two terms had to be linked in the thesaurus we created. But then sentences containing the words “multiple,” “female,” and “mating” were considered to be “hits” even when the context of the sentence was not about polyandry (e.g., “…males mating with multiple females…”). Thus, validation, i.e. assessment to determine whether the program is correctly linking the concept to its appropriate context, is critical for an accurate analysis. Human knowledge is required for validation. In our case, the researchers have doctoral degrees in evolutionary biology, animal behavior and gender studies—a diverse group with deep knowledge of the scientific content, including its relation to social movements for gender equality. Additionally, we paid careful attention to the construction of the database, focusing on a collection of papers with a taxonomic focus and manually verifying that the included papers matched our criteria. The technique should be used in conjunction with other methodologies, including thematic coding of text and image analysis, as we have done in other publications (Fuselier et al. 2016; 2018).

Conclusions

We advocate for the textbooks in a novel way to integrate students understanding of NOS within the context of their study of content. Rather than presenting the textbook as an authoritative source of information, we suggest guiding students through a process of comparing it with the relevant research literature to understand decision making about what aspects of evolution are presented as ‘fact’ to students. This engages students with several tasks shown to be beneficial to the understanding of evolution—metacognitive vigilance (González Galli et al. 2020), appreciation of the Nature of Science, especially the tentative and provisional nature of science and the importance of multiple theories, understanding of epistemological beliefs–specifically that learning is changeable, not innate, and knowledge does not come from all-knowing sources– which provide the foundation for a robust understanding of both NOS and evolution (Cho et al. 2011).

Appendix A

See Table 6

Table 6 Continuation of Table 3 showing the remaining 11–64 most commonly encountered terms in an overall analysis of sexual selection in insects.