- Education Article
- Open Access
What’s the Matter with Kansas?: The Development and Confirmation of the Evolutionary Attitudes and Literacy Survey (EALS)
Evolution: Education and Outreach volume 4, pages117–132(2011)
The present survey was designed to assess predominant regional belief systems and the roles these beliefs play in science understanding and attitudes, and curricular effectiveness in colleges and universities. To this end, we created a wide variety of theory-driven subscales (lower order factors) reflecting, for example, exposure to evolutionary material, young earth creationist beliefs, moral and social objections, political ideology, endorsement of intelligent design fallacies, knowledge (and distrust) of the scientific enterprise, and attitudes of evolutionary theory’s relevance in several domains (e.g., sciences and humanities). We also included potentially important demographic variables (e.g., rural upbringing, family size). Finally, we assessed openness to experience, a key facet of personality. Hierarchical Structural Equation Modeling (SEM) analysis showed the 16 constructs to have a theoretically meaningful and quantitatively coherent higher order factor structure. In this large Kansas sample of university students, creationist reasoning and conservative orientation (political and religious) were negatively associated with exposure to evolutionary theory, knowledge about it, and positive attitudes toward its relevance. At the same time, exposure to the theory was positively associated with knowledge and positive attitudes. Importantly, though most Kansas-specific demographic variables (e.g., rural origins) were largely unrelated to outcomes of interest in this university-based sample, the personality factor openness to experience appears to be highly relevant for several higher order factors (e.g., exposure, knowledge and relevance, and creationist reasoning). We close with implications for educators and the next steps in survey development.
Evolution is a theory. God is the true creator and He created animals to change w/their environment.
Thank you for trying to teach what they’re not in Kansas public schools. I was educated mostly by Kansas public schools. Even in high school, we barely touched evolution, and my biology teacher was too afraid to talk about human evolution! No wonder my relatives outside of the state don’t take Kansas seriously. Footnote 1
The state of Kansas has had lengthy and very public problems with the theory of evolution, the lion’s share of which were centered on high school science curriculum standards and culminated in the “Kansas Evolution Hearings” in May of 2005, an event widely held by the scientific community to be a “kangaroo court.” Rather than staying local, the science debates in Kansas have been noted in international news outlets and have led the state to be the brunt of jokes on the web (The Onion). Television (Jon Stewart’s The Daily Show), and print magazines (The New Yorker).
What’s the matter with Kansas?Footnote 2
In fact, the problem is not limited to Kansas. Anti-evolution sentiment permeates not only public perception but also science education nationwide. The United States, however, may be unique in its politicization of evolutionary theory (Miller et al. 2006). During recent presidential administrations, the Republican Party explicitly advocated teaching “creation science” as part of high school curricula (and the Discovery Institute was founded to this end). In 1985, 45% of the nation’s population accepted that humans had evolved, with 7% being unsure. By 2005, only 40% expressed acceptance of human evolution (Miller et al. 2006; see also Pew 2009; Gallup Poll 2009), while 21% were unsure (31% of the population believes that humans have always existed in their present form). In a widely distributed, multinational survey, only Turkey scored lower on acceptance of evolution than the United States (Miller et al. 2006). In terms of widely held stereotypes, pro-evolutionists tend to characterize evolution objectors as backwards, biblical literalists who fear the shaking of their worldview that promises redemption and salvation. Anti-evolutionists portray pro-evolutionists as immoral, Godless materialists who undermine the fabric of society.
Is there any truth to these stereotypes and, if there is, what are the implications for science education, especially as it pertains to teaching the theory of evolution? To explore these and other related questions, we created a multi-item survey to assess not only political and spiritual leanings but also knowledge of evolution, distrust of the scientific enterprise, knowledge about the scientific enterprise, and attitudes toward and objections against evolutionary theory. Our goal ultimately is to create a standard tool to assess the curricular effectiveness of courses on evolution and/or biology in colleges and universities (e.g., O’Brien et al. 2009) and to assess the effectiveness of the ID propaganda machine on a regional basis by way of assessing the endorsement of intelligent design fallacies.
The Present Study
The present survey represents an important step in the process of designing a useable tool for university and college educators to assess (a) predominant regional belief systems and their roles in science understanding and attitudes, (b) curricular effectiveness and attainment of specified learning goals, and (c) changes in attitudes about course material. Though several interesting measurement tools exist (e.g., Rutledge and Warden 1999; Rutledge and Sadler 2007; Aikenhead and Ryan 1992; Siegel and Ranney 2003; Lombrozo et al. 2008; Johnson and Peeples 1987; Miller et al. 2006), to date, the present survey represents the most comprehensive attempt to incorporate a broad array of explanatory variables of theoretical importance.
Accordingly, the present work has three primary goals. First, we seek to model the structure of beliefs about and attitudes toward evolutionary theory and its application. To this end, we will apply confirmatory factor analysis to our multi-item survey to derive a reduced array of meaningful constructs. Doing so will enable us to pursue our second goal, namely, to explore and model the complex interrelationships among the reduced number of constructs for maximal communicative and explanatory utility. Finally, we will employ hierarchical regression analysis to explore the degree to which key demographic and personality variables predict said constructs through which we will be able to address questions particularly concerning our regional sample.
Creating the Survey
Many learned authors have laid important groundwork for our present measures. Accordingly, we may accurately claim that our intended dimensions are theoretically grounded. We will take each of these subscales in turn.
Exposure to Evolutionary Theory
Research has shown that those who seek information about evolutionary theory may in fact know more about it and accordingly hold more positive attitudes (Lombrozo et al. 2008). Natural history museums, for example, function in part to teach the public about evolution (Diamond and Evans 2007). Therefore, museum visitors appear to (a) know more about evolution than non-visitors (McFadden et al. 2007) and (b) hold more positive views. Accordingly, 28% of museum visitors engage in creationist reasoning compared to 45% of the general public (Evans et al. 2006). For these reasons, it is important to assess respondents’ exposure to evolution during youth (presumably due to high school education and/or parental influence) and self-selected exposure in adulthood.
Religious Identity and Activity
One’s religious/spiritual orientation appears to play a primary role in one’s standing on evolutionary theory. For example, conservative Christian denominations are less accepting than more liberal denominations (Scott 2004). In addition to assessing participants’ qualitative religious affiliation (identity), we also queried them about how active they are in their spiritual behavior (e.g., frequency of prayer and service attendance), previously documented to be negatively associated with knowledge of and attitudes toward evolution (e.g., Lombrozo et al. 2008).
Young Earth Creationist Beliefs
In order to assess the degree to which biblical literalism (i.e., the earth was made by God in its present form over a span of six days less than 10,000 years ago, etc.) is represented on college campuses, and its role in attitudes toward and knowledge about evolution, we assessed young earth creationist beliefs as inspired by our reading of, for example, Dinosaurs by Design (Gish et al. 1992; see also Whitcomb and Morris 1961).
Arguably, objections to evolutionary theory based on moral grounds (e.g., evolutionary theory stripping meaning from our lives) are associated with religious orientation (Dennett 1995; Scott 2004). Indeed, 55% of Americans believe that science and religion are in conflict (Pew 2009). At the same time, moral objections may contribute explanatory power over and above religiosity.
Political Ideology and Activity
When construed as motivated social cognition with deep psychological roots (i.e., a belief system that reduces uncertainty and threat; Carney et al. 2008), political conservatism has been found to be associated with support for a traditional authority, religious forms of morality, perceptions of the world as dangerous, an intolerance for ambiguity and unpredictability, and more moral condemnation (Jost 2006). As such, conservatism has been described as having a palliative function for the stress associated with uncertainty and change (Jost and Hunyady 2002). In contrast, liberalism is associated with more open-minded thinking, a predilection for novelty and creativity, and a willingness to engage in prolonged, effortful cognition (Jost 2006; Kruglanski et al. 2006; Jost et al. 2003). Because the theory of evolution does not ascribe special meaning to human life, nor offer prescriptions on how a life can be well lived, its acceptance might be expected to positively associate with liberalism and negatively associate with conservatism (Miller et al. 2006 supporting material; Paterson and Rossow 1999). In addition to assessing political self-identity, we also asked participants to report the degree to which they are politically active or to what degree they perceive that their political selves influence their daily lives.
Attitudes Toward Life
Relevant to religious and political ideology are one’s beliefs about the timing of life’s beginning (Miller et al. 2006). If one holds an ideological commitment to human exemptionalism (e.g., humans standing apart from the natural world; Richards 2008), then one also might view human life as sacred at the moment of conception. Accordingly, we assessed such beliefs.
The intelligent design proponents are technologically sophisticated and well organized. These facts make it easy to identify key intelligent design (ID) talking points and slogans (e.g., talkorigins.org; discovery.org). To our way of thinking, the degree to which respondents endorsed such items (e.g., “Evolution is a theory in crisis”; Denton 1986; Perloff 1999) reflects the degree to which the ID movement has been successful in its dissemination enterprise.
Scientific, Genetic, and Evolutionary Literacy
In terms of literacy, we assessed both knowledge of evolutionary processes and genetic literacy (Miller et al. 2006), fully anticipating that both of these constructs would be positively correlated. In addition, we assessed a basic knowledge about the scientific enterprise (i.e., scientific epistemology: e.g., understanding the distinction between “theory” and “belief”; see Liang et al. 2006; Johnson and Peeples 1987; Lombrozo et al. 2008). Regarding knowledge about evolution, we included two types of questions: those that can be easily answered with some basic exposure (e.g., tree thinking; Baum et al. 2005; the role of mutation; Garvin-Doxas and Klymkowsky 2008) and those that appear to reflect common and persistent misconceptions that obstruct understanding (e.g., Alters and Nelson 2002; Evans 2001). For example, students of biology tend to hold on to Lamarckian-esque notions (i.e., the inheritance of acquired characteristics) that are difficult to change even after direct instruction (e.g., Bishop and Anderson 1990). In this sense, then, Lamarckianism is a conventional wisdom in that it is a simplistic and widely accepted false proposition. Thus, we included two constructs encompassing knowledge, evolutionary misconceptions and evolutionary knowledge, with the suspicion that they may behave differently from each other.
Distrust of the Scientific Enterprise
Additionally undermining the acceptance of evolutionary theory might be one’s general attitudes toward the scientific enterprise and its methods (Rutledge and Warden 1999), especially as the enterprise is associated with evolution (Lombrozo et al. 2008; Downie and Barron 2000; Scott 2004).
Relevance of Evolutionary Theory
Additional obstacles to accepting evolutionary theory are attitudes regarding its relevance (e.g., Wilson 2005). Regarding attitudes, we queried respondents about their attitudes on evolutionary theory’s explanatory relevance. We hypothesize that the more participants know about the theory of evolution, the more they will find it relevant for fields evoking it (e.g., zoology, botany, and human affairs). Intentionally, the items were graded from the relatively non-controversial application to plants, biology, and animals and then stepping into applications involving humans such as human origins, human behavior, and ultimately the humanities as a discipline (Wilson 2005). Many authors have found little opposition to evolution when it is applied to domains outside the purview of humans. When it comes to human origins or behavior, however, there is resistance even in college students (Sinclair et al. 1997; Richards 2008).
An additional obstacle to the acceptance of evolutionary theory is its perceived implications. Many students reason that if natural selection acts on variability within a species, and that some of that variability is “better” than other variants for the ecological context, then such variability must exist in humans. From there, it is a short leap to racist and sexist conclusions, many believe. Thus, the implications of the theory for human affairs are seen as unsavory. Early fallacious applications of Darwin’s theory (e.g., Spencer and social evolution) seem to suggest that the theory has a racist, sexist past (see also Eugenics movement). Understandably then, social objections abound, and they may come from both the left and right (as recent history suggests; Segerstråle 2001). Indeed, after course instruction, students appear to be even more aware of the historical baggage associated with the theory, and endorsement of these items increases (e.g., O’Brien et al. 2009; see also Brem et al. 2003).
Though there is a perception (perhaps accurate) that people of the “bible belt” would be less accepting of evolution, that predictability may be amplified when additional demographic variables are considered such as the education of parents, size of graduating class, whether the community is more rural or less rural, etc. In these senses, with its large rural community and multitude of very small high schools, Kansas provides an excellent opportunity to explore these stereotypes.
Personality reflects the basic ways in which humans differ in relatively stable patterns of acting and experiencing. Many psychologists endorse what is known as the five-factor model which describes five fundamental dimensions of personality (i.e., neuroticism, agreeableness, conscientiousness, extroversion, and openness to experience; McCrae and John 1992). Among other qualities, openness to experience represents intellectual curiosity and preference for variety. As such, we measured openness to experience because it has been previously found to be related to the willingness to entertain difficult subject matter like the theory of evolution (Sinatra et al. 2003).
Materials and Methods
We developed the present instantiation of the survey over the course of three semesters. The present analyses are based on data drawn from two classes (Child Psychology and Social Psychology) via a web interface. The classes were chosen based on the fact that they both attract a wide variety of students representing many majors and disciplines and that neither course exposes students to an evolutionary curriculum of any note.
In total, 371 undergraduates at the university participated and represented nearly 40 declared majors. Three hundred twenty-seven were drawn from Child Psychology (37 majors comprised this class, 32% were declared Psychology majors) and 44 came from a Social Psychology course (11 majors were represented, 63% were declared Psychology majors). The average age was 20.67 years; 102 were men and 269 were women. Additionally, the average high school graduating class was 374.21 (SD = 321.71), and the most frequent response for both the participant’s father‘s (N = 109, 30.36%) and mother’s (N = 126, 34.24%) education was 4-year college degree. Finally, the average rating for the degree to which one’s pre-university environment was rural was 3.45 (SD = 1.62) on a seven-point scale. For both classes, the students were invited to participate for extra course credit and were asked to complete the survey outside of class time via an easy-to-access link emailed to them. The students reported informally that it took them about 25 minutes to complete.
The Evolutionary Attitudes and Literacy Survey
The survey consists of 17 pages of web-presented items where respondents rated the degree to which they agree or disagree with 104 statementsFootnote 3 on a 1 to 7 scale (1 = strongly disagree, 4 = neither agree nor disagree, 7 = strongly agree). Five statements were removed from the current analyses because of very low item variance for these items that was accounted for by their corresponding constructs. The bulk of the survey was organized to measure 16 meaningful constructs introduced above: Self-exposure to evolution, youth exposure to evolution, religious activityFootnote 4, young earth creationist beliefs, moral objections, political ideology, political activity, attitudes toward life, intelligent design fallacies, knowledge about the scientific enterprise, genetic literacy, evolutionary knowledge, evolutionary misconceptions, distrust of the scientific enterprise, relevance of evolutionary theory, and social objections. (Example items are provided below. The survey in its entirety can be found in Table 1). Key demographic variables were also assessed and explored here, including gender, size of high school graduating class, father’s education, mother’s education, the degree to which one’s pre-university environment was rural, and number of siblings.
The constructs were as follows: (1) Self-exposure to evolution (five items): I read science magazines featuring evolution (e.g., Discover, National Geographic, Nature). (2) Youth exposure to evolution (four items): How much training in evolution did you receive in high school? I have visited natural history museums on field trips or with family. (3) Religious activity (six items): To what degree does religion impact your daily life? How much do you believe in God? (4) Young earth creationist beliefs (nine items): The Earth isn’t old enough for evolution to have taken place. Present animal diversity can be explained by the Great Flood. (5) Moral objections (five items): People who accept evolution do not believe in God. Darwinism strips meaning from our lives. (6) Conservative self-identity (i.e., political ideology; five items): To what degree are you conservative? In general, how do you self-identify politically? (7) Political Activity (six items): To what degree are you politically active? To what degree do your political views influence your daily life? (8) Attitudes toward life (three items; all from Miller et al. 2006): All stages of human life—embryo, fetus, child, adult—should have the same legal protections. Life begins at conception. (9) Intelligent design fallacies (11 items): Complex biological systems cannot come about by slight successive modifications (i.e., they are irreducibly complex). Natural selection cannot create complex structures; it is like a tornado blowing through a junkyard and creating a 747. (10) Knowledge about the scientific enterprise (six items): Scientific ideas can be tested and supported by feelings and beliefs. For scientific evidence to be deemed adequate, it must be reproducible by others. (11) Genetic literacy (nine items; six items directly from Miller et al. 2006): Today it is not possible to transfer genes from one species of animal to another. Ordinary tomatoes do not have genes, whereas genetically modified tomatoes do. (12) Evolutionary knowledge (seven items): Increased genetic variability makes a population more resistant to extinction. The more recently species share a common ancestor, the more closely related they are. (13) Evolutionary misconceptions (six items): Natural selection is a random process. Characteristics acquired during the lifetime of an organism are passed down to that individual’s offspring. (14) Distrust of the scientific enterprise (seven items; three drawn from Rutledge and Sadler 2007): Contemporary methods of determining the age of fossils and rocks are untrustworthy. Evolutionary theorists believe that if something is natural then it is good or right. (15) Relevance of evolutionary theory (nine items): The theory of evolution helps us understand plants. Evolutionary theory is highly relevant for the humanities (e.g., history, literature, philosophy). (16) Social objections (six items): The theory of evolution has contributed to racism. Applying the theory of evolution to human affairs implies we are not fully in control of our behavior.
The Big Five Inventory
In its entirety, the Big Five Inventory (BFI; Benet-Martinez and John 1998) is a collection of 44 short descriptive phrases designed to assess openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism. Only the ten-item scale for openness to experience (e.g., “I see myself as someone who is an ingenious, deep thinker”) was used for the present study as it conceptually addresses the “breadth, depth, originality, and complexity of an individual’s mental and experiential life” (John et al. 2008, p. 120). The BFI is widely used, has been internationally validated, and is currently available in at least ten languages.
Prior to analyses, we screened the data for univariate and multivariate outliers, and normality. None of the items demonstrated significant skew or kurtosis (i.e., no values were greater than ±2 in either measure of non-normality). Additionally, we examined scatter plots of all item pairs within each hypothesized construct and found support for linear relations and bivariate normality among them. Overall, <5% of the data were missing. Thus, missing data were inputted one time via Markov chain Monte Carlo estimation using Proc MI in SAS, version 9.1.3 (Enders 2010; SAS Institute 2005).
Because the 16 constructs were developed a priori, a confirmatory factor analysis (CFA) was used across the 104-scale items in order to verify that each scale item (i.e., manifest indicator) loaded on the latent construct that it was developed to measure. For example, a participant’s response to the item “individuals don’t evolve, species do” was hypothesized to be an indicator of evolutionary knowledge. Therefore, a loading was specified between the latent construct evolutionary knowledge and the scale item. With CFA, the collection of hypothesized paths is estimated to determine if the items’ covariations are adequately accounted for by their corresponding latent constructs.
We fit a second-order CFA model to ease interpretation of the scale and increase parsimony. This higher order CFA model was fit to the unit-weighted factor scores of the item-level CFA and was used to determine if some of the 16 lower order latent factors possess covariations that can be accounted for (i.e., further explained) by a higher order factor. Specifically, we took the mean of all of the items within a particular construct and used these construct means as the indicators for the higher order CFA. This approach gives the same generalizability as a simultaneously fitted model but has the advantages of an evaluated model fit separately for the lower order and the higher order levels. This approach also has added benefits in that it requires fewer parameter estimates, reduces sampling error, and decreases the likelihood of correlated residuals (Little et al. 2002) compared to item-level analyses.
Once the structure of the higher order factors was confirmed via the CFA model, we conducted sequential latent regressions for each higher order factor. Because the constructs are corrected for measurement error, latent regressions provide unbiased estimates of the regression relations among constructs. Latent regressions also allow for the predictive role of numerous latent variables (e.g., demographics and openness to experience) to be examined across multiple dependent variables (in this case, the six higher order factors of the Evolutionary Attitudes and Literacy Survey (EALS)). The first set of predictors consisted of the measured demographic variables. Using item parcels (Little et al. 2002), we examined whether the nine-item openness to experience construct accounts for unique variance above and beyond demographic variables. In other words, the unique effect of openness to experience was examined as a predictor for each of the six higher order factors of the EALS. All factor analyses and latent regressions were conducted using maximum likelihood estimation via the software package Mplus, version 5.2.
Confirmatory Factor Analysis
Table 1 displays all items with their variance accounted for by the underlying factor for the EALS. Additionally, the latent factors are accordingly named and displayed with their alpha reliabilities, means, and standard deviations. Because the model was identified by fixing the variances of the latent constructs to 1.00, the standardized factor loading for a particular item can be calculated by taking the square root of the item’s variance explained by the underlying factor. Overall, the item-level CFA supported our hypothesized model with acceptable fit (χ2 (5,132, N = 371) = 10,304.92, p < 0.0001; comparative fit index (CFI) = 0.79; Tucker–Lewis fit index (TLI) = 0.78; RMSEA = 0.052(0.051,0.054), SRMR = 0.065). Although the CFI and TLI were low, the other fit statistics demonstrate acceptable model fit. In order to ensure that the CFA produced the best fitting model, modification indices were examined. These indices were relatively low and mainly included within-construct correlations, which should be expected with a large model.
Table 2 contains the lower order latent factor correlations. These lower order correlations were examined to ensure there were 16 distinct constructs. A high correlation (r = 0.99) between intelligent design fallacies and distrust of the scientific enterprise was a potential concern, and thus, a new nested model was created to test if this correlation could be considered 1.00. For the nested model, all other latent correlations with the constructs intelligent design fallacies and distrust of the scientific enterprise were equated. For example, the correlation between intelligent design fallacies and evolutionary knowledge was forced to equal the correlation between distrust of the scientific enterprise and evolutionary knowledge. This process was repeated for the remaining relationships between these constructs so that only one set of latent correlations was estimated for these constructs instead of two. Finally, the latent correlation between intelligent design fallacies and distrust of the scientific enterprise was fixed to 1. Thus, this new model contained fewer parameter estimates than the original CFA (i.e., nested within the original CFA) and could be tested for improved fit. A chi-square difference test of the resulting model (χ diff 2 (15, N = 371) = 42.466, p < 0.0001) suggested that these two constructs should not have their correlations equated and that they indeed have differential relationships with other constructs. Thus, based on these results as well as strong theoretical support, the two constructs were judged to be distinct enough from one another to keep them separate.
Figure 1 displays a path diagram depicting the higher order factor model. Overall, the hierarchical CFA produced very good model fit (χ2 (203, N = 371) = 421.61, p < 0.0001, CFI = 0.95, TLI = 0.93, RMSEA = 0.054(0.047,0.061), SRMR = 0.048). Although two higher order constructs each significantly accounted for the variance of only a single lower order construct (political activity and evolutionary misconceptions), four other higher order constructs contained multiple lower order constructs. Specifically, a higher order construct (hereafter referred to as Political/Religious Conservatism) was positively indicated by conservative self-identity (political ideology), religious activity, attitudes toward life (pro-life direction), intelligent design fallacies, and young earth creationist beliefs and was negatively indicated by relevance of evolutionary theory. Another construct, hereafter referred to as Creationist Reasoning, was positively indicated by intelligent design fallacies, young earth creationist beliefs, moral objections, social objections, and distrust of the scientific enterprise. The construct Knowledge/Relevance was positively indicated by relevance of evolutionary theory, genetic literacy, evolutionary knowledge, and knowledge about the scientific enterprise. Lastly, both self-exposure to evolution and youth exposure to evolution were positively indicated by a higher order factor we refer to as Exposure to Evolution.
Inter-Higher Order Factor Correlations
Table 3 contains the latent correlations for the higher order factors. As can be seen in Table 3, Creationist Reasoning was positively correlated with Political/Religious Conservatism (r = 0.687) and negatively correlated with Knowledge/Relevance (r = −0.798) and Exposure to Evolution (r = −0.588). Accordingly, Political/Religious Conservatism was also negatively correlated with Knowledge/Relevance (r = −0.459). Though Exposure to Evolution was positively associated with Knowledge/Relevance (r = 0.555), Evolutionary Misconceptions was wholly uncorrelated with Exposure to Evolution (r = 0.013).
Table 4 displays the R2 before and after openness to experience was added as a predictor, as well as the standardized regression coefficients (β) for each predictor after openness to experience was added. All significant values are based on the Wald statistic.
As a set, the demographic variables only predicted a small amount of variance for each construct, with Evolutionary Misconceptions having the least amount of variance explained (R2 = 0.02) and Exposure to Evolution having the most variance explained (R2 = 0.083). No demographic variables significantly predicted Political Activity. High school size significantly predicted Political/Religious Conservatism (β = −0.176, p < 0.01). Gender (being male versus being female) significantly predicted Knowledge/Relevance (β = 0.137, p < 0.05). Father’s education significantly predicted Creationist Reasoning (β = −0.126, p < 0.05). Finally, gender (β = 0.139, p < 0.05) and father’s education (β = 0.167, p < 0.01) significantly predicted Exposure to Evolution.
The increase in R2 when openness to experience was added as a predictor varied across each construct. Openness to experience was a significant positive predictor for Political Activity (β = 0.208, p < 0.001, R2 = 0.064), Knowledge/Relevance (β = 0.316, p < 0.001, R2 = 0.152), and Exposure to Evolution (β = 0.258, p < 0.001, R2 = 0.168). Conversely, openness to experience was a significant negative predictor for Creationist Reasoning (β = −0.248, p < 0.001, R2 = 0.118) and Political/Religious Conservatism (β = −0.152 p < 0.05, R2 = 0.092).
The present study had two goals: to confirm the structure of our survey instrument in order to maximize its utility as an assessment tool and then to use the said structure to explore predictors of knowledge and attitudes in Kansas university students. Regarding the first and primary goal, our 104-item survey distilled down well into 16 multi-item constructs (all with excellent reliabilities) in this large sample of university students from varied disciplines. Specifically, the absence of correlated residuals, few dual-factor loadings between items, as well as low modification indices demonstrated both convergent and divergent validity. Similar items loaded uniquely onto a single construct (i.e., convergence) and, as demonstrated in Table 2, were distinct from other constructs (i.e., divergence). Moreover, the hierarchical SEM analysis showed the 16 constructs to have a theoretically meaningful and quantitatively coherent higher order factor structure. For the sake of parsimony and the avoidance of redundancy (because of the high correlations among the 16 constructs), we can now focus our discussion on this higher order structure with which we addressed our secondary goal of predicting attitudes and knowledge of Kansan university students.Footnote 5
Higher Order Survey and the Structure of Cognition and Attitudes
Political activity, though it did not map onto any other constructs in our model, was positively correlated with Exposure to Evolution (0.436), negatively correlated with Creationist Reasoning (−0.273), and openness to experience alone accounted for the bulk of its variance accounted for (4%; see Table 4). These results seem to suggest that politically active individuals, regardless of political self-identity, are critical knowledge seekers and as such may be attracted intellectually to the politically charged topic of evolution (however, Political Activity does not appear to be significantly related to Knowledge/Relevance in this sample).
Given the sociopolitical atmosphere surrounding evolution education over the last decade, we are not surprised to see a high positive correlation between Creationist Reasoning and Political/Religious Conservatism (see also O’Brien et al. 2009). Similarly, the significant correlation between Political/Religious Conservatism and Exposure to Evolution stands to reason: The conservatively oriented who engage in creationist reasoning (replete with moral and social objections, as well as a distrust of the scientific enterprise) do not presently seek out exposure to evolution, nor were they exposed to it early on. Accordingly, both Creationist Reasoning (especially) and Political/Religious Conservatism are negatively associated with Knowledge/Relevance in this university sample.
Exposure, Knowledge, and Attitudes
At the same time, Exposure to Evolution is strongly positively associated with Knowledge/Relevance; that is, those who have been exposed and choose to expose themselves to evolution seem to have some understanding and views of its relevancy (see also O’Brien et al. 2009; Moore and Cotner 2009). This point might be of particular interest to educators, though this relationship should be interpreted with caution because causal direction cannot be assured. Positive attitudes may cause people to seek out information, exposure can lead to positive attitudes, or some other variable can cause them both (see, for example, the discussion of openness to experience below). Moreover, self-reports of exposure may be biased depending on one’s attitudes; those positively inclined may either report or remember more exposure experiences.
An additional novelty offered by the present survey is the Evolutionary Misconceptions higher order factor. Perhaps seeming paradoxical at first, Evolutionary Misconceptions is uncorrelated with Exposure to Evolution. To try to make sense of these patterns (which should be validated on additional samples), we offer some speculations about cognitive biases.
A significant body of empirical research suggests that misconceptions are common, difficult to change, and manifest in predictable ways (D’Avanzo 2003; Donnelly et al. 2007; Sinatra et al. 2008). These cognitive biases can be thought of as cognitive organizational “rules of thumb” and as such reflect “common sense” thinking (Sinatra et al. 2008). The most prominent naïve views relating to evolution are essentialism (species and their traits are immutable), teleology (species evolve toward “perfection”), and intentionality (evolution is due to the intention or “need” of the species). Informal explanations of natural selection often ascribe agency to evolutionary processes (Moore et al. 2002)Footnote 6, and such falsehoods may become more believable with repeated exposure (i.e., the “illusion of truth”; Halpern 2003). In the end, the null relationship between Evolutionary Misconceptions and Exposure to Evolution in this university sample suggests that we may not be able to predict falling prey to these falsehoods based on one’s exposure to evolutionary theory. At the very least, this relationship suggests that educators should be mindful of the ways they describe evolutionary processes to students of all levels. At the same time, Evolutionary Misconceptions moderately negatively related to Creationist Reasoning (r = −0.257). In contrast to the possible process described above, creationists’ misunderstandings of evolution probably derive from fundamental religious beliefs that are perceived to be at odds with evolutionary theory, be that reasoning sound or not.
Though stereotypes about Kansans might suggest that those living rurally or coming from very small high schools would be more susceptible to political inactivity, religious conservatism, negative attitudes, and lack of literacy and exposure, little evidence supported these clichés in the present university sample. The latent regressions showed that high school size was modestly related to conservatism (the smaller the class, the higher endorsement of conservatism). But whether those participants were raised in a rural community or came from large families appears to be largely immaterial. Similarly, the education of the parents played less of a role than some might expect; education of the mother was not significantly related to any of the constructs, and education of the father slightly negatively related to Creationist Reasoning and positively associated with Exposure to Evolution. Gender, in contrast, was associated with both Knowledge/Relevance and Exposure to Evolution (favoring males).
Openness to Experience
Demographic variables (gender, rural, education of parents) played little role in predicting Knowledge/Relevance or Creationist Reasoning. On the other hand, the personality factor openness to experience (a stable aspect of personality) appears to play a rather large role in this Kansas university sample (see also Sinatra et al. 2003; Sá et al. 1999). Namely, openness to experience positively predicted Political Activity, Knowledge/Relevance, and Exposure to Evolution and negatively predicted Creationist Reasoning and Political/Religious Conservatism (see Kruglanski et al. 2006; Jost et al. 2003). In fact, a good deal of the variance accounted for in these factors was due to personality, especially for Knowledge/Relevance (10% of the variance uniquely), Creationist Reasoning (approximately 6% uniquely accounted for), and Exposure to Evolution (approximately 7% uniquely).
These patterns raise an important question: Can educators increase openness to experience with the possible trickle-down effect of students pursuing, understanding, and finding relevant their science education? Although openness to experience has generally been found to normatively increase through adolescence and young adulthood (Roberts et al. 2006; Lüdtke et al. 2009), significant changes in openness to experience over the college years appear to be due to a small proportion of individuals (Robins et al. 2001). Perhaps most instructive on this point is the work addressing creativity enhancement in so far as creativity is a central feature of openness to experience. Work in this area has addressed, among other things, fostering intrinsic motivation and collaborative atmospheres, teacher qualities, attitudes, and behaviors that enhance instructional effectiveness and innovative and flexible curricular approaches (see Esquivel 1995 for review).
Implications for Educators
A number of implications for educators can be derived from the present study. First, for example, Political/Religious Conservatism seems to hinder understanding of and attitudes of relevance toward evolutionary theory. Some authors argue that understanding is more important than acceptance (e.g., Ingram and Nelson 2006; Rutledge and Sadler 2007), and we agree to a point. But, knowledge and attitudes appear to be strongly linked, as evidenced by our singular Knowledge/Relevance higher order factor (see also Sinatra et al. 2003; Lawson and Worsnop 1992 for an extended discussion of cause and effect).
Moreover, standard practices of teaching philosophy of science have been criticized as inadequate (Moore et al. 2002; Alters and Nelson 2002) as they can fail to provide learners with an adequate organizational framework for understanding evolution (Turner and Sullenger 1999). Put simply, to understand basic principles in science and to overcome common misconceptions, one must possess sound scientific schema from which to confront existing cognitive biases and to integrate and accurately reproduce scientific knowledge. As Carl Sagan observed, “In the course of their training, scientists are equipped with a baloney detection kit.…What’s in the kit? Tools for skeptical thinking.…What skeptical thinking boils down to is the means to construct, and to understand, a reasoned argument and—especially important—to recognize a fallacious or fraudulent argument” (Sagan 1995; p. 196). Without skeptical thinking, students’ understandings will more easily fall prey to pervasive cognitive biases, a situation well understood by the Intelligent Design movement.
Ultimately, we may come to find that our derived structure is limited to our Kansas university sample (but preliminary analyses on data from New York students suggest that this concern may be unwarranted). Additionally, the very high correlation between the lower order constructs of intelligent design fallacies and distrust of the scientific enterprise should be interpreted with caution. Although the chi-square difference test indicated a significant difference between the nested model where the constructs were equated and the model where they were distinct, this difference test may reflect a type I error as the chi-square difference test can produce significant differences when testing within large samples. Therefore, a clear distinction between intelligent design fallacies and distrust of the scientific enterprise subscales cannot be assured. Future research should explore the relationship between these two constructs with new and different samples.
Conclusions and Future Directions
We hope that the present survey will provide a valuable tool for educators to assess some of the issues regarding scientific learning (especially as it pertains to evolution) specific to their unique regional populations. The relationship of the present survey to widely used epistemological measures would also be of interest to educational psychologists (e.g., Schommer-Aikins 2004). To readers who (understandably) find the survey in its present version still too long for everyday use (students tell us it takes approximately 25 minutes to complete), we are in the process of creating an informed short form in part based on the analyses presented here. Moreover, to fill in some gaps regarding causal directions, longitudinal analyses are needed. Indeed, to this end, we have already collected time 1 time 2 data on over 1,000 Kansas university students.
Regarding our educational mission, to us the message is clear: We should be cognizant not only of what we teach, but how we teach (e.g., O’Brien et al. 2009) to best educate and enthrall our students about the scientific enterprise, to illuminate evolutionary theory and its fascinating application to human affairs, and to arm them against fallacious, pseudo-scientific messages.
Comment cards submitted at the Explore Evolution Exhibit at The Natural History Museum of the University of Kansas.
After Frank (2004).
Throughout the process of survey construction, many items were discarded due to various reasons to create the final reported 104 items. For example, at least one item was needlessly long, complex, and had very low item variance explained (0.094) relative to other items in the construct (i.e., “If human beings are solely the product of chance, meaningless forces, then one can rightly question the value of man, the significance of life, and whether there is any basis for morality.” Similarly, “The vast majority of scientists accept evolution as a scientifically valid theory” (Rutledge and Sadler 2007) and “Up to this point in college, how many courses have you had with evolutionary themes (e.g., biology, anthropology, botany, psychology)”? had low item variances explained. Finally, one item from the evolutionary knowledge construct was removed because it created a negative residual variance (e.g., Heywood case) in the CFA that resulted in a non-positive definite matrix (“Individuals better suited to their environment are more likely to survive and reproduce”).
As mentioned, religious identity was also assessed. However, because this is a strictly categorical variable with 21 nominal response possibilities, it is not appropriate to include when establishing the structure of the instrument. Instead, it is more correctly used in secondary analyses (e.g., exploring mean differences). For this reason, religious identity will no longer be discussed in the present article.
Though Kansas is widely held to be both highly conservative and a hotbed of young earth creationist beliefs, it is impossible at this time to interpret the means across the higher order factors without other appropriate regional university samples for comparison.
An advanced student recently explained to the first author that there “needed” to be more chromosomally male conceptions because of the higher death rate of males throughout the life span. This way of thinking and lax use of language becomes part of the academic culture and is accordingly passed on to the undergraduate.
Aikenhead GS, Ryan AG. The development of a new instrument: “Views on Science-Technology-Society” (VOSTS). Sci Educ. 1992;76(5):477–91.
Alters BJ, Nelson CE. Perspective: teaching evolution in higher education. Evolution. 2002;56(10):1891–901.
Baum DA, Smith SD, Donovan SS. The tree-thinking challenge. Science. 2005;310:979–80.
Benet-Martinez V, John OP. Los Cinco Grandes across cultures and ethnic groups: multitrait multimethod analyses of the Big Five in Spanish and English. J Pers Soc Psychol. 1998;75:729–50.
Bishop BA, Anderson CW. Student conceptions of natural selection and its role in evolution. J Res Sci Teach. 1990;27(5):415–27.
Brem SK, Ranney M, Schindel J. Perceived consequences of evolution: college students perceive negative personal and social impact in evolutionary theory. Sci Educ. 2003;87(2):181–206.
Carney DA, Jost JT, Gosling SD, Potter J. The secret lives of liberals and conservatives: personality profiles, interaction styles, and the things they leave behind. Polit Psychol. 2008;29(6):807–40.
D’Avanzo C. Research on learning: potential for improving college ecology teaching. Front Environ Ecol. 2003;1(10):533–40.
Dennett DC. Darwin’s dangerous idea: evolution and the meanings of life. New York: Simon and Schuster; 1995.
Denton M. Evolution: a theory in crisis. Chevy Chase: Adler & Adler; 1986.
Diamond J, Evans EM. Museums teach evolution. Evolution. 2007;61(6):1500–6.
Donnelly LA, Kazempour M, Amirshokoohi A. High school students’ perceptions of evolution instruction: acceptance and evolution learning experiences. Res Sci Educ. 2007;39(5):643–60.
Downie JR, Barron NJ. Evolution and religion: attitudes of Scottish first year biology and medical students to the teaching of evolutionary biology. J Biol Educ. 2000;34:139–46.
Dudley RL, & Cruise RJ. Measuring religious maturity: A proposed scale. Review of Religious Research. 1990;32:97–109.
Enders WK. Applied missing data analysis. New York: Guilford; 2010.
Esquivel GB. Teacher behaviors that foster creativity. Educ Psychol Rev. 1995;7:185–202.
Evans EM. Cognitive and contextual factors in the emergence of diverse belief systems: creation versus evolution. Cogn Psychol. 2001;42:217–66.
Evans, E. M., Spiegel, A., Gram, W., Frazier, B., Cover, S., Tare, M., & Diamond, J. 2006. A conceptual guide to museum visitors’ understanding of evolution. Poster Session Presented at the American Educational Research Association, San Francisco, CA, April.
Frank T. What’s the matter with Kansas?: how conservatives won the heart of America. New York: Metropolitan Books; 2004.
Gallup Poll (February 11, 2009). On Darwin’s birthday, only 4 in 10 believe in evolution. Retrieved December 5, 2009. http://www.gallup.com/poll/114544/Darwin-Birthday-Believe-Evolution.aspx.
Garvin-Doxas K, Klymkowsky MW. Understanding randomness and its impact on student learning: lessons learned from building the biology concept inventory (BCI). CBE Life Sci Educ. 2008;7(2):227–33.
Gish DT, Clanin G, Snellenberger E, Snellenberger B. Dinosaurs by design. Green Forest: Master Books; 1992.
Halpern DF. Thought and knowledge. 4th ed. Mahwah: Lawrence Erlbaum; 2003.
Ingram EL, Nelson CE. Relationship between achievement and students’ acceptance of evolution or creation in an upper-level evolution course. J Res Sci Teach. 2006;43(1):7–24.
John OP, Naumann LP, Soto CJ. Paradigm shift to the integrative Big Five trait taxonomy: history, measurement, and conceptual issues. In: John OP, Robins RW, Pervin LA, editors. Handbook of personality: theory and research. New York: Guilford; 2008. p. 114–58.
Johnson RL, Peeples EE. The role of scientific understanding in college: student acceptance of evolution. Am Biol Teach. 1987;49(2):93–8.
Jost JT. The end of the end of ideology. Am Psychol. 2006;61(7):651–70.
Jost JT, Hunyady O. The psychology of system justification and the palliative function of ideology. Eur Rev Soc Psychol. 2002;13:111–53.
Jost JT, Kruglanski AW, Glaser J, Sulloway FJ. Political conservatism as motivated social cognition. Psychol Bull. 2003;129(3):339–75.
Kruglanski AW, Pierro A, Mannetti L, De Grada E. Groups as epistemic providers: need for closure and the unfolding of group-centrism. Psychol Rev. 2006;113(1):84–100.
Lawson AE, Worsnop WA. Learning about evolution and rejecting a belief in special creation: effects of reflective reasoning skill, prior knowledge, prior belief and religious commitment. J Res Sci Teach. 1992;29(2):143–66.
Liang LL, Chen S, Chen X, Kaya ON, Adams AD, Macklin M, Ebenezer J. 2006. Student Understanding of Science and Scientific Inquiry (SUSSI): revision and further validation of an assessment instrument. Paper Presented at the 2006 Annual Conference of the National Association for Research in Science Teaching (NARST), San Francisco, CA (April).
Little TD, Cunningham WA, Shahar G, Widaman KF. To parcel or not to parcel: exploring the question, weighing the merits. Struct Equ Modeling. 2002;9:151–73.
Lombrozo T, Thanukos A, Weisberg M. The importance of understanding the nature of science for accepting evolution. Evol Educ Outreach. 2008;1:290–8.
Lüdtke O, Trautwein U, Husemann N. Goal and personality trait development in a transitional period: assessing change and stability in personality development. Pers Soc Psychol Bull. 2009;35(4):428–41.
McCrae RR, John OP. An introduction to the five-factor model and its applications. J Pers. 1992;60:175–215.
McFadden BJ, Dunckel BA, Ellis S, Dierking LD, Abraham-Silver L, Kisiel J, et al. Natural history museum visitors’ understanding of evolution. Bioscience. 2007;57(10):875–82.
Miller JD, Scott EC, Okamoto S. Public acceptance of evolution. Science. 2006;313:765–6.
Moore R, Cotner S. Educational malpractice: the impact of including creationism in high school biology courses. Evol Educ Outreach. 2009;2:95–100.
Moore R, Mitchell G, Bally R, Inglis M, Day J, Jacobs D. Undergraduates’ understanding of evolution: ascriptions of agency as a problem for student learning. J Biol Educ. 2002;36(2):65–71.
O’Brien DT, Wilson DS, Hawley PH. “Evolution for Everyone”: a course that expands evolutionary theory beyond the biological sciences. Evol Educ Outreach. 2009;2(3):445–57.
Paterson FRA, Rossow LA. “Chained to the devil’s throne”: evolution & creation science as a religio-political issue. Am Biol Teach. 1999;5:358–64.
Perloff J. Tornado in a junkyard: the relentless myth of Darwinism. Arlington: Refuge Books; 1999.
Pew Research Center. Scientific achievements less prominent than a decade ago: public praises science; scientists fault public, media. Washington: Pew Research Center; 2009 (July).
Richards RA. Philosophical challenges in teaching evolution. Evol Educ Outreach. 2008;1:158–64.
Robins RW, Fraley RC, Roberts BW, Trzesniewski KH. A longitudinal study of personality change in young adulthood. J Pers. 2001;69(4):617–40.
Roberts BW, Walton KE, Viechtbauer W. Patterns of mean-level change in personality traits across the life course: a meta-analysis of longitudinal studies. Psychol Bull. 2006;132(1):1–25.
Rutledge ML, Warden MA. The development and validation of the measure of acceptance of the theory of evolution instrument. Sch Sci Math. 1999;99(1):13–8.
Rutledge ML, Sadler KC. Reliability of the measure of acceptance of the theory of evolution (MATE) instrument with university students. Am Biol Teach. 2007;69(9):332–5.
Sá WC, West RF, Stanovich KE. The domain specificity and generality of belief bias: searching for a generalizable critical thinking skill. J Educ Psychol. 1999;91:497–510.
Sagan C. The demon-haunted world: science as a candle in the dark. New York: Random House; 1995.
SAS Institute. Statistical analysis software (SAS), version 9.1.3. Cary: SAS; 2005.
Schommer-Aikins M. Explaining the epistemological belief system: introducing the embedded systemic model and coordinated research approach. Educ Psychol. 2004;39:19–29.
Scott EC. Evolution vs. creationism. Westport: Greenwood Press; 2004.
Segerstråle U. Defenders of the truth: the sociobiology debate. New York: Oxford University Press; 2001.
Siegel MA, Ranney MA. Developing the changes in attitude about the relevance of science (CARS) questionnaire and assessing two high school science classes. J Res Sci Teach. 2003;40(8):757–75.
Sinatra GM, Southerland SA, McConaughy F, Demastes JW. Intentions and beliefs in students’ understanding and acceptance of biological evolution. J Res Sci Teach. 2003;40(5):510–28.
Sinatra GM, Brem SK, Evans EM. Changing minds? Implications about conceptual change for teaching and learning about biological evolution. Evol Educ Outreach. 2008;1(2):189–95.
Sinclair A, Pendarvis MP, Baldwin B. The relationship between college zoology students’ beliefs about evolutionary theory and religion. J Res Dev Educ. 1997;30:118–25.
The American National Election Studies. The ANES guide to public opinion and electoral behavior. Center for Political Studies, University of Michigan, Ann Arbor, MI. 2009. Retrieved from http://www.electionstudies.org/
Turner S, Sullenger K. Kuhn in the classroom, Lakatos in the lab: science educators confront the nature-of-science debate. Sci Technol Human Values. 1999;24(1):5–30.
Whitcomb JC, Morris HR. The genesis flood: the biblical record and its scientific implications. Phillipsburg: Presbyterian and Reformed Publishing; 1961.
Wilson DS. Evolution for everyone: how to increase acceptance of, interest in, and knowledge about evolution. Public Libr Sci Biol. 2005;3:e364. doi:10.1371/journal.pbio.0030364.
Special thanks are owed to Hannah Parkinson, Carrie Mendoza and Thomas Sunderland for their superb intellectual contributions to earlier versions of the survey. We also extend our gratitude to David Sloan Wilson, Robert Hagen, Teresa MacDonald, Robert Deaner, James Orr, Dan O’Brien, Omri Gillath, Deb Smith, and Jennifer Gleason for early feedback and survey implementation. Finally, we thank the students of Kansas who participated in our survey.
The third and fourth authors contributed equally to the manuscript and are listed in alphabetical order.
About this article
Cite this article
Hawley, P.H., Short, S.D., McCune, L.A. et al. What’s the Matter with Kansas?: The Development and Confirmation of the Evolutionary Attitudes and Literacy Survey (EALS). Evo Edu Outreach 4, 117–132 (2011) doi:10.1007/s12052-010-0294-1
- Confirmatory factor analysis