Interactive Evolution Modules Promote Conceptual Change

Abstract

We developed a suite of online modules (http://ats.doit.wisc.edu/biology/) to improve student understanding of challenging concepts in our introductory biology course at the University of Wisconsin-Madison. Here we assess the effectiveness of two of the modules, Species and Speciation and Natural Selection. In Year 1, unannounced pre-tests and post-tests were used to assess students’ prior knowledge and any gains resulting from lecture attendance alone. Then to test the effectiveness of the modules we divided the class into three groups. Group 1 was assigned the interactive speciation module and their analysis of the final case study in the module was graded. Groups 2 and 3 were controls. In Group 1, the subgroup of students whose mean scores on the first two exams in the course were <80% scored an average of 10.5 percentage points better on exam 3, a significant improvement. In contrast, none of the other student groups showed significant improvements in their grades. In Year 2, we tested the effectiveness of the online modules when offered as optional, ungraded, activities. We again saw significant improvement (+3.8 percentage points) only in those students who completed the modules and whose averages on the previous two exams were <80%. The differences in improvement between years 1 and 2 suggest that it is not enough simply to provide students with tools that help them learn; they also need an incentive in the form of a grade or course credit to use the tools most effectively.

Introduction

Evolution is a fundamental unifying concept in biology, yet it is often misunderstood by students. A number of inventories of key concepts in biology, including concepts in evolution, are available (Anderson et al. 2002; D’Avanzo 2008; Garvin-Doxas and Klymkowski 2008). The corresponding misconceptions held by students about natural selection and speciation have also been well documented (Anderson et al. 2002; Nehm and Reilly 2007; Sinatra et al. 2008; Tanner and Allen 2005). Common misconceptions include the ideas that individuals undergo changes in traits driven by need; evolution occurs at the level of individuals (versus populations); natural selection is random; and formation of new species is intentional. Misconceptions such as these have been attributed to a number of causes. For example, some arise from fundamental misunderstandings about causal relationships between the environment, the individual, and genotype (Bardapurkar 2008). Others arise from miscategorizing evolution as a process that occurs in individuals, rather than as an emergent property of populations of individuals interacting with their environment (Ferrari and Chi 1998). Yet others result from beliefs and intuitions influenced by personal, social, and cultural factors (Evans 2008; Sinatra et al. 2008).

Recognizing that these misconceptions are common, we made sure to address them in the lecture component of our introductory biology course for majors. We also developed a variety of exercises for students to use in both discussion/recitation sessions and in lab sessions. Still these misconceptions about evolution persisted. Given this, we examined the potential benefits of online activities for enhancing student learning. In doing this, we recognized that to be successful, any learning tools we developed had to be easily accessible outside the classroom and should allow students to work at their own pace. We were particularly interested in developing inquiry-based learning activities that promote conceptual change, that is, activities that help students correct their existing misconceptions.

Web enhancements to traditional courses have been shown to provide such active learning opportunities for students (Allesi and Trollip 2001). For example, Sandoval and Reiser (2004) describe an online environment for learning about natural selection through inquiry that improves students’ written explanations of the process of natural selection. Their design includes instructional scaffolding that prompts students to develop explanations that incorporate the epistemic frameworks, that is, the theoretical and empirical approaches of evolutionary biology. A number of other studies have examined learning with computer-based simulations of natural phenomena. A key finding of these studies is that when simulations are paired with appropriate instructional guidance, they allow students to construct meaning based on their experiences in the simulation environment (de Jong and van Joolingen 1998). For example, Jonassen and Ionas (2008) highlight the value of student use of simulations to help them learn to infer causal relationships between factors through experimentation. Windschitl and Andre (1998), using a simulation of the human circulatory system, found students who were prompted to formulate and test their own hypotheses experienced a greater degree of conceptual change than students who worked through the same simulation following prescribed steps. Similarly, Winn et al. (2006) reported that students using simulations of ocean water movement and salinity to frame and test hypotheses were better able to connect their experiences to concepts about physical processes learned in class.

With these ideas in mind, we developed a suite of online modules (“Connecting Concepts” at http://ats.doit.wisc.edu/biology/) to improve student understanding of challenging concepts in our introductory biology course. Our goal for the modules was to provide our students with interactivity and feedback to extend learning beyond what can be provided in the classroom. Our instructional strategy was to design motivating activities such as games, simulations, and case studies to reinforce concepts learned in lecture, help students uncover and devalue misconceptions (Nehm and Reilly 2007), and promote critical thinking. Two of the modules cover evolutionary biology concepts: Natural Selection and Species and Speciation (see Table 1.) To address the evolutionary misconceptions that interfere with some students’ understanding, each module begins with activities that promote conceptual change and revision through cognitive conflict and cycles of concept revision (Clement 2008). As they progress through each module, students apply these concepts in increasingly complex scenarios in which they formulate hypotheses, evaluate evidence and explain underlying causes of evolutionary events.

Table 1 Inventory of concepts, activities, and thinking skills in the Natural Selection and Species and Speciation modules

At the start of the Natural Selection module, students follow Darwin on a virtual field trip around the world. On the trip, Darwin contrasts three common modern-day misconceptions about natural selection with his own reasoning about the contributions of heritable variation and environmental selection to survival and reproduction. Next, playing a game called “Fitness Fever,” students are given multiple opportunities to experience and resolve misconceptions about fitness as an intentional response of the individual to its environment. Finally, a simulation of peppered moth populations provides a framework for students to gather data and evaluate whether selection by predators has led to changes in gene frequencies in the moth population over several generations.

Students begin the Species and Speciation module by examining the strengths and weaknesses of the biological, morphological, and phylogenetic concepts of species and sort populations of frogs into species using each concept. Next, they generate explanations about the causal relationships between geological events, dispersal of populations, and formation of new species of Fuchsia. At the end of the module, students analyze two case studies to determine whether the organisms described are unique species. To scaffold the complex reasoning needed to complete this task, the activity provides students with an “information library” of several types of evidence (morphology, haplotype, population histories, ecology, molecular phylogenies, hybridization, and geographic distributions). Students arrange each piece of evidence on a linear “evidence bank” according to its strength and validity and decide whether the evidence supports the separate species hypothesis.

In this study we assess the effectiveness of these two modules, Natural Selection and Species and Speciation, as learning-enhancement tools. To do this, in Year 1 we measured the performance of students who completed the interactive modules relative to those who completed non-interactive alternative assignments. In a subsequent year, with a new class of students, we tested both the number of students who would use the modules and the effectiveness of the modules when they were highly recommended, but presented as optional, ungraded exercises.

Methods

Evaluation of Module Effectiveness

Year 1: Species and Speciation

We tested the effect of the lecture material separately from the effect of the online modules in the first-semester course of our two-semester introductory biology sequence for majors (N = 283) at the University of Wisconsin-Madison. This course covers three topic areas—cell biology, genetics, and evolution and diversity—with an exam at the end of each unit.

As a test of the effectiveness of lecture alone, all students giving consent were asked to complete an evolution/speciation bioinventory test (Heitz et al. 2005) one week before (‘pre-test’) and one week after (‘post-test’) three lectures on speciation (Fig. 1). Neither test was announced in advance. Different but related questions were asked on the full-unit exam, exam 3 (the last exam in the course).

Fig. 1

Sequence of assessment steps, Year 1

The week following the post-test, we randomly divided the students into three groups to test the effectiveness of the online Species and Speciation module on student learning (Fig. 1). Group 1 was assigned and completed the interactive speciation module online, and their analysis of the case study was graded. Groups 2 and 3 were controls. Group 2 was given the information from the Species and Speciation module in non-interactive PDF format. Group 3 was assigned both a multi-part homework question on speciation and the PDFs for only one of the case studies from the speciation module. Together, the multipart question and case study were designed to focus students on the same concepts contained in the module (Table 1). The Group 2 and 3 assignments were also graded. After the assignments had been completed we opened both the Speciation and the Natural Selection modules to the whole class. We announced that the modules were available and that completion of these was completely voluntary.

All students were required to take exam 3, which was administered approximately three weeks after the post-test. This exam contained 31 questions on evolution (44/104 total pts or 42.3%) and 37 questions on diversity of organisms (56/104 pts or 53.8%). The remaining 3.9% contained the survey questions asked to determine which, if any, of the modules each student completed. About half of the 31 questions on evolution tested concepts that were specifically addressed both in the modules and in the control group assignments.

In Group 1 only students who verified that they actually completed the modules as well as the pre-tests and post-tests and exam 3 were included in the analysis (N = 51). In Groups 2 and 3 only students who verified that they did not access the modules online and that they completed the pre-tests and post-tests and exam 3 were included (N = 43 and 53, respectively). For the analysis of the effect both of lecture and of the modules, each test group was broken down into two approximately equal subgroups: students whose averages on the first two exams were <80% and students whose averages were ≥80%.

One-way ANOVA (analysis of variance) was used to determine whether differences among test groups were significant. Scores on pre-tests were compared among test groups to determine whether there were any significant inherent differences among the groups. Scores on pre-tests versus post-tests were compared for each test group as a whole and then for the <80% vs. ≥80% subgroups. Total exam 3 scores were compared to averages on the previous two exams. Total exam 3 scores were also compared to the scores students received for the evolution questions alone.

Year 2: Species and Speciation and Natural Selection

We conducted a second study on a different class of students the following year (N = 186). We have no reason to believe that the make-up of the student bodies in the two years differed demographically from one another. In year 2, we strongly recommended that the whole class complete both online modules. This second study allowed us to test both the number of students who would voluntarily use the modules and the effectiveness of the modules if they were presented as optional ancillary exercises. In other words, unlike in the previous year, no group was required to complete a module and no grades were associated with completion of the module; participation was entirely voluntary. Using students’ self reports that they did or did not complete the modules, we again analyzed the effectiveness of the online modules on their exam 3 grades using the same criteria as above.

The professor, the lecture content and the content of discussion/recitation sessions and lab sessions were the same for both years 1 and 2.

Results

Year 1: Species and Speciation

In year 1, students were divided into 3 Groups. Group 1 was given the interactive Speciation module as a graded assignment. The other two groups were given graded assignments covering similar content in non-interactive formats.

Pre-Test Versus Post-Test Scores

Using ANOVA to compare pre-test results in Year 1 revealed no significant difference among the three test groups (Table 2). Comparing scores on pre-tests versus post-tests, however, showed significant differences for all groups. This effect was largely driven by the subgroups who scored ≥80% on the previous two exams. In each group, the ≥80% subgroup showed a significant increase in post-test scores relative to pre-test scores (8.2 to 13.4 percentage points). On the other hand, improvement on post-test versus pre-tests by the <80% subgroup was not significant for Groups 2 and 3 and was significant at the .05 level for the <80% subgroup in Group 1.

Table 2 Scores on pre- vs. post-tests, Year 1. Data are mean scores (%±SD) for each group and subgroup

Exam 3 Scores

Comparing total exam 3 scores to averages on the previous two exams, only Group 1 showed a significant difference (Table 3, p < .001). In contrast to the pre-tests and post-test results, this difference was driven by a significant improvement in the <80% subgroup (an average of +10.5 percentage points: 72.7% vs. 83.2%; p < .001). For comparison, in Group 1 the ≥80% subgroup scored an average of 85.5% on the first two exams and 85.4% on exam 3 (p = .93). In other words, the Group 1 <80% subgroup scored within 3 percentage points of the mean of the ≥80% subgroup (85.4%). All other subgroups (in all three Groups) showed no significant improvement on their exam 3 scores over their previous exam scores (p = .17 to .93).

Table 3 Scores on exams, Year 1. Data are mean scores (%±SD) for each group and subgroup

Interestingly, the <80% subgroup in Group 1 showed similar improvement for both the evolution questions (81.0%) and exam 3 as a whole (83.2%; Table 4). For the other groups, scores on the evolution questions compared to scores on exam 3 as a whole were also similar, with p values for ANOVAs ranging from .34 to .99.

Table 4 Overall exam 3 scores, Year 1. Overall exam 3 scores (%±SD) are compared to scores for the subset of evolution questions alone

Year 2: Species and Speciation and Natural Selection

Given the findings in Year 1, we decided to test a new class of students the following year (Year 2). However, this time we asked how many students would voluntarily use the modules and what the effectiveness of the modules would be if they were presented as optional, ancillary exercises. To do this, both the speciation and natural selection modules were made available to the class as a whole and the faculty lecturer highly recommended that all students complete the modules. The lecturer, discussion/recitation activities, and lab exercises were the same as for Year 1.

Exam 3 Scores

Using students’ self reports that they did or did not complete the modules, we again found significant increases in the exam 3 scores for the <80% subgroup who completed the modules. This group had an average of 69% on exams 1 and 2 and an average of 72.8% on exam 3, an average increase of +3.8 percentage points (p = 0.03, Table 5). No other subgroup showed a significant increase in average scores. In fact, all other groups showed a slight decrease in exam 3 scores relative to their previous exam averages. For students whose previous exam scores were less than 80% and who opted not to complete the modules, their 2.8 percentage-point decrease on exam 3 was significant (Table 5).

Table 5 Scores on exams, Year 2. Data are mean scores (%±SD) for each subgroup.

Discussion

In Year 1, in a test of the effectiveness of lecture alone, all three groups showed significant improvement in post-test scores over pre-test scores. However, these increases were largely due to the performance of the ≥80% subgroups while the <80% subgroups gained little or not at all. This suggests that the ≥80% subgroup may be studying more, may be able to absorb more from lecture alone, or both. In all cases, however, the overall scores were low, ranging from 47.9 to 52.5% on the pre-test and 54.8 to 62.2% on the post-test. Since neither test was announced in advance, the pre-test scores are likely to be fair reflections of the knowledge students had at the time. Post-test scores are interpreted as gains resulting from lecture attendance alone. The assumption here is that it is unlikely that students spent significant time studying the material in the two weeks between the pre-tests and post-tests.

For all groups, average scores on exam 3 were significantly higher than average scores on the post-test. This indicates that while lecture experiences can help direct students to what and how to learn, lecture attendance alone is not sufficient for full development of understanding. Most students require both an incentive (e.g. an exam) and individual study time to more fully develop their understanding.

In contrast, a comparison of student scores on exam 3 provides clear evidence that the online interactive modules did benefit some students significantly. Both Groups 1 and 2 were given the same information in their homework assignments. The key difference was that Group 1 did the module interactively online and Group 2 read the information in non-interactive PDF format. The Group 1 students with grades <80% prior to the exam 3 gained an average of 10.5 percentage points on their exam 3 scores. On the other hand, the Group 2 students with grades <80% did not gain significantly. This leads us to conclude that the interactivity of the modules, with the opportunities they offer for conceptual change and reinforcement, provided clear learning benefits for the Group 1 <80% subgroup students.

Compared to their performance on the previous two exams, the Group 1 <80% subgroup did better not only on the evolution questions but on exam 3 as a whole. This suggests that the interactive module experience provides learning enhancement not directly related to the specific subject material. Discovering exactly why this occurs warrants further investigation.

During Year 2, we tested how effective the online modules would be if offered as optional ancillary activities. In this year, the online modules were highly recommended and made available to the entire class. Again we asked the students to self-report whether they did or did not complete the modules. The data provided additional evidence that the interactive modules do benefit some students significantly. Of those students who completed the modules, students with grades of <80% prior to exam 3 gained an average of 3.8 percentage points on exam 3. This gain is accentuated by the fact that the other groups all showed 1.7 to 2.8 percentage-point decreases in their exam 3 scores.

Based on these data, it is clear that interactive modules helped a specific subset of students, those with prior exam scores <80%. But are they of use to other students? In Year 1 we assigned the speciation module only to Group 1. However, after the assignment had been completed we opened both the Speciation and the Natural Selection modules to the whole class. We announced that the modules were available and that completion of these was completely voluntary at that point. Over 73% of students in Group 1 (in both <80% and ≥80% subgroups) completed the second module (natural selection). This indicates that the vast majority of these students found the first module to be valuable enough to motivate them to voluntarily complete the second. In comparison, only 11% of those in Groups 2 and 3, who had no previous experience with the first interactive module, opted to complete the natural selection module.

When we highly recommended the modules in Year 2 about half of the class reported that they completed both modules. However, the learning gains (as increases in exam 3 scores compared to previous exam scores) for the <80% subgroup were lower in Year 2 than for the Year 1 (3.8 percentage points compared to 10.5). In Year 1 the students’ analyses of the case studies in the speciation module were graded. In Year 2, there was no grade associated with the modules. The difference in gains between the two years may be a reflection of how much value was placed on the modules in one year versus the other. It is possible, for example, that without the incentive of a grade in year 2, students were less thorough in completing the modules.

We conclude, as others have (McDaniel et al. 2007), that inquiry-driven, Web-based modules can significantly enhance learning. In our situation improvement was particularly strong in students who were struggling with key concepts. For students already doing well, they provided little benefit as measured by changes in exam scores. However, that this group perceived benefit is evident in that 73% of those who completed the first assigned module also voluntarily completed the second, optional module. Based on the difference in performance between Groups 1 and 2 (who had the same information in interactive versus PDF format respectively), the critical ingredient behind the success of the modules appears to be their interactivity. Comparing our results when the modules were assigned versus optional suggests that it is not enough simply to provide students with tools that help them learn; they also need an incentive (e.g. in the form of a grade or course credit) to use the tools most effectively.