One of the greatest challenges for a teacher or instructor is to discern what students are actually learning and thinking. All teachers have the experience of expounding on a key topic, with wonderful images, diagrams, and examples, only to find out on the subsequent test that half the class completely missed the point (see “Our expert advice remains unheeded” by Terry McGlynn). With classroom response systems, such as clickers and web-based bring-your-own-device systems, teachers don’t have to wait until the exam; they can find out within minutes, and try other ways of getting the point across.
Classroom response systems first came to national attention among college instructors via Eric Mazur’s Peer Instruction. Many instructors began to apply clickers in a variety of ways, as described by Derek Bruff in Teaching with Classroom Response Systems. These clickers allowed instructors to pose multiple-choice or true/false questions (some also allowed numeric or short alpha-numeric responses). Especially in large classes, clickers encourage all students to engage and answer the questions, and can lead to productive peer discussion (Smith et. al 2009).
We began using clickers several years ago in our large Intro Biology classes. I sprinkled several clicker questions through each of my lectures, and found them particularly useful in pre- and post-instruction assessment (see my Lac Operetta post as one example), and for exposing student misconceptions.
As with any technology, we did run into a few issues. Infra-red clickers had limited range and capacity, and we were restricted to lecture halls that were outfitted with the receivers. A switch to RF technology with USB receivers allowed us to use them in any classroom, but we still had occasional issues with dead spots in large lecture halls, and making sure that the receiver was placed in a suitable location. We also found some students who had multiple clickers, so their friends who skipped class would be counted in attendance and get credit for class participation. As we began to incorporate case studies, and especially after I adopted the flipped class model, the restriction to multiple-choice questions began to seem confining. If we want students to engage in a variety of activities during class, surely they should be answering a variety of question types.
This past year, in the 200-student Intro Biological Principles class in the fall and in the 30-student Developmental Biology class in the spring, I used Learning Catalytics (LC) as the classroom response system. LC is a web-based, bring-your-own-device system where students can use a laptop, tablet or smartphone to solve problems and answer questions either during or outside class. Developed by Eric Mazur, LC was sold to Pearson, who now packages it with their Mastering series of online homework and tutoring solutions that accompany their textbooks. Unlike Mastering, however, LC is available as a stand-alone product (no Pearson textbook required), for $12/semester per student (as of May 2014).
I and essentially all of my colleagues who had been using clickers switched to LC because we thought the added benefits were well worth the extra work. I’ll list the primary benefits from my own point of view, with some explanation:
1. Learning Catalytics is a step towards open education. Yes, it is a commercial product and costs money. However, instructors who write questions can make them available to other instructors anywhere in the world via the LC question database. All of the questions I have written are in the database. Instructors can search the question database by field (e.g., physics, chemistry, biology, engineering) and by instructor or meta tag. I have found a number of wonderful questions, and found inspiration for other questions, in the database. Multiple instructors can share a course and access each other’s modules (class lessons). A module of questions can be saved as a pdf and shared or printed.
2. LC enables student sketches, drawing, and graphing. Student sketches or drawings can be viewed individually or as composite sketches. You can ask students to graph onto axes that you provide, or finish incomplete illustrations. I found this a truly powerful tool, that I need to exploit more. A framework allows comparisons among the student drawings and identifying common patterns. Students can engage with diagrams and illustrations in other ways, by identifying a correct region of a figure, or drawing a directional vector arrow.
3. A non-synchronous mode for group work on problem sets, case studies. In the non-synchronous mode, students can access all of the questions and answer them in any order. This is useful for homework, but also for flipped class sessions where groups of students can work through a case study or a set of problems at their own pace. The instructor can see student progress in real-time, and see what questions are posing the greatest difficulties. For short-answer or long-answer questions, I could see when students were misinterpreting a question or when they needed more information, and I could intervene. I could also see on the seat map which groups of students were struggling, and wander around the classroom to provide help.
4. A seat map facilitates group formation for peer discussion, and for identifying groups of students who need help. Students indicate at the start of each session where in the classroom they are sitting. LC uses this information to pair students who had same or different answers for peer discussion. With one click on the instructor screen, student receive information as to whom they should pair with to discuss the question and respond again. The instructor can also view the seat map to see which students got a question right or wrong.
5. The team mode provides IF-AT (immediate feedback assessment technique) capability. I have tried this a few times, and this provokes intense discussion. The team mode is restricted to questions that have correct answers (no sketches, or open-ended response questions). At the start of the session, students form teams (they register a team name). Then they answer questions individually. The instructor can end the individual round at a set time, or when most students have finished, and start the team round. During the team round, only one member of each team answers the questions. If the team gets the correct answer in the first try, the team gets maximum credit. If the first answer is wrong, they have to try again until they get to the correct answer. Each attempt reduces the credit for the team. At the conclusion of the team round, everyone knows the correct answers. Scores are tabulated as sums of the individual and team scores. The weighting of the individual and team round scores is adjustable from 0 to 100%.
6. Many choice question type for multiple true-false. This quickly became one of my favorite question types. Structured like a multiple-choice question, zero to all of the answer choices may be designated correct. Students have to evaluate each answer choice. Very useful for exposing common misconceptions.
7. Open-ended short and long-answer questions. Even in a class with 200 students, we found these questions quite useful. Think of this as a minute paper, with no time spent to distribute or collect the papers. The instructor sees the responses in real-time, as some students respond more quickly than others. Since the students cannot see names associated with any of these responses, the instructor can point to and discuss aspects of individual responses.
8. Multiple other question types provide variety and engage higher order Bloom’s taxonomy skills. LC has a total of 18 different question types. Others I have used are: matching, ranking, highlighting, numerical, confidence, and priority.
What’s the downside? What can (and does) go wrong? I’ve talked with my colleagues on campus and other faculty at other universities who have used LC. With very large classes (over 500 students), the lecture hall (or auditorium) may have limited wifi bandwidth. One instructor at another university told me that they have to have students choose just one device (laptop or phone) and turn off their other devices. I have run into wifi deadspots where I or a student had a troublesome wifi connection. The one thing I miss about clickers is that we used them for our multiple-choice exams. With LC, we had to revert to Scantron forms, as we had no way to prevent students from using their laptops to exchange emails or look up information during the exam.
Some instructors will worry that students will be use their laptops or cell phones for web surfing, texting or other off-task purposes during class. I think it’s up to the instructor to keep the students engaged and on task. Even without laptops or cell phones, students found ways to disengage, whether it’s exchanging notes, talking to each other, doing homework for another class, playing cards, or reading the newspaper (all things I found students doing in the back of the class while the instructor lectured). My own experience is that I can see what percentage of students have responded to each question and can pace the class appropriately to keep students engaged. Wandering around the class, I saw that laptop screens were on Learning Catalytics, with very few exceptions.
I have NOT had any issues with students not having a device. Georgia Tech does have a laptop requirement. But it’s also rare that a student does not have a smartphone. The situation may be different at other colleges or universities, but with cell phone providers increasingly pushing smartphones, and the availability of cheap laptops and tablets for less than the cost of a textbook (or a new clicker), this issue will disappear entirely. In fact, I previously experienced almost daily issues with one or two students forgetting to bring their clickers. I had not one student come to me this past year about forgetting to bring a device for LC. Just one student had a low battery charge on her cell, but she was able to answer at least a couple of questions to record her presence in class.
I think that LC and similar web-enabled, BYOD systems like Top Hat and LectureTools are the logical next wave of classroom response technology. Even if all you want to do is ask multiple-choice questions, allowing students to use devices they already have is quite attractive. My own experience has been that LC involved less effort (no software to download and install, no additional infrastructure, no clicker registration) and was an easier transition than first adopting clickers, or transitioning from one clicker brand to another.
Student response to Learning Catalytics has been overwhelmingly positive. Seniors and even a graduate student in my Developmental Biology class saw the value and thought LC absolutely rocked compared to clickers. My favorite comment is from a freshman in the Intro class:
I like the way the professors let us engage in the class by solving learning catalytics questions. I wonder if other biology professors are using this methods too. If not I will be disappointed.
Now we just have to assess whether the greater variety of questions leads to any significant gains in student learning.