Writing in Online Asynchronous Courses

Celeste Tường Vy Sharpe, Nate Sleeter, and Kelly Schrum

Directions: Select the correct answer for the following question.

1. Why are multiple-choice quizzes a common assessment tool in online courses, even with topics that require more complex and nuanced thinking?

• A. Instructors love to write them.
• B. Taking such tests is an enriching experience for the student.
• C. Multiple-choice quizzes are the best way to assess understanding of complex topics.
• D. None of the above.

Pencils down.

What follows is the story of how we decided to embrace the implications of the above question for Hidden in Plain Sight and Virginia Studies, two online courses designed for practicing K-12 teachers.^[1] While we make no broad claims about the general utility of multiple-choice quizzes, for our purpose of teaching historical thinking^[2] we concluded that an emphasis on iterative writing exercises better suited our pedagogical goals. The opportunities for course participants to revisit and revise their interpretations over the span of a module and the course as a whole allowed for a stronger focus on the process of historical thinking over rote memorization. Accordingly, we dropped the multiple-choice quizzes. The difficulty and frustration with writing good multiple-choice questions needs little documentation, but participant responses also played a central role in the decision.

Click to open: “Hidden in Plain Sight” course website, Roy Rosenzweig Center for History and New Media.

“Hidden in Plain Sight” course website, Roy Rosenzweig Center for History and New Media.

Several years ago, the Roy Rosenzweig Center for History and New Media at George Mason University, with funding from the Virginia Department of Education, designed Hidden in Plain Sight and Virginia Studies to provide quality online professional development focused specifically on history education. We started by asking ourselves a series of questions:

• What did we want teachers who completed the courses to learn?
• How could we model historical thinking in an online environment?
• How could we make the courses engaging?
• How could we incorporate writing into the process?
• How could we frame the feedback on writing in a way that encouraged teachers to engage deeply with primary sources and historical analysis, as well as with strategies for teaching both in the classroom?
• And perhaps most important, how could we measure progress and assess growth in the ability to think critically about the past and to teach historical thinking in the classroom?

We structured the content and navigation in Hidden in Plain Sight and Virginia Studies with these questions in mind. Each course begins with an introduction to historical thinking followed by a series of content modules. Each module opens with an object, such as an 18th-century homespun coat or a can of instant coffee, presented without label or context. Participants are asked to form a hypothesis based on close observation—taking note of the details of the object—before drawing conclusions. They are also prompted to think about the object in historical context, drawing on their own knowledge to hypothesize how the object fits within American or Virginia history, respectively. Participants then proceed to Resources to learn more about the object and its historical context. Materials in this section include maps, prints, posters, handbills, personal letters, songs, and diary entries, with accompanying text or video providing historical interpretations and models for analyzing sources.

Initially, after exploring Resources, participants completed a multiple-choice quiz to assess comprehension. After the quiz, participants were asked to review their initial hypothesis, which appeared on the screen, and to Rethink their conclusions by writing a revised hypothesis. Participants then applied their newly acquired skills and content in Classroom Connection, where they developed an activity for their own classrooms drawing on the resources of the module and the lessons learned. They could then view and comment on the submissions of others on the final wrap-up page.

We originally included the multiple-choice quiz for several reasons: to assess basic learning, to ensure that course participants read through all of the materials in the module, and to provide immediate feedback at a specific point in the process. We created it with some ambivalence, a feeling that deepened during the development and pilot testing phases as technical, functional, and pedagogical questions arose. Should teachers, for example, be allowed to take the quiz more than once? Should they be allowed to return to Resources while taking the quiz? Doing so could allow for deeper learning and encourage course users to spend more time with the sources. At the same time, it might lead some to skip to the quiz and then look through Resources for specific answers to the questions, allowing the quiz to define the learning experience. Another set of questions loomed even larger: did the quiz questions support the overall objectives of the course? Did they facilitate historical analysis and critical thinking? Did they allow the course instructors to see growth in thinking?

Another drawback of including a quiz emerged while teaching the course. Participants put far more weight on the meaning and value of the quiz than we did as instructors. Despite extensive, personalized feedback on written responses, we received regular emails expressing concern about quiz scores. This was especially notable because most participants took the course for professional development credit and not for graduate credit. One teacher, whose written responses were exemplary, emailed the instructors to reassure us that, “I read the stuff, really,” and to ask “[will] quiz scores affect our recertification?” Another who clearly demonstrated learning throughout the course felt “embarrassed” by less than perfect quiz results.

This concern was by no means unique, as participants communicated similar worries each time the courses were offered. Some asked to retake the quiz. In response to the number and tenor of the requests for retakes, we reformatted the course settings to allow participants to take the quizzes multiple times.  Some students decided to take the quiz repeatedly until they achieved a perfect score. Given that the goal of these courses is not the retention of facts, but the development of historical thinking skills and their application in the classroom, the attention and mental energy that course participants devoted to the multiple choice quizzes was concerning.

In contrast, the writing assignments (Hypothesis, Rethink, and Classroom Connection) proved very successful at engaging participants in historical thinking. Individuals taking the course demonstrated growth, often significant growth, in their ability to engage with primary sources, analyze historical contexts, develop their own interpretations, and integrate strategies for teaching students to do the same. They built on experiences from the first module, improving their ability to hypothesize about the context and meaning of the initial object. Perhaps equally important, the written submissions provided discrete evidence of this growth in thinking, growth that did not necessarily correlate with quiz results. While the emphasis on writing and providing detailed feedback necessitates a considerable workload — two graduate students working twenty hours per week can administer a semester course to 100 participants — we have found it a worthwhile investment in light of the demonstrable results.

The Hypothesis-Rethink-Classroom Connection-Feedback arc of each module provides an opportunity for iterative learning, for course users to rehash and strengthen their written interpretations over the course of the module. We see this combination of writing assignments as central to encouraging historical thinking in an asynchronous learning environment: since teachers taking the course have the chance to revisit, amend, and expand on their hypotheses based on additional information and evidence, much of the pressure to write the one “right” answer is lifted. Growth is encouraged and rewarded. In addition, these educators are engaging in the same exercises that they often pose for their students—and the same processes that historians use regularly.

Hypothesis

The first writing activity in each module involves forming a hypothesis on the historical significance of an everyday object—an important first step in the iterative writing process that is central to these courses. Handmade nails, a suffragette’s dress, and a split-rail fence are a few examples. Participants are prompted to frame their hypotheses in response to two questions: “What do you notice about this object?” and “How might this object connect to broader themes in American (or Virginia) history?” Knowing that some may hesitate to give detailed responses, afraid of not saying the “right thing,” course instructors frequently encourage participants to take risks and use the hypothesis for observing and brainstorming. As one instructor wrote, the hypothesis is “a space to speculate and include your thoughts on the objects and their connection to broader histories—and also where there’s no such thing as a wrong answer.”

In response to an image of a World War II-era tin can, for example, one participant noted its age and appearance and then hypothesized that the can was used by American soldiers during the war.^[3] To answer the question about the object’s broader historical significance, the participant suggested that the can was related to “either the challenges of mobilizing an army and the growing military technology, or just the industrialization of agriculture and modern society.” After submitting the hypothesis, course participants view the module’s primary sources and commentary by historians before revisiting their original hypothesis in a written assignment entitled Rethink.

Rethink

Rethinks are structured writings prompted by two primary questions. The first question asks course participants how the module’s object and resources have influenced their thinking on the historical themes or topic. In the porcelain module in Hidden in Plain Sight, for example, the first question is “How does porcelain connect to broader themes in 18th-century history? Use specific examples.” The second question prompts an analysis of the materials presented in the module: What sources or points of view are missing? What is emphasized (or not)? How does the module connect to prior knowledge of the subject? In the case of the porcelain cup, the second question asks, “What additional information (not included in this module) would you want to know before making an argument about the causes of the American Revolution?”^[4]

A quality Rethink connects the Resources to the relevant themes and historical context, and engages meaningfully with the interpretations presented. For example, one participant of Virginia Studies discussed how the fence module enriched her perspective on the relationship between geography and history:

The fence symbolizes ownership and settlement. The shape and materials show how the settlers tried to control their environment. They used natural resources, wood, to build the fence and by building it they were taking ownership of the land, a concept that was different from the native peoples. The fence is an important part of Virginia’s history. It shows us that when the settlers arrived they were planning on staying here. Building fences began the process of transforming the landscape of Virginia. I thought it was interesting that all the modules were pieces of the Virginia Studies curriculum, which most teachers teach from a viewpoint of history. Yet, each module used geography as the starting point. . . Virginia’s history might be better understood and engaging to students if you use the themes of geography—location, environment, landscape, human-environment as a starting point for instruction.

The Rethink provides a formal opportunity for course participants to revisit their initial thoughts on a historical object, and to synthesize and interpret the arguments and information learned while exploring the module resources. It encourages thoughtful engagement with the material and with historical thinking through a structured writing exercise rather than the rote recounting of historical facts. In feedback, instructors help course users expand on their responses, often by posing questions to prompt more critical analysis and the further development of connections made in their reflections. In response to a thoughtful Rethink on the porcelain module that discussed relationships between the tea cup and 18th-century themes of political representation, revolution, and mercantilism, the instructor encouraged the teacher to explore the consumer and political culture angle, including colonist decisions to purchase (or not purchase) imported goods as an expression of political beliefs and the decisions of American merchants who simultaneously faced public pressure to refuse imports or face lost revenue from boycotts.

Classroom Connection

After submitting the Rethink, course users move to Classroom Connection and are asked to apply what they have learned in the module by creating an activity for students. Unlike the more structured Rethink, Classroom Connection is an unstructured writing activity. The emphasis is on developing an inventive, practical activity based on the module’s content and historical thinking through primary sources. Similar to the previous writing assignment, course instructors provide feedback. In addition, participating teachers also receive feedback in Classroom Connection from each other in the form of comments on activity ideas.

In a module on the Triangle Shirtwaist Fire, for example, one teacher’s Classroom Connection asked students to investigate workplace safety in local businesses. Students compared federal (U.S. Occupational Safety and Health Administration, OSHA) and local regulations with photographs and documents from the 19th-century, before such laws were created. By studying workplace safety and regulation over time, students could examine the Triangle Shirtwaist Fire in its historical context. The teacher posed thoughtful questions for students, including the benefits and drawbacks of extensive safety regulations for businesses and workers. They then investigated the causes and development of specific regulations and policies.

Participating teachers are often highly motivated to incorporate primary sources into their classroom activities but sometimes struggle to connect primary source analysis with the larger historical narrative. The quizzes did not effectively address this gap, whereas the written Classroom Connections in each module raised these issues to the surface and provided the opportunity for thoughtful feedback.

The Virginia Studies module on the colonial period, for example, features a source that many participants incorporate into classroom connections: a letter written by an indentured servant named Richard Frethorne describing the harsh conditions of servitude and labor.^[5] One teacher developed an activity in which students would read this moving letter and then write their own letter from the viewpoint of an indentured servant. Through feedback on Classroom Connection submissions, the instructor encouraged the teacher to incorporate other sources and direct the lesson to a larger historical question, suggesting that the teacher use advertisements designed to lure individuals to the new world. Promises by the Virginia Company could be contrasted with stories of hardship in Virginia, asking students to consider what the colonies might represent to prospective colonists and why they might choose to come despite rumors of hardship from individuals such as Frethorne. This teacher’s writing offered an opportunity to provide specific feedback on how historical evidence can help students understand the choices made by those in the past (e.g., why individuals would choose to become indentured servants in 17th-century Virginia) and address larger historical questions, such as how the Virginia colony grew in population despite a high mortality rate.

Conclusion

The team teaching these courses met regularly during pilot testing and the first iterations of each course. At each meeting, we improved functionality, revised content based on feedback, and adjusted the structure of the course. We compared evaluations, student work, and instructor experiences and discussed strategies for improving the courses for future iterations. The decision to eliminate the quiz evolved out of these conversations and reflections.

The act of writing and, perhaps most importantly, the cycle of Hypothesis-Rethink-Classroom Connection-Feedback, encouraged participants to go beyond descriptive history and to think of new ways to engage students in learning about the past. Through writing, receiving feedback, and revising initial hypotheses, participants made insightful connections between sources and history, showed growth in their understanding of historical thinking, and applied new knowledge to their classrooms in creative ways.

The courses’ approach to writing has also inspired participants to use a similar hypothesis-rethink model using historical objects with their own students. Several participants have asked about the technological aspect with interest in adding more online work to their classrooms. Some have even reported sharing their own work and feedback received in the course with their students to model their own learning process as well as to demonstrate the benefits of iterative writing for learning. Success on the quiz did not demonstrate the same growth, and often did not correlate with a significant development of historical thinking skills across modules.

Removing the quiz from Hidden in Plain Sight and Virginia Studies, however, eliminates the benefits of automatic grading and instant feedback. This is a somewhat daunting prospect for an asynchronous online course where participants can work through modules entirely at their own pace. At the same time, through trial and error, we learned that omitting the quiz allows us to keep the focus of the courses on the original goals—developing and assessing historical thinking—and to focus all of the instructional time on the structured and unstructured writing of participants. Feedback is comprehensive, ongoing, and focused on each individual’s work and progress. This regular engagement with instructors has proven to be a key feature of the course’s success as indicated by course evaluations. We have found it a worthwhile investment in light of the demonstrable results.

Course evaluations have been overwhelmingly positive. Ninety-eight percent of participants either agreed or strongly agreed that they learned history content, teaching strategies, and tools for analyzing and teaching with primary sources while taking the course. One hundred percent reported that the course structure stimulated their thinking, and that they would recommend the course to a colleague. Individuals who completed the courses described them as “thought provoking,” “very user friendly and engaging,” “meaningful,” and “eye opening.” One teacher noted, “This course really helps you understand the STORY behind the standard.” Formal and informal interactions among course instructors and participants throughout each semester via assignment submissions, comments, and emails confirmed the value of writing in the online course. On the same post-course surveys, the feedback on the quiz was far more equivocal, and in fact stood out as the only feature of the course that did not receive uniformly positive reviews.

The structure and emphasis on writing in these courses allows us to assess not only whether participants are retaining content (the original point of the quiz), but also whether they are engaging with that content in meaningful ways. We are exploring ways to apply this focus on deeper understanding and critical learning to other online, asynchronous courses in place of the emphasis on rote learning and discrete facts that plague many of the current open-enrollment, online educational offerings.

From our perspective, the online asynchronous model has much to recommend it. Courses such as Hidden in Plain Sight and Virginia Studies are available to individuals whose location or work-life schedules make attending a traditional in-person course impossible. In addition, these courses have the potential to reach large numbers of students, and demonstrate that writing assignments, both structured and unstructured, can be effectively incorporated into an online learning environment. The completion rates for these courses have been encouraging as well: over 90% of participants who begin the courses go on to complete them. Currently these courses are offered on a semester model, but they could allow for continuous enrollment, significantly increasing the number of participants each year.

It is this potential for online learning—the opportunity to provide meaningful learning experiences while reaching a broad audience—that provides the core focus for our reflections on and modifications to the course. While online learning never has been and never will be a panacea, in this context, Hidden in Plain Sight and Virginia Studies offer exciting possibilities for liberal arts education beyond selecting quiz answer bubbles labeled A, B, C, or D.

About the authors: Celeste Tường Vy Sharpe is a PhD candidate in history and art history at George Mason University. She works as a graduate research assistant at the Roy Rosenzweig Center for History and New Media, and a teaching assistant for the online courses Hidden in Plain Sight and Virginia Studies. Nate Sleeter is a PhD candidate in history at George Mason University and graduate research assistant at the Roy Rosenzweig Center for History and New Media. He works as a teaching assistant for the online courses Virginia Studies and Hidden in Plain Sight. Kelly Schrum is the director of educational projects at the Roy Rosenzweig Center for History and New Media and an associate professor in the Higher Education Program and the Department of History and Art History at George Mason University. She has worked for more than a decade to create innovative, open digital resources and tools for teaching and learning, including Teachinghistory.org and History Matters.

How to cite:

Celeste Tường Vy Sharpe, Nate Sleeter, and Kelly Schrum, “How We Learned to Drop the Quiz: Writing in Online Asynchronous Courses,” in Web Writing: Why and How for Liberal Arts Teaching and Learning, ed. Jack Dougherty and Tennyson O’Donnell (University of Michigan Press/Trinity College ePress edition, 2014), http://epress.trincoll.edu/webwriting/chapter/sharpe-sleeter-schrum.

See an earlier version of this essay with open peer review comments.^[6]