Portland Community College | Portland, Oregon

Sustaining your online course: Do online course materials “expire?”

Posted by Ann Cary

The goal of this blog post is to pose a sequence of three questions with regard to best practices and online teaching:

• Do online course materials have an “expiration date?”
  • If so, how do we know when they’ve “expired?”
    • What do we do about it?

With regard to the first question, I’m going to answer this briefly, and emphatically, with yes. Technology changes. Our student demographic changes. Curriculum changes. We as faculty grow, and although the changes we make to our teaching are often small and incremental, over time they add up to this: we change, too.

To answer the second two questions briefly, I suggest this cycle:

• If something seems “off,” identify what isn’t working. This may be something that’s worked well in the past.
• Make a small change. It’s generally helpful to consult other online colleagues, your faculty mentor, or technology support staff. Asking students can be helpful, too.
• Assess the impact of your change.
• If your change made the improvement you wanted, you’re done (for now). If not, restart the cycle.

To put this cycle into context, I’m going to end by sharing an example from my own course.

The problem:

There were these quizzes I used—10 quizzes with 10 questions each. And each question had 4 similar versions to help alleviate cheating concerns. And each of those had 5 multiple choice answers. That’s 400 questions total, and 2,000 answer blanks. I could go on for a really long time about how much work I put into both creating and revising these quizzes (on multiple occasions), but I’ll spare you the details. Let’s just get to the main point: After investing so much time and energy in these quizzes, I used them well past their expiration date. They were originally a really well thought-out component of my course. Combined with a weekly worksheet, they provided students with a maximum amount of feedback while minimizing their time spent completing assignments. But I had a series of interactions that made me genuinely question whether these quizzes were worth keeping. I realized that very few students still typed their answers (due to evolutions in both technology access and expectations). A colleague looked at my course and had an initial reaction of, “Both a quiz and a worksheet every week? That’s a lot!” A student, coincidentally, said that they wished they could focus more on just the worksheet and that having both made it hard to focus their attention. And finally, an interaction with a student who was retaking the course (to change their B grade to an A grade) made me come to the distinct realization that they were able to answer the multiple choice quiz questions on a specific topic but lacked any conceptual understanding.

That last interaction pushed me over the edge, and I decided to cut them. Entirely. And it was great.

The change:

I eliminated the quizzes, and added 1-2 more questions to each worksheet. The problems I added focused on conceptual understanding. Given that quizzes had taken up about 1 hour each week, I was able to ask either more questions or more challenging questions on the weekly worksheets without increasing student workload. I also focused on helping students connect problems they’d missed with problems from their “practice” homework (a larger set of problems done either from their textbook or online) by writing comments on their worksheets and/or emailing them, a connection that was easier to make without both a quiz and a worksheet.

The assessment:

I deemed this change “great,” and my measurements for this are based on four things:

• Student worksheet submission rates
• The proportion of students who attempted each exam
• Student exam scores
• Performance on specific conceptual topics

Each of these were noticeably improved from years past. But I’m going to disappoint anyone expecting data, and withhold specifics. For one, this wasn’t anywhere near a random sample. And second, there was a giant lurking variable: Our placement test changed shortly after I implemented my change. So strong conclusions are statistically tough. And given how many things constantly change in higher ed, this is often the case. I can say this with certainty though: I addressed notable concerns. Students were asked more conceptual questions more frequently. And student success definitely did not get worse. So no restarting of the cycle is warranted–for now.

Do you have any examples of successful updates/changes you made to your course? Share them below–it’s always great to hear more examples.