Flipping with a MOOC-- A very new approach to teaching for me

This semester (spring, 2013), I integrated my on-campus Duke University class (which I've taught twice before using a "traditional" lecture format) with my online class (which I'd taught once before via Coursera MOOC), both bearing the title "Introduction to Genetics and Evolution."  My on-campus class had 453 students, while the online one peaked at 27,000 enrolled (though MOOC enrollment figures are misleading).  Needless to say, I was more than slightly nervous about this experiment messing up, given the number of students who would be affected!  My initial reaction is that the integration (via "flipped classroom") was a success and thoroughly enjoyable by me (I'll have to wait to see the formal course evaluations before I know how much most of the students liked it), but I learned some lessons for future iterations.

FORMAT

Duke students had more expected of them outside of class than in my prior course iterations.  Prior to every class period, Duke students were expected to watch 3 videos (average ~15 minutes each) in Coursera bearing 80-90% of the primary "raw information content" that I expected them to learn from the course.  This content is what I delivered in years past via traditional lecture format in class, but now students had to set aside ~45 minutes before each class period to get it, perhaps longer if they needed to replay segments or stop to think. 

Following the videos, the students had to take a graded preclass quiz, formatted similarly to their in-class tests and of comparable difficulty.  The purpose was two-fold: for them to identify areas of misunderstanding or gaps in their knowledge, and for me to see what students struggled with from my lectures.  The last question of the quiz was a free-response, "What did you find interesting or confusing?"  I reviewed their progress the night before class, and their answers guided the slides I'd make for the class period.

I set aside the first few minutes of the class period for students to collaborate in completing the preclass quiz.  After that, I spent ~10 minutes going over areas of misunderstanding based on the preclass quiz performance and free responses.  The bulk of the class period was devoted to students working in self-selected small groups on ~7 ungraded practice problems designed to help them integrate and apply the concepts from the lectures.  The teaching assistants (TAs) and I would walk around the room and engage the students to see what problems they were stuck on.  At the end of the class period, I'd go over the answers, and then spend ~5-10 minutes presenting a practical or medical application of the concepts related to the lecture, or recent primary research building from it (the remaining 10-20% of "information content"), often building on what students said they found interesting.

Besides that, the rest of the class format was as before-- the students had a weekly problem set to complete, weekly laboratory sections, a few other online practice problems, and three open-book, in-class tests.  The class period activities were also recorded and made available to any students who couldn't make it due to illness, travel, etc. (again, just like before).

WHAT WORKED

Most of the class's students who answered the informal mid-semester and end-semester course surveys indicated they were pleased (and often pleasantly surprised) with how well the flipped class format worked.  Students did not have to initially assimilate the primary class material at 8:30am as previously, but could work at whatever time of day was optimal for them.  They could also easily pause at any time, or immediately replay anything they missed.  Most reported that the in-class reinforcements were helpful for them to really understand the material and to perform better on assessments (see figures below).  Most appreciated the "reactive" and "interactive" nature of the class, where the class period content responded directly to their areas of interest or confusion.  Some students told me that the format forced them to study as they went along, such that they noticed they barely had to study right before the tests since they'd already done so much along the way.  A handful said they liked the online discussion forums, and found the elaborations by either Duke or online students on some of the material helpful.

Figures

Important note: these informal evaluations were done online, so they were not biased in being completed by just the subset of students who regularly came to class. 

Fig 1: Response to question: Did you find that the in-class presentations and exercises were useful and reinforced concepts and approaches from the recorded lectures?  (For scale, 1 was never chosen, and 2 was chosen twice)
1 = NEVER, 2 = RARELY, 3 = SOMETIMES, 4 = OFTEN, 5 = ALWAYS

Fig 2: Response to question: Did you find that the in-class activities provided you with additional practice that improved your performance in assessments (e.g., problem sets and tests)? (For scale, 1 was chosen twice, by the same people who put "2" in previous question.)
1 = NEVER, 2 = RARELY, 3 = SOMETIMES, 4 = OFTEN, 5 = ALWAYS

Even some of the more critical students appreciated the outcome of the class format-- one person said snappishly, "Of course we did better on the tests-- we had to work harder."  I'll just say "true."

In terms of my feelings, I perceived that the students really understood and interpreted the material better than in either previous course iteration.  I added more challenging concepts than I had covered in previous years, and I was impressed that the students seemed to really understand them.  I was also impressed that some areas of common misunderstanding from previous years just didn't seem to come up.  I should emphasize-- these are all my perceptions, not quantitatively confirmed conclusions with statistical significance.  But, the class did better on the first test than I've ever seen in my career teaching.

Finally, it was just a lot more fun for me.  A traditional class is like a performance-- you get up on stage and do it, with the only feedback being quizzical looks, laughs, or the very rare hand up for a question.  This was truly interactive-- I started the class period by giving students what they asked for, and then spent much of the rest of the period in one-on-one or small group discussions about the material.  I like teaching for the interactive nature of meeting people and watching them get excited, interested, or at the very least "more knowledgeable" about a subject, and this format really allowed me to experience those pieces.  It also really, really helped me learn what concepts students understood and what concepts need new forms of presentation for future class iterations.  As my colleague Dan Gauthier in Physics said about his flipped class, "I've never felt more attuned to my students' understanding."

WHAT WAS LESS-THAN-IDEAL

Some of the students were unhappy with the added workload of having to watch ~1.5 hours of video each week.  I personally don't find that unreasonable given the small amount of other outside-class work that was expected of them-- I am confident many humanities courses assign more than that in readings each week.  Here was perhaps the most critical response I received on the format: "While the flipped class format benefits many of my fellow students, this format hinders my learning. In a normal class, I would go to lecture and take notes and such, and then do whatever work was required of me, but no more; I normally take a minimalistic learning approach, does as little as possible to succeed. As such, I do not normally watch the online lectures because I find that doing so is more time consuming than just glancing over the lecture slides to find the information needed".  Fortunately, this was a minority opinion (and, in my opinion, one unbefitting of a student at Duke University or any college/ university).

One negative I did anticipate was that a fraction of the students fell far behind (especially between test 1 and test 2) and did not really catch up in a timely manner to do well on the later tests.  If one fails to watch the lecture videos before class, there really is no point to coming to class since the student would be just lost on the material.  At that point, it can become self-reinforcing-- a student falls behind, and the work needed to "catch up" is greater than in a traditional class, possibly making them more likely to further procrastinate and fall further behind.

A possibly related negative I did NOT anticipate was a drop in class attendance... attendance probably was consistently at or below 50% after the third week.  My previous traditional classes were all recorded, so it was not uncommon for people to skip and presumably just watch the recording of my video.  We often had 50-70% attendance in the traditional classes, unlike the <50% with the flipped class.  I was surprised that the fraction of the class that now didn't come to class (one day after a late basketball game dipping to ~20%), even though it seemed like there was "more" reason to come than before since the in-class piece was not basic content just-as-easily assimilated by video.  A few students I talked with cited two reasons 1) the increase in overall amount of outside-class work (the fact they already had to spend 90+ minutes a week on the material made some students less likely to spend even more by coming to class), and relatedly, 2) some simply felt confident with the material from the video, and would thus skip the reinforcement and application in-class.  The negative here is that, for a fraction of these skippers, this confidence was unfounded-- I warned students that the test 2 material was more challenging than test 1, and despite my repeated warnings, there was a noticeable drop in attendance after the high performance on test 1 (and proving my point, a 10-point drop in average grade).

The one class format element that I think "flopped" was the few minutes at the beginning of class for completing the preclass quiz in the classroom-- almost no students ever did this, and most people just arrived 5 minutes late for class to skip that piece.

OTHER SURPRISES

The biggest surprise to me was the absence of correlation between class attendance and grade.  The three test grades were very highly correlated with each other despite the differences in mean, but there was no clear association between which students came to class and student test performance.  However, upon further consideration, this result ties in with what I mentioned above.  Some of the students who felt they had a good handle on the material just from the primary content delivery in the video recordings didn't come to class (or didn't come regularly).  I think the absence of correlation suggests that many of them were right-- they did understand the material well enough from the video lecture that in-class reinforcement may not have had a noticeable effect on their assessment performance.  Similarly, some of the people who came to class were those who struggled with the material the most-- their coming was a good thing, since they had more practice and better performance as a result.  Some of the students with whom I interacted in the class had clearly watched the video lectures, and were still struggling to assimilate the content.  As such, my hypothesis is that the format, even with its sparse attendance, served to homogenize student performance-- basically "almost everyone" in the class did well in assessments, either due to intrinsic high ability (or prior background) but perhaps little or no extra practice in class, or potentially lower ability (or less background) but with a lot of extra practice in class.

(Obviously, many students who came to class also had high ability and good prior background, too-- I certainly don't mean to imply that only struggling students attended class.  Some came regularly just because they were very interested in the material, too.  Similarly, I know many students who performed poorly yet rarely or never came to class.)

I recall our forward-thinking university provost chatting about this a year ago at an informal dinner-- that with flipped classes, "everyone might do well" in the class.  Some of the literature also says this format "levels the playing field" of classes.  The grade improvement opens the much stickier question of the purpose of grades (which I shall pose but not pursue)-- are we truly grading to assess how well students can understand and apply the material (wherein it's fine for the whole class to have A's), or are we also trying to distinguish students with higher intrinsic ability from others?  Personally, I have no intention of "toughening" my class solely to "counter" high student performance on what I think are perfectly fair and reasonably challenging tests-- doing so directly violates my philosophy of the purpose of teaching and needlessly alienates students.

(Quick note: my question about "all A's" was rhetorical-- not all my class's students received A's... there were many who did not achieve high grades.)

My other surprise was the general absence of engagement of the on-campus students in the online Discussion Forums associated with the online class.  Many of my Duke students said they looked in the forums, but even though I offered the Duke students extra credit for posting in the forums, 80% of the Duke students never posted in the forums, and most of those who did post made 5 or fewer short comments or questions.  A handful really engaged with others in the forums, but <10 students total.

WHAT I MIGHT DO DIFFERENTLY

Despite the absence of a significant association of test grade with attendance, I know at least some students skipped class to the detriment of their performance.  This was partially my fault-- in the canonical flipped class format, there are in-class assessments with stable student groups that are assigned by the instructor, thus forcing students to attend and be accountable to their classmates and themselves.  I deviated from this format because I anticipated (correctly, I think) that a lot of students would miss classes each day due to illness, family travel, student activities, and sports.  I did not want to be in the position of having to judge daily whether particular absences were "excused" or not for tens of students.

What I'm considering next year is to have one group-activity question graded each class period (or once per week).  If there are 14 such activities, perhaps I'd allow them to keep their highest 6 grades of the 14 only.  Hence, one has to show up to the class at least that small fraction of the time to get the grade.  However, I would then not accept any excuses-- any sports- or activity-travel, or any illnesses, have to come out of the "8 dropped grades."  This may seem like an exceptionally low bar, but my goal is to get students to come to some classes such that they have the opportunity to assess for themselves whether coming to class is helping their understanding and test performance.  If the answer is no, then so be it.  I just don't want them to "assume" that coming to class is unnecessary without having experience with trying it.  I'm still deciding if this is a change I'll employ or not...

SYNOPSIS

Although requiring a TON of upfront work, this experience has definitely changed my view of teaching.  As I commented in a prior blog post, I feel now that, as faculty become less "lecturers" and more "facilitators" in the classroom, they work with the humanity of students rather than treating students as consumers of prepackaged products.  Teaching this class was very, very enjoyable.  Further, these students are interesting and talented people, many of whom are spending tens of thousands of dollars to be at an elite university, and are receiving some classes in a format of not much greater value than what could be found online.  My statement is not to suggest that the total college experience is what happens in the classroom (clearly untrue), nor that the students themselves are incapable of obtaining more from traditional lecture classes by taking the initiative to engage professors directly.  However, my goal this semester was to add as much value as possible to the in-class experience by making it visibly dramatically more than what is available freely online-- far more opportunities for interaction, extensive reinforcement, "going further" with the material, etc. (in addition to the laboratory sections, of course).  I hope that the vast majority of the students viewed it as successful and beneficial... I guess I'll find out, to some extent, when I get the formal evaluations back!

Thoughts and feedback welcome!

LINKS
Article in Duke Today, February 11, 2013
Editorial in Duke Chronicle, February 25, 2013
Article in Duke Chronicle, March 5, 2013
Article in Duke Towerview magazine, March 27, 2013
Student project on flipped class discussing this one in particular, April, 2013