This semester (spring, 2013), I integrated my on-campus Duke University
class (which I've taught twice before using a "traditional" lecture
format) with my online class (which I'd taught once before via Coursera MOOC),
both bearing the title "Introduction to Genetics and Evolution." My on-campus class had 453 students, while
the online one peaked at 27,000 enrolled (though MOOC enrollment figures are
misleading). Needless to say, I was more
than slightly nervous about this experiment messing up, given the number of
students who would be affected! My
initial reaction is that the integration (via "flipped classroom")
was a success and thoroughly enjoyable by me (I'll have to wait to see the
formal course evaluations before I know how much most of the students liked it),
but I learned some lessons for future iterations.
FORMAT
Duke students had more expected of them outside of class
than in my prior course iterations.
Prior to every class period, Duke students were expected to watch 3
videos (average ~15 minutes each) in Coursera bearing 80-90% of the primary
"raw information content" that I expected them to learn from the
course. This content is what I delivered
in years past via traditional lecture format in class, but now students had to
set aside ~45 minutes before each class period to get it, perhaps longer if
they needed to replay segments or stop to think.
Following the videos, the students had to take a graded
preclass quiz, formatted similarly to their in-class tests and of comparable
difficulty. The purpose was two-fold:
for them to identify areas of misunderstanding or gaps in their knowledge, and
for me to see what students struggled with from my lectures. The last question of the quiz was a
free-response, "What did you find interesting or confusing?" I reviewed their progress the night before
class, and their answers guided the slides I'd make for the class period.
I set aside the first few minutes of the class period for
students to collaborate in completing the preclass quiz. After that, I spent ~10 minutes going over
areas of misunderstanding based on the preclass quiz performance and free
responses. The bulk of the class period
was devoted to students working in self-selected small groups on ~7 ungraded practice
problems designed to help them integrate and apply the concepts from the
lectures. The teaching assistants (TAs) and I would walk
around the room and engage the students to see what problems they were stuck
on. At the end of the class period, I'd
go over the answers, and then spend ~5-10 minutes presenting a practical or
medical application of the concepts related to the lecture, or recent primary
research building from it (the remaining 10-20% of "information
content"), often building on what students said they found interesting.
Besides that, the rest of the class format was as before--
the students had a weekly problem set to complete, weekly laboratory sections, a
few other online practice problems, and three open-book, in-class tests. The class period activities were also
recorded and made available to any students who couldn't make it due to
illness, travel, etc. (again, just like before).
WHAT WORKED
Most of the class's students who answered the informal mid-semester
and end-semester course surveys indicated they were pleased (and often
pleasantly surprised) with how well the flipped class format worked. Students did not have to initially assimilate
the primary class material at 8:30am as previously, but could work at whatever
time of day was optimal for them. They
could also easily pause at any time, or immediately replay anything they
missed. Most reported that the in-class
reinforcements were helpful for them to really understand the material and to
perform better on assessments (see figures below). Most appreciated the "reactive" and
"interactive" nature of the class, where the class period content
responded directly to their areas of interest or confusion. Some students told me that the format forced them to study as they went along, such that they noticed they barely had to study right before the tests since they'd already done so much along the way. A handful said they liked the online discussion
forums, and found the elaborations by either Duke or online students on some of
the material helpful.
Figures
Important note: these informal evaluations were done online, so they were not biased in being completed by just the subset of students who regularly came to class.
Fig 1: Response to question: Did you find that the in-class presentations and
exercises were useful and reinforced concepts and approaches from the recorded
lectures? (For scale, 1 was never chosen,
and 2 was chosen twice)
1 = NEVER, 2 = RARELY, 3 = SOMETIMES, 4 = OFTEN, 5 = ALWAYS
Fig 2: Response to question: Did you find that the in-class activities provided
you with additional practice that improved your performance in assessments
(e.g., problem sets and tests)? (For scale, 1 was chosen twice, by the same
people who put "2" in previous question.)
1 = NEVER, 2 = RARELY, 3 = SOMETIMES, 4 = OFTEN, 5 = ALWAYS
Even some of the more critical students appreciated the
outcome of the class format-- one person said snappishly, "Of course we
did better on the tests-- we had to work harder." I'll just say "true."
In terms of my feelings, I perceived that the students really
understood and interpreted the material better than in either previous
course iteration. I added more
challenging concepts than I had covered in previous years, and I was impressed
that the students seemed to really understand them. I was also impressed that some areas of
common misunderstanding from previous years just didn't seem to come up. I should emphasize-- these are all my
perceptions, not quantitatively confirmed conclusions with statistical
significance. But, the class did better
on the first test than I've ever seen in my career teaching.
Finally, it was just a lot more fun for me. A traditional class is like a performance--
you get up on stage and do it, with the only feedback being quizzical looks,
laughs, or the very rare hand up for a question. This was truly interactive-- I started the
class period by giving students what they
asked for, and then spent much of the rest of the period in one-on-one or
small group discussions about the material.
I like teaching for the interactive nature of meeting people and
watching them get excited, interested, or at the very least "more knowledgeable"
about a subject, and this format really allowed me to experience those pieces. It also really, really helped me learn what
concepts students understood and what concepts need new forms of presentation
for future class iterations. As my
colleague Dan Gauthier in Physics said about his flipped class, "I've
never felt more attuned to my students' understanding."
WHAT WAS LESS-THAN-IDEAL
Some of the students were unhappy with the added workload of
having to watch ~1.5 hours of video each week.
I personally don't find that unreasonable given the small amount of
other outside-class work that was expected of them-- I am confident many
humanities courses assign more than that in readings each week. Here was perhaps the most critical response I
received on the format: "While the flipped class format benefits many of
my fellow students, this format hinders my learning. In a normal class, I would
go to lecture and take notes and such, and then do whatever work was required
of me, but no more; I normally take a minimalistic learning approach, does as
little as possible to succeed. As such, I do not normally watch the online
lectures because I find that doing so is more time consuming than just glancing
over the lecture slides to find the information needed". Fortunately, this was a minority opinion
(and, in my opinion, one unbefitting of a student at Duke University or any
college/ university).
One negative I did anticipate was that a fraction of the
students fell far behind (especially between test 1 and test 2) and did not
really catch up in a timely manner to do well on the later tests. If one fails to watch the lecture videos before
class, there really is no point to coming to class since the student would be
just lost on the material. At that
point, it can become self-reinforcing-- a student falls behind, and the work
needed to "catch up" is greater than in a traditional class, possibly
making them more likely to further procrastinate and fall further behind.
A possibly related negative I did NOT anticipate was a drop
in class attendance... attendance probably was consistently at or below 50%
after the third week. My previous traditional
classes were all recorded, so it was not uncommon for people to skip and
presumably just watch the recording of my video. We often had 50-70% attendance in the
traditional classes, unlike the <50% with the flipped class. I was surprised that the fraction of the
class that now didn't come to class (one day after a late basketball game dipping to ~20%), even
though it seemed like there was "more" reason to come than before
since the in-class piece was not basic content just-as-easily assimilated by
video. A few students I talked
with cited two reasons 1) the increase in overall amount of outside-class work (the
fact they already had to spend 90+ minutes a week on the material made some students
less likely to spend even more by coming to class), and relatedly, 2) some simply felt confident with the material from the video, and would thus skip
the reinforcement and application in-class.
The negative here is that, for a fraction of these skippers, this confidence was
unfounded-- I warned students that the test 2 material was more challenging
than test 1, and despite my repeated warnings, there was a noticeable drop in attendance
after the high performance on test 1 (and proving my point, a 10-point drop in
average grade).
The one class format element that I think
"flopped" was the few minutes at the beginning of class for
completing the preclass quiz in the classroom-- almost no students ever did
this, and most people just arrived 5 minutes late for class to skip that piece.
OTHER SURPRISES
The biggest surprise to me was the absence of correlation
between class attendance and grade. The
three test grades were very highly correlated with each other despite the
differences in mean, but there was no
clear association between which students came to class and student test
performance. However, upon further
consideration, this result ties in with what I
mentioned above. Some of the students
who felt they had a good handle on the material just from the primary content
delivery in the video recordings didn't come to class (or didn't come
regularly). I think the absence of
correlation suggests that many of them were right-- they did
understand the material well enough from the video lecture that in-class reinforcement may
not have had a noticeable effect on their assessment performance. Similarly, some of the people who came to
class were those who struggled with the material the most-- their coming was a good thing, since they had
more practice and better performance as a result. Some of the students with whom I interacted in the
class had clearly watched the video lectures, and were still struggling to
assimilate the content. As such, my hypothesis is
that the format, even with its sparse attendance, served to homogenize student
performance-- basically "almost everyone" in the class did well in
assessments, either due to intrinsic high ability (or prior background) but perhaps
little or no extra practice in class, or potentially lower ability (or less background) but
with a lot of extra practice in class.
(Obviously, many students who came to class also had high ability and good prior background, too-- I certainly don't mean to imply that only struggling students attended class. Some came regularly just because they were very interested in the material, too. Similarly, I know many students who performed poorly yet rarely or never came to class.)
I recall our forward-thinking university provost chatting
about this a year ago at an informal dinner-- that with flipped classes, "everyone
might do well" in the class. Some of the literature also says this format "levels the playing field" of classes. The grade improvement opens the much stickier question of the purpose of grades (which I shall
pose but not pursue)-- are we truly grading to assess how well students can
understand and apply the material (wherein it's fine for the whole class to
have A's), or are we also trying to
distinguish students with higher intrinsic ability from others? Personally, I have no intention of
"toughening" my class solely to "counter" high student performance on
what I think are perfectly fair and reasonably challenging tests-- doing so directly
violates my philosophy of the purpose of teaching and needlessly alienates students.
(Quick note: my question about "all A's" was rhetorical-- not all my class's students received A's... there were many who did not achieve high grades.)
My other surprise was the general absence of engagement of
the on-campus students in the online Discussion Forums associated with the
online class. Many of my Duke students
said they looked in the forums, but
even though I offered the Duke students extra credit for posting in the forums,
80% of the Duke students never posted
in the forums, and most of those who did post made 5 or fewer short comments or
questions. A handful really engaged with
others in the forums, but <10 students total.
WHAT I MIGHT DO DIFFERENTLY
Despite the absence of a significant association of test grade
with attendance, I know at least some students skipped class to the detriment
of their performance. This was partially
my fault-- in the canonical flipped class format, there are in-class
assessments with stable student groups that are assigned by the instructor,
thus forcing students to attend and be accountable to their classmates and themselves. I deviated from this format because I
anticipated (correctly, I think) that a lot of students would miss classes each
day due to illness, family travel, student activities, and sports. I did not want to be in the position of
having to judge daily whether particular absences were "excused" or
not for tens of students.
What I'm considering next year is to have one group-activity
question graded each class period (or once per week). If there are 14 such activities, perhaps I'd allow
them to keep their highest 6 grades of the 14 only. Hence, one has to show up to the class at
least that small fraction of the time to get the grade. However, I would then not accept any
excuses-- any sports- or activity-travel, or any illnesses, have to come out of the "8 dropped
grades." This may seem like an
exceptionally low bar, but my goal is to get students to come to some classes
such that they have the opportunity to assess for themselves whether coming to
class is helping their understanding and test performance. If the answer is no, then so be it. I just don't want them to "assume"
that coming to class is unnecessary without having experience with trying it. I'm still deciding if this is a change I'll employ or not...
SYNOPSIS
Although requiring a TON of upfront work, this experience
has definitely changed my view of teaching.
As I commented in a prior blog post, I feel now that, as faculty become less "lecturers"
and more "facilitators" in the classroom, they work with the humanity
of students rather than treating students as consumers of prepackaged products. Teaching this class was very, very enjoyable. Further, these
students are interesting and talented people, many of whom are spending tens of
thousands of dollars to be at an elite university, and are receiving some classes in a format of not much greater value than what could be found online. My statement is not to suggest that the total
college experience is what happens in the classroom (clearly untrue),
nor that the students themselves are incapable of obtaining more from traditional lecture classes by taking the initiative to engage professors directly. However, my goal this semester was to add as much value as possible to the in-class
experience by making it visibly dramatically more than what is available freely
online-- far more opportunities for interaction, extensive reinforcement, "going
further" with the material, etc. (in addition to the laboratory sections,
of course). I hope that the vast
majority of the students viewed it as successful and beneficial... I guess I'll
find out, to some extent, when I get the formal evaluations back!
Thoughts and feedback welcome!
LINKS
Article in Duke Today, February 11, 2013
Editorial in Duke Chronicle, February 25, 2013
Article in Duke Chronicle, March 5, 2013
Article in Duke Towerview magazine, March 27, 2013
Student project on flipped class discussing this one in particular, April, 2013
LINKS
Article in Duke Today, February 11, 2013
Editorial in Duke Chronicle, February 25, 2013
Article in Duke Chronicle, March 5, 2013
Article in Duke Towerview magazine, March 27, 2013
Student project on flipped class discussing this one in particular, April, 2013
