How (Not) to Evaluate Teaching During a Pandemic
How (Not) to Evaluate Teaching During a Pandemic
By Jody Greene APRIL 06, 2020
In the initial days of academe’s shift to remote teaching (formerly known as "spring break"), one of the many concerns weighing heavily on the minds of faculty members was how their teaching would be assessed this term. It made my heart, and head, ache to think that instructors — especially at adjunct and tenure-track ranks — worried about that at all in the midst of global catastrophe.
And yet they are worried. As someone with a foot in both the faculty world, as a professor of literature, and in low-level administration, as an associate vice provost, I have heard the concern firsthand from colleagues. How will their teaching this spring be judged and ranked in the months and years to come by tenure-and-promotion and hiring (or rehiring) committees? Will they be punished for the many things that have gone wrong, pedagogically, in what has been an unimaginably difficult and unstable year?
In the chaos of the past month, few institutions have issued clear and compassionate statements about faculty evaluation. At least in part, that’s because communications from the top have largely focused on urgent matters of policy and public health.
We’ve already seen some heated debates on whether to use this crisis to evaluate the effectiveness of online teaching (in summary: now-is-a-good-time versus it-absolutely-is-not). So I hesitate to even waste words on whether, and how, to assess anyone’s job performance during a pandemic.
But it does seem useful to offer some thoughts, suggestions, and reassurances on a more nuanced topic: How might we rethink and remake our practices of "evaluating teaching effectiveness" during the Covid-19 pandemic, and in the years ahead?
What follows are suggestions for: (a) adjunct and tenure-track faculty members on how to record and represent the teaching choices you have made this term, and (b) professors and administrators who will do the "evaluating" of a profession thrown into tumult this spring (and probably, this summer, too). My main advice is twofold:
If we’re going to assess anything about this year’s teaching, let’s limit it to a cautious and compassionate evaluation of what — if anything — we have learned about specific technological tools and flexible teaching practices. What worked in this crisis? What didn’t?
Don’t imagine that we can meaningfully assess the capacities of people rapidly deploying those tools and practices, with minimal support, in conditions unlikely to be repeated (one would hope) in quite this way ever again.
Our usual approach to evaluating teaching focuses on rankings and metrics. I am proposing we shift that approach — in the near and possibly even long-term future — toward narrative and reflection. By shifting our collective focus from an evaluative approach to a documentary one, we might diminish faculty anxiety during circumstances already inclined to provoke enormous insecurity and widespread suffering.
There may well be a secondary benefit to a documentary approach: By encouraging instructors to provide an account of what worked and what didn’t during our emergency remote teaching — and of how they responded to the latter — we might get a much more accurate picture of the utility and value of some of the tools and practices that have been around for a while but that many instructors have hesitated to adopt.
Like everything else this spring, the best way to proceed on this front is to keep things simple. With that goal in mind, here are three suggestions for instructors nervous about how their teaching will be "evaluated" this year:
Keep detailed records. Take the time now to document the changes you have made in your courses in the shift to remote teaching. Chances are you won’t remember all of the changes you made after the fact. This is a traumatic time for everyone, and large-scale upheaval messes with our memory. Keep a copy of the syllabus you originally planned to teach on your desk, or what passes for your desk these days, and annotate it with all the changes you made in how you delivered your course, how you communicated with students, how you rearranged assessments, and so on.
The more detail you can record, the more you’ll be able to recall later, when you tell the story of the effort, responsiveness, and creativity you put into your teaching during the Covid-19 pandemic. If you were able to be a resource to other instructors — mentoring colleagues or guiding teaching assistants — during this time, keep track of that and be sure to include it in your narrative.
Check in with your students — a lot. Standard evaluation forms are unlikely to yield high return rates and may contain a lot of feedback not specific to the instructor, and yet it has never been more important to find out how your students are responding to your teaching choices. Regular check-ins will allow you to course-correct, but they will also provide valuable evidence when you come to tell the story of your teaching during Covid-19 in future reviews.
Whether you use polls, exit tickets, email, or message boards, find out how the changes you’ve made in the course are working for students, and, where appropriate, make further changes accordingly. Be clear and transparent about your learning goals, and in addition to keeping an eye on what their tests and assignments tell you, ask students how they are doing in accomplishing those goals. Take their concerns seriously, and respond or, where necessary, make adjustments where you can.
Imagine that "responsiveness" is the highest pedagogical good under the present circumstances, and make it a priority.
Don’t be afraid to course-correct frequently. And view those course corrections as evidence of your teaching effectiveness.
Changing your approach when something isn’t working for students isn’t a sign of failure — it’s a sign of professionalism and expertise. If you chose primarily synchronous instruction and students turned out to be unable to meet the class in real time, modifying your plan to make room for asynchronous options is evidence of teaching effectiveness, and you should report it as such. If the workload you’ve assigned is too much for students trying to work from home, with other largely unanticipated responsibilities and concerns to attend to, dropping assignments is not a cop-out. It’s a sign of responsive teaching. And that’s how you should present it on your CV.
Fundamentally, for instructors, be prepared to tell the story of this semester:
The attention, reflection, and care you brought to both your teaching and your students during this time.
The help you received and the help you gave.
The things you learned that will change your pedagogy.
Now, for those administrators and professors who will — at some point — be evaluating a junior hire’s teaching effectiveness during the pandemic, here are three suggestions:
Be compassionate but, mostly, be amazed. Recognize and reward the labor put in by every instructor at every level who had to respond in next to no time to the demands of emergency remote instruction.
The vast majority of our institutions were, and are, pathetically underresourced when it comes to the kind of instructional support necessary to meaningfully accomplish a shift to remote instruction in a matter of days. At my own institution, the ratio of instructional-support personnel to faculty members is 1:200 — and that’s a generous approximation.
That nearly all instructors managed to make this shift — to learn new technologies, redesign courses, write new assessments, procure equipment (like home-document cameras and tablets), connect with colleagues with similar instructional challenges, and just plain finish the term (and at universities, like mine, on the quarter system, start another one), while also dealing with the stresses of their own life, health, and family under Covid-19 — is nothing short of miraculous.
Pause long enough to notice the scale and agility of that pivot. What we have just accomplished, and are still accomplishing, is a magnificent achievement that should give you enormous faith in your faculty members.
Put student complaints about a course in context. Students and their families are understandably frustrated, frightened, and unsure about the return on investment they will receive for staying in college right now. As is always the case, but never more so than now, recognize that when students give harsh feedback on evaluations or surveys, that may have a great deal to do with context — in this case, the overall upheaval of 2020 — and not much to do with the instructor in question.
There may be good and useful information from student evaluations of teaching. But you must anticipate that this round of course evals and surveys may be the only time an institution actually reaches out to students directly to ask about their educational experience this winter/spring. Students may use that invitation to articulate a range of frustrations that are not specific to the course instructor.
Be more interested in learning than in grading. I don’t mean what the students are learning; I mean what the institution as a whole can learn during this time about pedagogical innovation and transformation.
Many of us have been wondering for a long time about the scalability and effectiveness of using "universal design for learning," of hybrid courses (i.e., a mix of in-person and online instruction), of assigning multiple low-stakes assessments, of adaptive learning, of ungrading, of student-centered pedagogy, and so on. It has been hard to gather broad data about those techniques largely because their use has varied so much from instructor to instructor on any single campus.
Why not devote our evaluative resources in this time to gathering information about those collective innovations, and particularly about what works for students, what is viable for instructors, and where those two overlap? Rather than grading instructors along a static continuum of individual pedagogical effectiveness, now might be the moment to recognize that a great disruption has occurred in how we deliver instruction, and that disruption may or may not give us useful information about long-term prospects for how we "do" teaching and learning in higher education. Curiosity rather than critique might be the most appropriate, and informative, response.
Ultimately, my hopes for a shift in how we evaluate teaching go beyond the immediate moment. I hope this crisis prompts academe to adopt and nurture a developmental evaluation system — one in which we assume faculty members have something to learn about teaching, and are encouraged to make changes, assess, and improve as teachers throughout their careers.
For better or worse, that is exactly what’s happening at colleges and universities all over the world right now, on a mass scale, possibly for the first time in the history of higher-education pedagogy. The vast majority of us — albeit under extreme pressure — are clarifying what we most want our students to learn, making small and large changes in our teaching, communicating with students more than ever before to find out what is and isn’t working for them, and making further adjustments based on what we hear from them.
To put this another way, we’re in the midst of a global teaching experiment — nothing less than an improvised emergency response. Let’s acknowledge that assessing "teaching effectiveness" amid that crisis is somewhat beside the point. Let’s figure out, instead, what institutions as well as individuals can learn from this extraordinary outpouring of ingenuity, risk taking, adaptation, creativity, and love.
Jody Greene is a professor of literature, associate vice provost for teaching and learning, and director of a teaching center at the University of California at Santa Cruz.