Despite FAS’ reliance on the Q Guide in assessing the quality of teaching in Harvard’s classrooms, many administrators and professors in FAS said that the goals of student evaluations are not in line with the school’s recent and increased commitment to improving the quality of undergraduate pedagogy.

“It’s fairly clear that asking people right at the end of a class, ‘How do you feel about this class,’ doesn’t necessarily answer the questions we are asking: have you learned this well? Has this changed your life?” chemistry department Chair Eric N. Jacobsen said.

In fact, professors said, the current system that students use to rate their instructors and courses falls far short of an objective analysis of teaching quality.

“What if a student is lazy, and the professor goes down on them hard? The student might take out their resentment on the Q Guide,” Asani said.

More significantly, the importance of the Q Guide could skew the way professors approach their teaching.

“For graduate students and junior faculty, for whom these numbers may matter a great deal, it has to unconsciously have an impact on how they grade,” Harris said. “You’re more likely to give someone a 4 or 5, if they’ve given you an A.”

For this reason, some faculty are encouraging a rigorous reevaluation of the system Harvard employs to assess teaching quality—and are questioning the current method, which they said purely evaluates student satisfaction.

“Faculty are just worried about the Q Guide score,” Asani said. “If that becomes the driving force behind teaching, then good teaching has been left behind because then people are teaching to get a better score.”

EVALUATING THE EVALUATION

Finding methods to correlate student evaluations with student learning has yielded abundant scientific literature. But the jury is still out on whether a student’s satisfaction with a course indicates the teacher’s overall effectiveness.

Richard J. Light, a professor at the Graduate School of Education, cited a host of research studies that showed significant correlations between a class’s average test score at the end of the course and the class’s average ratings of the teacher and course overall, suggesting that the Q guide produces some

 

positive feedback on teaching effectiveness from its polling of s

tudent satisfaction.

“The Q Guide is not perfect—it can be improved—but these correlations are nowhere near 0, so it’s worth something,” Light said.

In a separate 2010 paper, researchers found that of students randomly assigned to professors teaching identical calculus courses, those who performed better in the class gave their professors the highest ratings at the end of the semester. But the study also found that higher student ratings were negatively correlated with “deep learning”—the ability to apply material taught in one class to more advanced classes. In other words, the people who rated their classes higher in student satisfaction tended to learn less effectively. In fact, these students sometimes performed less effectively in subsequent classes.

“It’s unknown what precisely [the Q Guide] is measuring,” Dean of Undergraduate Education Jay M. Harris said.