Opinion: Measuring teaching quality online

Wednesday, 7 August, 2013

Massive open online courses will improve university teaching by revealing how well students engage and by allowing lecturers to assess different approaches, writes Professor Merlin Crossley.

If you think back, your worst university experience may have been the quality of your lecturers. Conversely, the best thing may have been the few, particularly inspirational lecturers.

Many have written about how massive open online courses and online learning will change the world. But not much attention has been paid to how much harder it will be to hide poor lecturing with electronic learning. Online learning provides extensive metrics that indicate how well and how often students engage. It also provides a readily available platform for students to provide detailed feedback on their experiences and to send their comments far and wide.

There is an old saying - if you can't measure it, you can't manage it. Measuring teaching quality and managing it has proved to be very hard. Partly as a consequence of this, many managers have chosen not to measure teaching but to rely on other proxies instead.


The scholarship of teaching - that is, research into teaching and learning - also has been used extensively as a proxy for quality teaching, especially in deciding awards for teaching excellence. Sometimes this works well since some of the best teachers are pioneers in research into teaching and learning, but others are not. Some research into teaching is high impact and informs practice but much isn't. In some cases the results, which are never implemented, may end up existing in a parallel educationalist universe. So using scholarship of teaching as a proxy for teaching quality is risky.

As online teaching spreads, student feedback increasingly will replace the proxies as a readily available indicator of valued teachers.

There is another, important aspect: suddenly it has become possible to run experiments to test the effectiveness of teaching.

Historically it has been very difficult, even for dedicated lecturers, to really know whether what they are doing is effective. It is impossible to run a controlled experiment because there is no negative control class available, the quality of new students varies each year and the feedback can be ambiguous.

With large online courses, particularly free, open-access courses, a lecturer can run properly controlled experiments with statistically meaningful numbers, in very short periods.

Even in a single class it is possible to see which assessment task - and even which lectures or short videos - have been problematic, from the regions of a video that students have rewound or where they have disengaged.

Experienced lecturers who no longer remember what it was like to learn difficult concepts for the first time can appreciate what aspects are most challenging and pay special attention to them.

Academics who care deeply about their teaching (and there are many) will have ready access to metrics that will reflect how their teaching is perceived and how effective it is; and they will be able to assess the effectiveness of the innovations they incorporate into teaching.

It may take us some time to work out how to distinguish popularity ploys and attractive stunts from effective teaching.

But there is no doubt the online surveillance of teaching in the global educational environment is likely to be good for students and for university staff who care about, and are proud of, their teaching.

Merlin Crossley is dean of science at University of NSW.

This opinion piece appeared first in The Australian newspaper.