When many schools halted in-person teaching during the pandemic, they also stopped providing another function that has been perhaps less missed: giving tests under the watchful eyes of proctors.
Millions of college students facing final exams, professionals pursuing new qualifications and others were asked to take important tests at home using programs such as ProctorExam, Proctorio and ProctorU—software designed to fight cheating by getting a human or machine to remotely watch for suspicious behavior in test takers’ faces, rooms and audio levels.
It was a windfall for online proctoring companies, but thrust the pitfalls of the practice into the spotlight.
Being watched by a faceless stranger or artificial intelligence provokes anxiety or worse, according to some students and teachers. Educators and privacy advocates raised concerns about the software’s efficacy, invasiveness and potential to discriminate against some disabled candidates.
Online proctoring companies are now updating their user experiences, partly to address some of the critiques.
“In 2020 we were like a train going 100 miles an hour, and we couldn’t stop it,” said Proctorio Inc. founder and chief executive Mike Olsen, noting that the number of exams proctored by the company in April 2020 rose 900% from a year earlier. “We realized we had to change the wheels, otherwise we were going to derail.”
One criticism leveled at Proctorio, which uses machine-learning technology to monitor a student’s behavior during a test, is that its system sometimes fails to detect the faces of users with darker skin tones, prompting concerns that these students may be unable to begin an exam. Mr. Olsen said the software occasionally fails to pick up students’ faces if they are in badly lighted spaces, but a human member of Proctorio’s support team can assist and admit test takers into an exam if the software has issues detecting their face in the pre-check process.
Still, Proctorio in September hired ethical technology consulting firm Babl AI Inc. to help it identify and address possible sources of bias in the system, Mr. Olsen said. Proctorio plans to release details of its assessment’s findings before the fall 2021 semester, he added.
Some users reported trouble getting digital proctoring software to install or function properly on their devices, often because of technical issues such as an unstable internet connection.
Daniel Haven, the founder of the Netherlands-based Procwise Exam B.V., which does business as ProctorExam, said that while his company’s software works for the “overwhelming majority” of users, the firm is tweaking its platform to be more resilient to internet loss. It is also exploring a product that could run completely offline, Mr. Haven said.
ProctorU Inc., a company that virtually links test takers with human proctors, said it has replaced harsh warning symbols and red lettering that it previously displayed when users’ connections wavered, to avoid implying failure over a problem with technology. The company now uses “a more friendly, conversational tone” and offers practical tips to help test takers reconnect, said Evan Travers, lead user experience architect at ProctorU.
ProctorU also added illustrations and diagrams to its onboarding screens with the aim of making it clearer how to install the software and get started, said the company’s founder and chief strategy officer, Jarrod Morgan, noting that giving anybody too much text to wade through before a test “can amp up agitation if we’re not careful.”
More subtle changes were made to alleviate pretest nerves, too, including cutting the average load time before an exam begins by eight seconds and introducing a feature that lets users test their connection and settings before they begin, Mr. Morgan said.
Proctorio, meanwhile, will later this year roll out a new interface for test-setters that cautions against certain settings unless they are necessary, Mr. Olsen said. A room recording option, for example, may heighten test takers’ anxiety and should be switched on only after candidates have been appropriately prepared for it, he said.
Some critics are not mollified by such changes.
For D’Arcy Norman, manager of learning technologies at the University of Calgary’s Taylor Institute for Teaching and Learning, the concept of remote proctoring automatically frames students as assumed cheaters. “This can’t be resolved through interface tweaks or streamlined installation processes—the problem is the nature of the software, not the design of the interface or user experience,” he said.
Online proctoring companies said they are responding to demand from schools and institutions, which rely on proctoring to protect the reputation of their test-based qualifications.
“The goal of the software isn’t to police or surveil people,” said Proctorio’s Mr. Olsen.
Write to Katie Deighton at [email protected]
Copyright ©2020 Dow Jones & Company, Inc. All Rights Reserved. 87990cbe856818d5eddac44c7b1cdeb8
This post first appeared on wsj.com
