Adaptability, data and digital poverty: digital assessment at Brunel University

Brunel University London is a multidisciplinary research-intensive university, and with over 17,000 students and 2,500 staff from all over the world, it is one of the most diverse institutions in the UK. The past two years have been 'interesting'; like others, Brunel had to act quickly to mitigate the impact of the Covid-19 pandemic and national lockdowns on learning, teaching and assessment, and ensure that students were able to continue or complete their studies as planned.

Achieving this was made easier by already having a digital assessment platform; Brunel has been using WISEflow since 2015/16 across various departments. However, it was still a big task to move almost all exams online in spring 2020.

Initial response to lockdown

Although Brunel had been using its chosen digital assessment platform since 2016, only a minority of students had experienced a digital exam. However, all staff and students were familiar with WISEflow from their assignment submissions. This enabled Brunel to shift to remote open-book exams when the pandemic struck in 2020. Question types were changed, and Brunel ran exams of similar duration to the planned pen-and-paper exams to mitigate students’ access to other resources. Students were also given the option to defer their exams to later in the year, no questions asked.

Concerns were raised regarding availability of equipment, namely laptops, wifi and quiet spaces. As Professor Mariann Rand-Weaver, vice-provost education, notes:

"It became clear we had to do more than transferring exams online.

"We advertised a laptop loan scheme, to try and ease the disadvantaging of students across the University; however, we thought that more was necessary to safeguard students.

"We therefore took the decision to allow students to defer exams, without the need for approval. This gave students flexibility and reduced the burden of the impending exams amid constantly changing and challenging external circumstances."

Using data to monitor exams and understand student engagement

During the Spring 2020 exam period, every exam was closely monitored to mitigate against technical issues, such as students missing the upload cut-off time, and to ensure that students didn’t gain an unfair advantage.

"By monitoring every exam, we knew that only about 10% of submissions were late, mostly submitted within a couple of minutes of the deadline and therefore not of concern. Students who submitted later were followed up using the WISEflow exam log to understand whether their claims of technical problems were real. The pragmatic approach that Brunel took served us well, and being able to use the digital footprint has been key to ensuring that no one was taking advantage of the system."

Brunel was able to track daily exam engagement and flag particular patterns by different student groups.

As Robyn Fitzharris, executive officer to the vice-provost, explains:

"Combining this with wider student information, we compared those who took exams with those who didn't and asked whether they were unable to access assessments (digital poverty?), and if we needed to be proactive in reaching out to support them."

Data analysis showed that ~87% of students were able to access and successfully complete their exams from home in May 2020. Fitzharris goes on:

"However, engagement data by student characteristics highlighted that disabled students, mature students and those from deprived areas were less likely to sit exams in the Spring term. We proactively reached out to these students, which helped inform our additional support put in place for the August (and subsequent) exam period such as bookable quiet spaces, increases to hardship funds and more loanable laptops."

Further analysis of degree qualifications showed only small fluctuations in award outcomes for these groups, and Brunel will continue to monitor engagement data to ensure assessment methods do not inherently disadvantage particular groups of students.

Looking ahead

Rand-Weaver considers Brunel fortunate for having a digital assessment platform in place when the pandemic hit, as it allowed the university to continue assessing students and monitor what was happening in near real-time.

The enforced remote, open-book exam format also offered some staff the opportunity to rethink assessment to make it more authentic. For example, the traditional three-hour essay used in Politics and History has been replaced by an extended exam that allows more expansive questions to be set – a format that’s closer to a real-life experience.

"And students are positive too - they take digital assessments in their stride and reported enjoying working on their laptops – essentially, what’s the big deal?"

According to Rand-Weaver, the institution has shifted:

"The exams in 2020/2021 were almost non-events. The majority of disciplines are using WISEflow, remotely or on campus. Credit should go to our staff who have risen to the challenge!"

Being able to use data to identify issues related to exams has made clear a need to think more broadly.

"The data helped Brunel to think about what kind of support is needed for students in regard to all teaching and learning, not just specific assessments. The experience has really shone a light in that sense – we must always be thinking about how assessment can be better and equitable for all our students."

Find out more

• Read our guide on the principles of good assessment and feedback