It’s hard to cheat an IOA with AI in person  

The IOA is a conversation between an assessor and a student framed in a real-world scenario.  The face-to-face IOA on campus is, at the time of writing, an assured way of assessing learning – what we call “Secure” (Lane 1) Assessment at the University of Sydney. This means that the F2F IOA, like the on-campus exam, is one of the few types of assessment that a program of study will be required to have to meet TEQSA’s guidelines on assessment in the AI Age. It is predicted to become a popular way of assuring learning as AI pervades all aspects of society, work and education. 

The IOA drives deep learning

In our year-long investigation of IOA in 2024, we confirmed what was already evident in the literature; that having an IOA promoted a different kind of learning for students. We spoke to students who had experienced IOAs and discovered that they liked the way the IOA forced them to study in a different and ‘better’ way than if they had exams. Students indicated that, as they knew they would have to talk to someone, they would really have to know the subject well.

It’s enjoyable for the student – and the marker

Also, in line with the IOA literature, we found that IOAs were an enjoyable experience for students and assessors. Students find the IOA motivating to prepare for, as they can see the relevance of this type of ‘rehearsal’ for their future careers. One student spoke of getting dressed in a suit and shaking hands to greet the assessor, playing the CEO of a company. Students clearly saw the value in this kind of role-based conversation where they get to practise skills they will need to find jobs or be successful in their careers. Assessors too, though noting that IOAs required them to be fully ‘there’, also spoke positively that they were enjoyable to conduct because each IOA was different.

It’s a valid way to assess learning

Oral assessment has been around for centuries, dating back to the Middle Ages, and has proven to be a valid way to assess learning. The course coordinators in our trial found that the time assigned to the IOA (around 15 minutes) was enough to form a firm judgement on how well a student performed in the conversation against a marking rubric designed specifically for the IOA conversation.

For a more detailed scholarly rationale for the IOA, visit the CLaS Design Patterns website. 

---