2. Designing the IOA
We have found that the Interactive Oral Assessment (IOA) delivers positive learning benefits for students. Students told us that they studied in a deeper way for an IOA than an exam, as they knew they would have to explain themselves verbally. So, we found that the pedagogical justification for using an IOA was strong as it had a positive impact on learning. Earlier in this website, we explained the importance of the alignment of the IOA to the unit learning outcomes. On this page, we will share one approach to designing an aligned IOA that elicits conversation that evidences their learning.
Creating the marking rubric
One approach to designing the IOA is to begin by developing the marking rubric. This is the instrument that enables you to assess students on-the-fly (Norman et al., 2024). The process for developing a marking rubric takes some careful consideration so leave ample time for this in your IOA development timeline. It’s best to have the rubric finalised early in the teaching period so that students can use it to assess a mock IOA before they sit the real thing. This will help reduce their anxiety about doing a ‘new’ type of assessment and help them prepare.
Unlike marking rubrics for other assessment types, the IOA marking rubric serves another important purpose – it is used by the assessor as a scaffold for the conversation so that the conversational experience is consistent for all students.
Following is a suggested four-step method for developing an IOA marking rubric.
Step 1: Identify the phases of the conversation
The first step involves imagining how the conversation will flow and how you will elicit the required performance from the students. Think about the phases of the conversation from greeting to goodbye and write down some possible prompts for each phase. Then work out an assessment criterion for the key phases of the conversation. See below for an example.
Note. Throughout the design process and especially in this step, be mindful of the learning outcomes. Ensure that the prompts elicit evidence of learning outcome achievement.
Step 2: Refine the Assessment Criteria
After the assessment criteria have been drafted, refine the details of the performance you want to elicit from students. For example, for the IOA shown above, these are the seven criteria, each with a weighting of 10 marks:
You will be assessed on your ability to:
- Present a clear, fit-for-purpose and well-argued response to the project brief. (10)
- Clearly explain your processes and evidence behind your advice. (10)
- Speculate on possible and probable impacts of advice on other aspects of business. (10)
- Deliver a response that is on-topic and considered when given new information (10)
- Justify and sell your recommendations with a clear articulation of the benefits and risks. (10)
- Ask insightful questions that reveal considerable understanding of the company, its position operations and challenges. (10)
- Behave in a friendly, professional and balanced way comfortable with challenge (10)
Step 3: Finalise the Marking Rubric
Next, write the descriptors for each criterion. What exactly will you be observing for each dimension of performance? In the example below, there are three levels of performance (Excellent – Satisfactory – Poor). At each level there is range that equates to the typical Fail-HD grading scale with Poor = Fail, Satisfactory = Pass/Credit and Excellent = Distinction/HD.
Involving students in finalising the marking rubric
Later in this series, we describe how to prepare students for the IOA by setting up a classroom activity in which students assess a mock IOA using the marking rubric. This activity could involve students finalising the rubric. This will deepen their understanding of what quality means in professional conversations and provide them with insights on how to perform to the best of their ability in the IOA.
Minimising cognitive load
The IOA marking rubric shown above deviates from the five-level marking rubric (Fail-Pass-Credit-Distinction-HD) that is commonly found in higher education. We found in our trial that the cognitive load on the assessor is quite high in marking IOAs as the assessor needs to facilitate the conversation, regulate their own affect, physicality and cognition as well as make judgements on student performance. There is no consensus in the literature about what a ‘rubric’ is (Dawson, 2017) and how many levels a marking rubric should have (Mueller, 2008).
So, a simple marking rubric to accommodate the reduced available cognitive load for IOA assessors makes sense.
In one of the units in our trial, the IOA marking rubric was set at discrete numbers for each level of performance, so this meant the assessor just had to click on a box to make a quantitative judgement, thus reducing cognitive load significantly.
Step 4: Implement the marking rubric
We recommend that the rubric be implemented digitally in your learning management system, so marks and feedback can be added for each student in the marking time allotted for each IOA session. See the screenshot below of how the first five criteria displayed in the Canvas LMS.
IO Assessment 