Don’t blame the AI: you’re accountable for your work! “VivaBuddy” aims to help students check they haven’t outsourced too much thinking when they co-author with AI.
Mohsen Ebrahimzadeh is a doctoral researcher in the Transdisciplinary School, supervised by Antonette Shibani and CIC’s Simon Buckingham Shum, supported by CIC Specialist Software Engineer Andrey Inkin.
As we move into an era when undetectable AI is both deepening and undermining thinking and writing, regardless of how much AI may have been used, Mohsen’s PhD asks students: Are you sure you understand your own writing?
This begs the question: How could we assess this, especially at scale with larger cohorts, and about any topic? While we certainly encourage teaching teams to meet with students to verify their progress and authorship, there are clearly limits to what is humanly possible. As students juggle their studies with work and carer responsibilities, across timezones, flexible 24/7 interactive orals are clearly not possible. It’s here that GenAI opens new possibilities.
To explore these, the team is developing a prototype codenamed VivaBuddy, a web app providing a formative assessment and feedback experience (a hybrid, part viva voce and part text comprehension test). It’s powered by UTS’s secure GPT hosted in Microsoft Azure, with a custom user interface developed in CIC. Mohsen’s teaching and assessment design expertise, combined with his newly acquired prompt engineering skills, have enabled him to develop a prototype that generates a range of tailored questions to verify comprehension of any text it is given. These questions are in different formats and of varying difficulty, according to Bloom’s revised taxonomy. Students receive feedback at the end, and are then offered the opportunity to open a conversation if they want.
Within the evidence landscape of assessment validity, Coauthorship Integrity is a proposed new category of validity evidence, with AI vivas such as this a new way to gather evidence of learning, firstly in formative scenarios on draft writing, but potentially leading to summative scenarios.
Pedagogical walkthroughs and interviews with educators and assessment experts suggest this model holds promise for both formative and summative use. This is reported in a preprint which we’re sharing to invite wider feedback, from which the above figures are taken:
Ebrahimzadeh, M., Shibani, A. & Buckingham Shum, S. (2025). Coauthorship Integrity: Reconceptualizing Assessment Validity for the Age of Generative AI. Available as an SSRN Preprint: http://dx.doi.org/10.2139/ssrn.5404123