Home / Tools / GenAI Apps & Analytics

GenAI Apps & Analytics

In CIC we have established the power of human-centred design approaches to Learning Analytics and GenAI, through working in close partnership with academics, learning designers and technologists.

This work continues as a distinctive element of CIC’s broader engagement with the challenges and opportunities of Generative AI in higher education.

Approach

Over a decade’s work at UTS as a research inspired innovation centre has seen us train  staff to deploy practical tools to tens of thousands of students in order to close the feedback loop:

We dubbed this systemic organisational innovation approach “Boardroom, Staff Room, Server Room, Classroom“, which has been our foundation for our GenAI work:

    • Research-inspired pedagogical agents for UTS teaching and learning, translating leading edge research concepts and evidence-based methods into practical learning tools
    • Empirically evaluating the student and educator experience, using mixed qualitative and quantitative methods
    • GenAI tools for professional, research and academic staff
    • GenAI workflows and analytics
    • Cultivating the student qualities needed to thrive in the age of AI
    • Disseminating via webinars and top tier journals and conferences

Research-inspired pedagogical agents

This slide deck gives a snapshot of pedagogically-tuned conversational agents deployed in the UTS Recast enterprise AI platform, which are grounded in the research of CIC members, often in collaboration with faculty colleagues:

    • Socratic dialogue: invites data science students to explore the assumptions behind their questions, and reframe their questions, as they research the environmental impact of LLMs. Builds on the open source Qreframer prompt developed by CIC’s Simon Buckingham Shum [story].
    • Reflection Assistant: Use motivational interviewing to scaffold student’s reflection on their progress towards the goals they set at start of session, in preparation to write a reflective submission (graded). Builds on the LLM coding for belonging research by CIC’s Ram Ramanathan, Lisa Lim and Simon Buckingham Shum [story].
    • Tutor Feedback Buddy: Support tutors to improve the quality of their feedback to students. Builds on the AI feedback coaching research by Antonette Shibani (TD School) and CIC’s Lisa Lim.

GenAI tools for professional, research and academic staff

LLMs aren’t just for students — CIC is working on tools to support the work of our Education Portfolio teams, and faculty academics.

GenAI workflows and analytics

LLMs hold great potential as text analytics tools in diverse educational contexts. In projects with the Transdisciplinary School, and University of Queensland, we orchestrate chains of LLM prompts and document workflows in order to generate custom reports and interactive user interfaces to support sensemaking.

Harnessing LLMs to make sense of student feedback and reflections

  • Until recently, qualitative data analysis (QDA), such as the deductive and inductive coding of textual data, was considered the preserve of human researchers. The nuanced judgements required to apply a complex coding scheme, or to discern themes that evolve into a coding scheme, were beyond algorithms. However, the emergence and mainstream availability of large language models (LLMs: e.g., GPT, Gemini, Claude, Llama) has catalysed rigorous research into their ability to perform such QDA in minutes.
  • Using LLMs hosted by privacy-respecting, secure, university instances, we have been testing LLMs for both inductive and deductive coding of large text corpora from student interviews and written reflection, but these tools and methods apply to all domains. This opens a  healthy debate on whether this could or should lead to the automation of certain kinds of analysis in certain contexts, and the potential augmentation of research teams with a new generation of interactive QDA tools enabling hybrid analysis. [slides]

Cultivating the student qualities needed to thrive in the age of AI

The profound question that the mainstream arrival of GenAI raises for higher education is what our graduates need to know, do and be (knowledge, skills, dispositions) in an age when machines can outperform them in certain respects. Nor can we frame this predicament in a vacuum, as though the challenges faced by society were merely technological and educational. We now confront what has become known as the global polycrisis. Here’s a taste of our work on these questions…

Top