Time: 03:30 PM
Location: CB22.01.IdeationStudio and Zoom
Registration
This is a hybrid event and registration is required. Please register for face-to-face attendance here or attendance via Zoom here.
Thesis Title: Automating Multimodal Data Storytelling for Supporting Embodied Team Learning
PhD candidate: Gloria Fernandez Nieto
Please note that this is an open Stage 3 PhD seminar, in which the candidate will summarise her research, prior to a viva with reviewers. Your support and constructive feedback are most welcome!
Abstract
There is a growing interest in creating Learning Analytics (LA) interfaces that support students and teachers directly. Thus far, many of these solutions have been materialised as dashboards and visualisations. However, although a growing number of prototypes and commercial products aimed at supporting students/teachers exist, their limitations are coming under careful scrutiny. For instance, many visual LA tools are failing to provide meaningful and relevant insights that can support students’ reflections on their embodied teamwork activity.
Moreover, there are additional challenges in visualising and communicating the wide variety of multimodal sensor data captured from physical spaces, in a way that supports educational stakeholders (e.g. teachers or students), who as casual users, have limited training in data analysis and interpretation. Thus, this thesis engages research in Information Visualisation (InfoVis) and specifically the Guidance visualisation paradigm that aims to support casual users, or those users with low analysis expertise, to narrow the gap of data visualisation interpretations. Data Storytelling is one way to provide guidance, as a compression technique to help an audience effectively understand what is important in an visualisation, communicating key messages combining data, visualisations, and narratives. ‘Telling stories’ with data in these ways should enable the elicitation of deeper reflections in an effective manner.
This thesis tackles the above challenges by investigating: “How can salient aspects of embodied team activity be automatically identified and derived insights be communicated to support timely, productive reflection?” Four research questions were derived: (1) What modelling techniques can enable identification of salient aspects of multimodal embodied team activity according to the learning design (i.e. teachers’ pedagogical intentions)? (2) How can insights be extracted from multimodal sensors and communicated to students to support reflection on embodied team activity? (3) To what extent can students reflect on their embodied teamwork activity using multimodal data storytelling interfaces? and iv) To what extent can MMLA interfaces be automatically generated to support students’ reflections after their embodied team activity?
This research adopted a mixed methods approach using quantitative and qualitative analyses to provide evidence in response to these research questions. Empirical studies of teamwork were conducted in authentic higher education settings in the context of healthcare (simulations) and science (physics lab) education. Automated multimodal Data Stories were co-designed, evaluated, and implemented to support students and teachers to reflect on different aspects of their embodied practice (patient-care, or co-teaching).
This research makes three types of contribution: modelling, prototypes, and implementation. Regarding modelling, two contributions are presented: (i) a methodology to map from educators’ pedagogical intentions to salient aspects of multimodal embodied team activity, and (ii) the exploration of a set of five multimodal data modelling techniques. In terms of prototypes two contributions are presented: (iii) a set of seven multimodal data storytelling prototypes were created, and (iv) the analysis and evaluation of them to understand their potential for reflection. Finally, in terms of implementations, this research contributes a (v) functional architecture description and documentation and (vi) a reference implementation to automatically generate multimodal LA data stories.
Results from this research point to the potential of creating alternative ways to communicate multimodal data insights to educators and students, by combining visualisation, narrative and storytelling, driven by teachers’ pedagogical intentions. Multimodal data stories of embodied team activity are effective tools to support: (a) teachers as they reflect on student progress; (b) students as they reflect, recall and improve their future practices; (c) teachers in providing timely feedback according to their pedagogical intentions; and (d) teachers as they reflect on their co-teaching practice. In addition, this thesis identifies that LA designers should identify representations that best fit teachers’ and students’ needs, by contextualising and aligning pedagogical intentions with the visual analytics.
Bio
Gloria is a Ph.D. candidate in Learning Analytics at the University of Technology Sydney, with an M.Sc. degree in Computer Science and a BS. degree in Telematics Engineering from Bogotá Colombia. Her current research focuses on exploring new forms of data-informed feedback to understand traces from sensor data collected during collocated teamwork activity, with the goal of prompting reflection in teaching and learning practices by communicating meaningful insights in different f2f contexts. She is an enthusiast of technology, data, Analytics, Knowledge Management, e-learning, and Learning Analytics. Gloria acknowledges the importance of data and the power of data visualisation to communicate strong ideas and stories.
PhD Supervisors: Simon Buckingham Shum, Roberto Martinez Maldonado, Kirsty Kitto.