CIC’s first PhD! The Pedagogy of Writing Analytics

https://opus.lib.uts.edu.au/handle/10453/136846

ABSTRACT: Academic writing is a key skill that contributes to essential learning outcomes for higher education students. Despite its importance, students often lack proficiency in writing and find it challenging to learn. While previous research suggests that students’ writing skills are enhanced through formative feedback, the time-consuming nature of providing formative feedback on individual student drafts, especially in large cohorts, makes it impractical for educators to provide detailed writing support in this way. A promising approach, therefore, is the use of writing analytics  to provide automated formative feedback on writing. This particular form of learning analytics, using computational techniques and natural language processing, provides timely, immediate, and consistent automated feedback to help students improve their writing. However, for such tools to work effectively in pedagogic settings, and be adopted by practitioners, academics need to feel a sense of ownership over how the tool fits into their practice. This recognition motivates an increased emphasis on aligning learning analytics applications with learning design, so that analytics-driven feedback is congruent with the pedagogy and assessment regime. The thesis investigates how writing practice can be augmented with a writing analytics tool called ‘AcaWriter’ by aligning it with learning design. The approach is evaluated across two disciplines in authentic higher educational settings using a design-based research approach. Mixed methods and multiple data sources are used to examine how students perceive and interact with automated feedback, and revise their writing. Based on this analysis, the thesis provides empirical evidence that students found the writing intervention and automated feedback from AcaWriter useful, and improved their subject-related writing skills, thus validating its applicability in writing contexts. It identifies varied levels of student engagement with automated feedback and ways to scaffold its application for effective use. Cross-fertilizing research and practice, the key insights gained from these design iterations are formalised as the Contextualizable Learning Analytics Design model. The model clarifies how the features, feedback and learning activities around AcaWriter can be tuned for different pedagogical contexts and assessment regimes, by co-designing them with educators. The thesis also studies the perspectives of educators, who play a key role in implementing such learning analytics innovations in their classrooms. The thesis advances theory and practice in the development of flexible learning analytics applications, capable of providing meaningful, contextualized support that enhances learning, and adoption by practitioners in authentic practice.