Immediate, personalised feedback on reflective writing

The start of the new year is a good moment to distill the key ingredients of the 2 year collaboration around writing analytics (automated feedback to students on their reflective writing), that CIC has built with UTS academic Cherie Lucas from our School of Pharmacy. As with other collaborations (such as with Pip Ryan on her students’ legal writing), it exemplifies the co-design process that we initiate with academics, in which we iteratively seek to design an automated feedback tool that students can use to improve their drafts, prior to submission:

• distill key insights from the scholarship into the teaching and learning of good reflective writing, to design a formal, implementable model that – in principle – should be applicable to a wide range of reflective writing contexts (learn more)
• understand the academic’s specific educational challenge in context (e.g. students are struggling to  produce good reflective writing about their work experience placements)
• establish the mapping to the features that our text analytics tool is able to detect
• evaluate the performance of the parser (in close partnership with the academic)
• co-design the feedback messages that students will receive depending on their writing
• evaluate students’ reactions to the new tool
• iterate…

In the video, Cherie Lucas describes the nature of the challenge, and what AcaWriter contributes to the learning experience. The interface looks like this, with the text editor frame on the left, and the Reflective Report annotation of the writing on the right, generated after a few seconds on clicking Get Feedback:

Not shown in the video is the Feedback Tab, e.g.

For us, one of the hallmarks of a successful collaboration is that our academic partners’ own disciplinary community of educators recognise the advance they’ve made. Here’s a brief, very helpful introduction that Cherie wrote for her peers. Her work has excited significant interest with colleagues around the world, who are now initiating their own projects to install our software for piloting with their students.

Learn more about how we design Writing Activities with Writing Analytics through the integration of learning design, analytics, educator and student resources, and evaluation evidence…

Dive deeper… (this list has been updated with new publications since this story was first published)

Lucas, C., Buckingham Shum, S., Liu, M., & Bebawy, M. (2021). Implementing AcaWriter as a Novel Strategy to Support Pharmacy Students’ Reflective Practice in Scientific Research. American Journal of Pharmaceutical Education, 8320. (Accepted: 23 March, 2021)

Liu, M., Kitto, K., Buckingham Shum, S. (2021). Combining factor analysis with writing analytics for the formative assessment of written reflection, Computers in Human Behavior: Elsevier BV, 120:106733.

Lucas C., Gibson A. and Buckingham Shum S. (2019), Pharmacy Students’ Utilization of an Online Tool for Immediate Formative Feedback on Reflective Writing Tasks. American Journal of Pharmaceutical Education, 83, (6), Article 6800, pp. 1260-1267.

Liu, M., Buckingham Shum, S., Mantzourani, E. and Lucas, C. (2019). Evaluating Machine Learning Approaches to Classify Pharmacy Students’ Reflective Statements. Proceedings AIED2019: 20th International Conference on Artificial Intelligence in Education, June 25th – 29th 2019, Chicago, USA. Lecture Notes in Computer Science & Artificial Intelligence: Springer.

Lucas C. (2018), Accessorizing the Science Foundation with Internal Mirrors: A Novel Open Source Tool to Enhance Reflective Practice. Pulses. Currents in Pharmacy Teaching and Learning Scholarly Blog. August 28, 2018.

Gibson A., Aitken A., Sándor Á., Buckingham Shum S., Tsingos-Lucas C. and Knight S. (2017), Reflective writing analytics for actionable feedback. Proceedings of LAK17: 7th International Conference on Learning Analytics and Knowledge, March 13-17, 2017, Vancouver, CA (ACM Press: NY). [Reprint] [Video] (AWARDED BEST PAPER)

Tsingos-Lucas C, Aitken A, Gibson A, Buckingham Shum S. (2017). Utilisation of a Novel Online Educational Toolto Assist Pharmacy Students to Self- Critique Reflective Writing Tasks. Proceedings of the 9th Pharmacy Education Symposium, Prato, Italy, 9-12th July 2017. (AWARDED BEST TEACHING INNOVATION POSTER)

Lucas C. (2016), The relationship between reflective practice, learning styles and academic performance in pharmacy education. Doctoral Dissertation, The University of Sydney, Australia. 2016