Interactive Workshops: All staff and students welcome, particularly with interests in formative assessment; language technology; health sciences

2.30-4pm Mon 14 Sept and 10-30-12pm Wed 16 Sept, CIC

Short Answers to Deep Questions:
Automating support for teachers in providing formative feedback?

Jenny McDonald (Visiting researcher in CIC for the whole week)
Higher Education Development Centre, University of Otago

We have already become used to the idea that we can talk to our phones and answering machines about relatively mundane topics. Now, as language technologies emerge from the research labs and into educational products, future students will be responding to open-ended questions by speaking or typing their answers, and receiving instant feedback. This raises many intriguing questions around how language relates to learning, the topic of this session.

In this interactive, hands-on session, I will present the early stages of a research project, which utilises students’ written responses to questions (in this case, concerning human physiology). The students’ responses were collected during trials of a tutorial dialogue system conducted in a large first-year health sciences course. Several thousand free-text responses to questions posed by the dialogue system were recorded. In the course of examining these student responses, as part of the evaluation of the dialogue system, we noticed that the language students used ranged from expressions of complete confusion such as ‘I dunno’, through chunks of appropriately or inappropriately recalled text, often derived from course materials or lessons, to reasonably coherent, but not necessarily correct explanations, which reflected students’ own understanding or synthesis. In a few cases students appeared to be adopting or trying to adopt the language of the discipline in expressing their ideas.

Based on this observation, and utilising emerging research in the area of disciplinary literacy or fluency, rapidly developing discourse-centric analytics techniques and developments in automated short-answer grading, we plan to investigate the following research questions:

1. Can we use passively collected, student free-text responses to identify linguistic features from which to develop indicators of student disciplinary literacy?
2. Will the provision of such indicators support teaching staff to write and revise formative questions for their students, which specifically target disciplinary literacy development?

In this session, you will have the opportunity to experiment with this dialogue tutorial system, discuss your experiences, review examples of what real health sciences students say when using it, and reflect on whether there are patterns that might be pedagogically useful.

Bio: Jenny McDonald has a medical degree and a PhD in computer science and education. She has particular research interests in natural language processing techniques for formative feedback and learning analytics but is broadly interested in any applications of technology which support educational goals.

The specific research questions she asks relate to how we construct and understand teaching and learning dialogues, how we conduct these conversations both online and in person, how these are used to support learning and teaching and how we know whether these conversations are beneficial.

Jenny has led or consulted on several large educational technology research and development projects including: Unitube, a media sharing platform at the University of Otago; Otago InForm for requesting and processing teaching and course evaluations; Online Information Literacy Modules to support academic skills development; Te Whanake Online for Maori language learning and most recently the Online Cardiovascular System Tutor.

She is a member of the Australasian Society for Computers in Learning in Tertiary Education (ascilite) and was co-convenor for the 31st ascilite conference held in Dunedin in 2014 and co-hosted by the University of Otago and Otago Polytechnic.

She is currently co-Principal Investigator on an Ako Aotearoa National Project Fund Project (NPF15-008): Building an evidence base for teaching and learning design using learning analytics data. Jenny’s work has been funded by the University of Otago, the NZ Tertiary Education Commission, the NZ Ministry of Education and Ako Aotearoa.