Evaluative Judgement has been identified as a key graduate competency by UTS Emeritus Professor Dave Boud, one of the world’s leading experts on the future of assessment in higher education. This is a lifelong skill worth having — if your judgements about the quality of your own, or a peer’s work, are close to what an expert would say, you’re doing well.
But how do you build this valuable ability? The answer is through a mix of skilful teaching and assessment design (the focus of Boud’s research), enabled by a web platform such as REVIEW, developed from Darrall Thompson’s research. REVIEW is now embedded in many subjects at UTS, and numerous other universities (and now schools seeking to shift their assessment regimes).
REVIEW helps educators articulate good assessment criteria, mapped to whatever transferable competencies they are targetting. The tool then provides sliders for each assessment criterion, so when submitting their work, students adjust the slider to indicate how well they think they have done. When they receive their grade, they can then see the difference between their self-assessment, and that of the academic, which is itself invaluable feedback to them, as shown below:
REVIEW can then report how well a cohort is able to self-assess, e.g. this scatterplot:
Moreover, CIC’s Simon Buckingham Shum and Mike Pracy are now working with the REVIEW team to interrogate the entire dataset of thousands of students, from years of use. Data scientist Mike Pracy is generating a range of fascinating visualisations, including one showing “evaluative judgement curves”. These show us how students start a new subject by overestimating their ability, but over the weeks, gradually calibrate to converge on accurate judgements. In fact they then ‘overshoot’, underestimating their ability. When they finish that subject, and start a new one, this pattern repeats, but they do seem to overestimate less, which may demonstrate improved evaluative judgement. Future research publications will document these findings in detail, but it’s exciting that for the first time, we have data large enough to see the process of evaluative judgement formation in action!