Ed & Tech must-reads 141025

TELedvisors Webinar - Human-Led, Tech-Empowered: Evaluating Educational Al Tools for Impact Thurs 30th Oct 12pm AEDT from TELedvisors Network

I’ve been looking forward to this session ever since I saw my Deakin Uni colleagues Joan Sutherland (not the dame) and Nicholas English discuss the comprehensive approach that their (our?) ed tech team takes to evaluating educational technologies. It is a thoughtful, transparent and thorough model that pays heed to all the things that the critical ed tech folks bang on about while also focusing on the practical realities of teaching and learning in tertiary education. One not to miss.

Closing the feedback loop: What we learned when our assessment changes failed from Teaching@Sydney

I mentioned recently how much knowledge can be gained by discussing our duds (while the tendency is to focus on triumphs) and lo and behold this post from the teaching@Sydney team popped up. (Unrelated I know). Pace, Bartlett and Patanwala highlight the common challenge of changing cohorts bringing radically different results under the same teaching approach, in this instance, in a postgrad pharmacy unit, following the success of a competency based approach in a similar undergrad unit. Following Brookfield’s four lenses of critical reflective practice, the authors note that postgrad students have less time to dedicate to study, and related factors worked against the students in terms of early skill development and feedback. They responded by tweaking the assessments and shifting hurdle requirements.

Heads we win, tails you lose: AI detectors in education - EdArXiv pre-print

 The talk of the town in Australian higher education last week was media reportage of a university (Australian Catholic University) issuing academic integrity breaches to, by some reports, more than 5000 students which were informed at least partially by their use of the Turnitin AI detection tool. In these numbers, it took some time to get to the bottom of these accusations leaving many students in academic limbo. A lot of universities abandoned technology based AI detection long ago, given unacceptable rates of false positives. This brief (24 pages) but detailed pre-print by some major names in the academic integrity space in Australia, Mark Bassett (CSU), Wayne Bradshaw (JCU), Hannah Bronsztejn (RMIT), Alyce Hogg (CSU), Kane Murdoch (Macquarie), Bridget Pearce (Brisbane Grammar School) and Colin Webber (SAE University College) provides a comprehensive breakdown of the current failings of such platforms, which will hopefully inform better policy and practice in the future. They include unverifiable probabilistic estimates and mutually exclusive linguistic markers and explains why common GenAI ‘tells’ aren’t always helpful. An important read.

Cradle suggests… Feedback and AI from CRADLE

On the happier side of assessment, Deakin Uni’s CRADLE provide this handy 2 page overview of student perspectives on human vs synthetic feedback from GenAI. They note that they appreciate the speed of GenAI feedback but also rely on the nuance and relevance of feedback from their lecturers. The team suggests working with students to help them to understand the limitations of the former and how to make the most of it.

The kids are alright from a friend

Chatting recently to a close friend, they shared a story that they knew I’d like. Their teen son was prepping for a debate with a topic along the lines of school assessments should be 50% written and 50% practical - with the boy taking the negative and arguing for more written assessment. My friend suggested that he could argue that this could facilitate more GenAI assessment of written work, reducing teacher workload. He was outraged by this suggestion. “AI is not accurate enough to be trusted”. The parent said “Really? What the hell do you think I do for a job? I sell AI assessment systems.” “Hmmph" was the reply - “reads like some dystopian novel I’ve read where they think AI can replace whole education systems”.

Reading that back, I can see how improbable it sounds - it has the vibe of a story that ends with “and everybody clapped”. But knowing them, I am absolutely sure it happened. (Rest assured, we also enjoy some robust discussions about ed & tech)