Turnitin’s AI writing detection report is scheduled to launch today (April 4th). This will appear in the form of a single extra number in the standard similarity report interface representing the percentage of the document that Turnitin is 98% certain may have been created with Generative AI tools like ChatGPT. They claim a 1% false positive rate. This functionality is based on GPT 3 and GPT 3.5, not GPT 4. The simultaneous global release of this report has sparked wide discussion in the TEL community, captured in Twitter threads by Sian Bayne, Tim Fawns and Anna Mills, on a range of topics from the detectability of AI, whether 1% false positives is acceptable and the need for alternative approaches to assessment.
While there was initially no option for institutions to opt out of the functionality – and there is no way for admins to disable it – a number of universities in the UK and some in Australia (Deakin, Sydney, Monash) have decided not to use the functionality just quite yet.
All of which makes this upcoming webinar about assessment and AI incredibly timely. Featuring a host of prominent academics including Thom Cochrane (UniMelb), Ruth Dimes (Auckland), Mitra Jayazeri (La Trobe) and Richard Hall (La Trobe), this session will look at practical approaches to assessment in this new AI age. (Or is that AIge?)
However things shake out, there is probably some value in learning to speak the language of our new digital overlords. This online course appears to be a good place to start, offering a tiered approach to developing skills in writing effective prompts for Gen AI tools.
On a lighter note, I’m a sucker for deliberately bad user interface design. This thread of 22 competition entries from 2017 would offer the most annoying experiences imaginable for listening to audio.
Last week I presented a small webinar about the latest in GenAI tools at work, which went well but would have been significantly easier if there hadn’t been major announcements in the field every day for the weeks leading up to it. ChatGPT, Google, Microsoft, Midjourney, Nvidia, Baidu, Adobe, Canva, and even Opera all having their little “oh and one more thing” moment. This thread from Lennart Nacke summarises the most recent updates nicely.
This article published on Friday has popped up in my feed numerous times and is clearly resonating with people in this space. Irina Dumitrescu puts forward some thoughtful ideas about the nature and value of writing and what we stand to lose as we adapt to the GenAI behemoth by moving away from the idea of first drafts. She suggests that it is this (human) generative work that is some of the most powerful in terms of learning, even when academic writing assessments tend to be highly formulaic by their nature.
Meanwhile, in the physical world, educators continue to grapple with declining attendances in lectures and the need to consider what is next. This article from Peter Bryant is the third in a series considering the value of the lecture and it offers some concrete suggestions for transitioning to interactive experiences centred around active learning. (As a GenXer, it does my jaded heart proud to see a model drawing on the soft-loud-soft stylings of the Pixies and Nirvana)
The Learner Experience (LX) Lab at UTS has been quietly chugging away for some years now doing great work in the digital learning design space (alongside its sister teams). This post from Anthony Burke, Matissa Strong and Rory Green describing their new framework makes the case that students deserve as good an online experience as they get in person and offers practical guidance on how to ensure this. The underlying principles are that learning is Authentic, Aligned, Active and Social.
While institutional leaders push for a move away from the lecture and the cool kids are all talking about ChatGPT, lecture videos still form a significant proportion of online content in Higher Ed. Commonly criticised as a passive and transmissive form of learning, researchers have been looking for pedagogical approaches that get the most out of the viewing experience. This article describes work around using prompts to signpost key concepts in these resources, with those that encourage students to explain concepts in their own language among the most effective.
While it is still fairly primitive, the onward march of GenAI tools to increasingly complex content continues unabated. The HuggingFace Modelscope Text to Video tool works in a similar way to AI image generation tools but creates short (2 seconds) clips of animated images. The examples in the linked Twitter thread make me wonder what we might be seeing later in the year.
Ethan Mollick (@emollick) on Twitter has been one of the more interesting explorers (AIstronauts?) of the GenAI space in recent times and this tweet thread showcases what happened when he shared a previous academic article that he had written with GPT4 and asked for “harsh but fair review from an economic sociologist”. He notes that it raised many of the same things that he received in his human feedback.
As the Metaverse hype seems to recede into the distance with the latest shiny toy, the question that I never felt that was satisfactorily answered was “How is this better than/different to Second Life?” So it was interesting to see this story pop up recently that SL is still chugging along and they are soon to launch a mobile version. I have many fond memories of building weird things in this space and I wish them well.
Some of you may remember the 2021 Netflix series “The Chair” and the equally delightful companion podcast “Academics talk about The Chair” from Inger (Thesis Whisper) Mewburn, Narelle Lemon, Anitra Nottingham and co. that dissected it and what it said about life in the academy. They have announced their next podcast season, exploring last year’s stylish and thinky corporate dystopia series Severance.
Speculation is rampant that a major upgrade to GPT, the language learning model behind ChatGPT, will be released this week following a comment at an event for partners and prospective clients of Microsoft Germany last week. No firm details are confirmed but it is believed that the update will increase the number of ‘parameters’ used by the tool from ~175 billion to 10-100 trillion and it may add multi-modal inputs and outputs (text, images, audio and possibly video). How will this impact learning and teaching? Outputs will probably be better, but it shouldn’t really alter the changes that are already occurring. The associated discussion on Reddit adds some useful surround details, including the fact that Microsoft does have an AI focused event scheduled for Thursday.
Coming back to Earth a little, student perspectives when it comes to the responsible use of GenAI tools like ChatGPT have been light on the ground in all the wider discussion. This twitter thread from Amanda White (UTS) captures the process she worked through with her students in deciding what usage is reasonable. Additionally, the Educational Innovation team at Sydney Uni recently held a couple of panel discussions with students covering their perspectives and the recordings are quite illuminating. While a certain type of student commonly appears in these sessions, it was interesting to note that they didn’t want to let the tools weaken their own writing skills.
While learning content and activities may be vital elements in good online learning courses, the visual and structural design (the User Experience or UX) has a massive impact on their efficacy. This valuable research from Adams, Miller-Lewis and Tieman of UniSA and CQU compared the ability of Learning Designers, healthcare professionals and end-users to identify UX problems in resources based on previously identified end-user errors. They observed that Learning Designers correctly identified nearly three times as many design issues as the other evaluators, highlighting their value in assisting the development of these resources.
There is a cohort in any discussion about the AIpocalypse in Higher Ed whose first question is some variation on ‘how can we detect AI generated writing?’ Given the change that is needed in teaching practice to respond to these tools, it is understandable that a first response might be in the ‘shut it down’ vein. As with most things in the ed tech space though, there is no silver bullet, as this set of basic tests conducted by Armin Alimardani (UoW) and Emma Jane (UNSW) indicate. Detecting AI content is unlikely to ever be reliable and clever users will usually be able to find a workaround.
This paper from Elaine Huber and a cadre of other heavy hitters in business education at USyd and UTS describes some very thoughtful work to develop an overarching framework for online assessment that holistically addresses learner, educator, institutional and disciplinary needs. While different discipline areas clearly have their own needs, the big picture takeaways from this work should be applicable to most educators, ranging across (but not limited to) authenticity, scale, quality feedback, resourcing, and accreditation
It has been interesting to see how all the GenAI talk recently has sucked the air out of a range of other important discussions in the technology enhanced learning space. I am not unhappy that the torrent of publications about remote emergency teaching has slowed to a trickle but things have also been quiet in micro-credentialling space. Happily this paper covers some rich work underway in Ireland, proposing some sensible models and describing some practical examples.
I group these works together – a doctoral thesis from Abilene Christian University and an article from the Australasian Journal of Educational Technology – because they share some interesting overlaps from rather different perspectives. Both relate broadly to effective use of learning technologies by educators and the growing contribution that ‘Third Space’ workers in Higher Ed can/should make to this. The Australians (Tay et al.) note concerns about centralisation, surveillance, institutional homogenisation, responsibility and efficiency when it comes to the use and support of ed tech and both they and January flag a need for greater awareness of support from learning designers (and education technologists) and institutional supports for collaboration between them and educators.
As semester kicks into gear, the perennial cry of students about the high price of textbooks can once again be heard throughout the land. Happily, institutional librarians are at least able to reduce the overall burden of supplementary readings through the use of digital reading list systems. This article from Kumara et al. explores current attitudes toward these platforms, notes different levels of use based on discipline area and the need to improve ease of use.
Good teaching has always been challenging for individual practitioners and as technology and pedagogy grow more sophisticated, this is evermore the case. Neil Mosley discusses the growth of specialist advisors in Learning Design needed to support the evolution of teaching as a design process. Entry paths into this field are still poorly defined, with a smattering of post-grad qualifications emerging but nothing at the undergraduate level yet.
Jenny Pesina reflects on the nature of working relationships between learning designers (and peers) and educators in Higher Education, considering some of the organisational structures that influence how these people can contribute to better learning and teaching. The way that relationships vary based on central vs faculty units and what might be done to strengthen bonds is noteworthy.
This is American news, but these broad policy changes do seem to tend to flow on eventually. In a nutshell it sees third party providers of services to universities that are tied to recruitment and delivery of online programs facing great accountability in their activities. In Australia, this would include Online Program Managers (OPMs) like OES and Keypath, who operate online only programs in many Australian universities.
Two very interesting looking AI webinars on Wednesday this week, with CRADLE/TEQSA continuing their great series of deep dives with Margaret Bearman, Rola Ajjawi, Lucinda McKnight (Deakin), Simon Buckingham Shum (UTS), and Sarah Howard (UoW) considering educator responses and the Education Innovation team at USyd creating much needed space for the student voice in this discussion. (The recording of last week’s TELedvisors Webinar – the Two AIs – is now available on YouTube as well)
With the myriad changes looming in the ed tech space, this insightful piece of crystal-ball gazing from Michael Sankey (CDU) and Stephen Marshall (VUW) about the current and likely future states of the LMS is well worth your time. The authors follow the steady progress of the LMS from single source of learning to the heart of a complex ed tech ecosystem. Along the way they raise interesting ideas about whether the future may look more like MS Teams or Slack (I’m unconvinced for now) and touch on necessary changes to teaching practice wrought by AI that these systems will need to accommodate.
I stumbled up this 2018 article recently and with the discussion of ‘cognitive offloading’ and the need for new approaches to assessment that is occurring in the AI space, it seems like something worth revisiting. Kirschner et al. expand previous work on cognitive load in learners to collaborative learning activities, seeking to understand why some collaborative activities succeed while others fail. Broadly, they find that the transactional nature of collaborative learning and group dynamics should be considered in designing these kinds of tasks.
Privacy is often discussed more in principle than practice in education but it is worth being aware that the Australian government is currently reviewing the 1988 Privacy act and institutions and individual educators will need to consider how they treat student data once the work is done. A closer alignment with the very user-centred EU GDPR model, which gives rights to be deleted from systems, appears likely.
Another day, another test of GenAI tools to see whether they could technically qualify as professional practitioners. This time we see ChatGPT making it into the 99th percentile for macroeconomists via the US Test of Understanding in College Economics.
Following on from the wildly popular AI (ChatGPT) Future webinar in early Feb, the TELedvisors Network presents another in the series, with a stronger focus on assessment and academic integrity questions. Alex Sims from Uni Auckland Business School explores these key issues in the first half, with a open discussion in the second.
One of the ‘tells’ that people have been noticing in ChatGPT (or ChattieG as Anitra Nottingham calls it) is a tendency to make up citations when you ask for references. Many of these seem plausible, with known authors or journals, and they are often correctly formatted, but on investigation, they are simply untrue.
I put a call out for examples of this on Twitter and you did not disappoint.
Hey tweeps, looking for examples of #ChatGPT generating citations, both real and plausible but fake.
A few more people sent me examples by DM and email that were equally entertaining
From Michael Larkin:
Böttiger, B. W., Böhrer, H., Bach, A., Brokmann, J. C., & Motsch, J. (2018). Fibrinolysis in acute myocardial infarction: A review of the current status and future perspectives. International Journal of Cardiology, 254, 1-7. doi: 10.1016/j.ijcard.2017.10.051 O’Gara, P. T., & Antman, E. M. (2016). ST-Elevation Myocardial Infarction: Management. In J. Loscalzo & J. T. Higgings (Eds.), The Brigham Intensive Review of Internal Medicine (pp. 319- 328). Oxford University Press.
The authors and journal is correct but the articles don’t exist and the numbering is made up.
Probably my favourite though came from Brenna Clarke Gray, who coincidentally was in the middle of compiling a presentation and had asked ChattieG about her own publications – figuring she knows her own work the best. Like a gaslighting alpha male, it made a lot of assertions about her. They were not remotely true.
She went on to ask it five times about work that she had written and it kept making things up. Each time she pointed this out and it apologised and produced a new list. Several times it simply repeated the previous list.
Finally, she asked ChattieG to summarise one of her made up works – it continued to double down.
I asked a few times for examples where ChatGPT actually found and shared real papers. Crickets. So I gave it a try myself.
I used the prompt “Annotated reading list of confirmed academic articles relating to relationships between educators, learning designers, education technologists, academic developers and leaders (and relevant synonyms of these roles) in higher education”
I have to admit, it did reasonably well. (It wasn’t quite on target with my question but certainly in the neighbourhood.) The first paper was out by 1 year, the next two were fine, the Mullen paper (as far as I can see) doesn’t exist, the Brown & Warschauer paper weirdly gives D Brown the initials MW but is otherwise correct, the Lajoie (2008) paper gets half the title, the year and the author correct but the journal wrong and the Siemens & Tittenberger book also appears to be correct. (I can’t speak for the annotations).
So there are ways to get correct citations but checking seems vital.
Here is the output, for what it is worth.
Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105.
This article explores the role of computer conferencing in facilitating critical inquiry in higher education, and highlights the importance of collaborative learning environments and effective facilitation by educators and learning designers.
Beetham, H., & Sharpe, R. (2013). Rethinking pedagogy for a digital age: Designing for 21st century learning. Routledge.
This book presents a framework for designing effective learning experiences in the digital age, and emphasizes the need for collaboration and co-creation among educators, learning designers, and academic leaders in higher education.
Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1), 60-70.
This article introduces the concept of technological pedagogical content knowledge (TPACK) and argues for the importance of developing this knowledge among educators and learning designers in order to effectively integrate technology into teaching and learning.
Mullen, C. A., & Wedwick, L. (2008). Building a network of expert teachers: The role of teacher leaders in technology integration. TechTrends, 52(4), 34-37.
This article discusses the importance of teacher leaders in promoting technology integration in higher education, and highlights the role of academic developers in supporting the development of these leaders.
Brown, M. W., & Warschauer, M. (2006). From the university to the elementary classroom: Students’ experiences in learning to integrate technology in instruction. Journal of Technology and Teacher Education, 14(3), 599-621.
This article explores the challenges and opportunities of integrating technology into instruction in higher education, and emphasizes the importance of collaboration among educators, learning designers, and academic developers in preparing students for this task.
Dron, J. (2007). Control and constraint in e-learning: Choosing when to choose. Hershey, PA: Information Science Publishing.
This book explores the role of control and constraint in e-learning, and argues for a more collaborative and flexible approach to learning design that empowers educators and learning designers to make informed choices about how to structure and facilitate learning experiences.
Lajoie, S. P. (2008). Metacognition, self-regulation, and self-regulated learning: Research recommendations. Educational Technology Research and Development, 56(1), 47-64.
This article discusses the importance of metacognition and self-regulated learning in higher education, and highlights the role of educators, learning designers, and academic developers in promoting these skills among students.
Siemens, G., & Tittenberger, P. (2009). Handbook of emerging technologies for learning. University of Manitoba.
This handbook provides an overview of emerging technologies for learning and emphasizes the need for collaboration and co-creation among educators, learning designers, and academic leaders in exploring and implementing these technologies in higher education.
Something that is become more and more apparent as the discourse around the generative AI revolution continues is the virtual absence of student voices. We are making a great many assumptions about what they think and what they will do, but are they valid? Emily Pitts Donahoe from the English department at Notre Dame university (US) runs through a recent discussion she had with her first year writing students, touching on academic integrity and the quality of AI outputs.
Looking at teaching from another angle, instructional (learning) designers sometimes use ‘personas’ in a design process to centre their work around the learner need. A persona might include demographic information alongside personalities and learner motivations. Rebecca Hogue explores using ChatGPT to have (basic) conversations with personas that might be interested in shifting careers to the instructional (learning) design discipline. As with many AI outputs, it can be a little basic but also offers a convenient framework to refine.
Without realising it, I appear to have hit upon a student centred theme this week. This article summarises some interesting discussions relating to what students want and need from the various education technologies in their institutions – and particularly the way their educators choose them and use them. Key ideas include academic freedom vs consistency, support & training, and cognitive overload from too many new tools.
Catalyst is a major host and developer for the Moodle Learning Management System (LMS), so it is always worth being mindful of your source with these posts. As someone who has worked with Moodle in a number of institutions of the years though, the key ideas here certainly resonated. What do you do when a perception emerges that your institutional system seems ‘tired’ or ‘dated’? Is this valid or are there other factors influencing this? How much can and will recent updates to Moodle change this?
Developing good learning and teaching practices in educators in Higher Ed has its challenges. This short interactive game developed by the Wharton Business School at the University of Pennsylvania does a lot of heavy lifting in showcasing some key concepts as players work through a fun 10 min teaching scenario.
First published in Campus Morning Mail 7th Feb, 2023
Hello colleagues – I must assure you that I am aware that there is a wider technology enhanced learning universe beyond AI/ChatGPT but at the moment it is hard to find anything else.
There is a concept in Internet research referred to as participation inequality or the 90/9/1 rule. This essentially states that 90% of people involved in an online community don’t participate much, preferring to ‘lurk’. 9% contribute from time to time and 1% does the majority of the talking. This paper from Zhu and Dawson explores the differences in informal learning outcomes between members of these groups in popular education communities on Reddit. While ‘lurkers’ and posters report that they learn from the community at roughly equal levels, the authors note that posters apply and analyse what they have learned more frequently.
As the discussion about the practical use of generative AI tools moves forward, the importance of designing good prompts to get the most from the technology becomes increasingly apparent. This in depth post from Philippa Hardman describes her process for designing a rich prompt to generate a learning activity centred around the educational strategy of “Undoing”. She explains seven key elements of her prompts and offers practical suggestions.
I haven’t used this service and in no way endorse it – I simply present it as an interesting example of the ways that ed tech companies are starting to monetise this space. I guess it is Prompts As A Service. From what I can make out, it is essential a set of prompt templates tied to specific learning and teaching needs. It ranges from generating a title for your new course to generating a presentation task to assess learning.
Sometimes when there is a world of content out there about a new topic, the easiest thing to do is to listen to some experts – as much as anyone can be an expert currently – talk through the issues. The first of these is one that I organised last week and the second features our panellist Anna Mills and the great Maha Bali.
You must be logged in to post a comment.