AI assessment AutoGPT ChatGPT CMM ed tech GPT4 higher education

Ed tech must reads: column #77

First published in Campus Morning Mail 18th April, 2023

What is Auto-GPT? Everything to know about the next power AI tool from Zdnet

Another day, another mind bending development in AI land. One of the new exciting new toys is AutoGPT, a Python app published to GitHub on the 30th of March. It can access the Internet and uses GPT4 to iterative set itself a series of tasks to perform to achieve whatever goal you initially set for it. Essentially you can tell it to do X, it will identify that it needs to do A, B and C to achieve this and then also that it needs to do D, E and F to get A done. Then it goes off and does it, checking in with you from time to time that it is on track. One example cited mentions that AutoGPT had been asked to create an app and recognised that the user didn’t have Node software. So, it worked out how to install that, did that and continued. Wild times. I’m not entirely sure what the educational applications are but it might have saved Aneesha Bakharia (UQ) some time.

Introducing my latest AI creation, EduWeaver from Aneesha Bakharia

Aneesha Bakharia was one of the expert panellists in the TELedvisors/CCCL AI webinar earlier this year and continues to impress with her new AI based online module creation tool, Eduweaver. It allows users to nominate a topic and the tool outputs a set of simple text content and MCQ pages. She offers examples covering Meteorology, Learning Analytics, Javascript and more.

Auto-GPT Unmasked: The Hype and Hard Truths of Its Production Pitfalls from Jina

As with most new tools, AutoGPT has its challenges to work through. This in dept post from Han Xiao, founder of Jina AI offers a fairly comprehensive breakdown of some of the practical challenges that users might face with this just-over-two-week-old technology. It gets techy but the jist isn’t too hard to pick up.

Empowering learners for the age of AI from Computers and Education: Artificial Intelligence (Open Access)

For a slightly more scholarly take on the current mess/age of wonder, this recent issue of Computers and Education: Artificial Intelligence features articles from George Siemens, Dragan Gašević, Lina Markauskaite, Kirsty Kitto, Simon Buckingham Shum, and a host of other luminaries on many facets of what we can and should do next with AI in education.

We need to change the way universities assess students, starting with these 3 things from The Conversation

And now for something (almost) completely different, this article from Joanna Tai, Margaret Bearman, Mollie Dollinger and Rola Ajjawi proposes some simple yet essential changes in the way that universities handle assessment, including more student choice, more feedback and greater inclusivity.



The appearance of computers in the workplaces at the turn of the 21st century has added ‘algorithmic thinking’ and ‘computing literacy’ to the repertoire of thinking skills and literacies that have been seen as essential for successful functioning and employment in society (surname, 2012). The proliferation of personal computers and other digital devices in people’s everyday lives raised the need for different kinds of skills and literacies, such as ‘ICT skills’, ‘media literacy’ and ‘digital literacy’ (Markauskaite, 2005, 2006). The recent emergence of big data, machine learning, robotics and Al gave the birth to ‘data literacy’, ‘computational thinking’, ‘AI literacy’ and other new skills (Bull, Garofalo, & Hguyen, 2020; Long & Magerko, 2020; Mandinach & Gummer, 2013). Simultaneously, the increasing interconnectivity, complexity, and fast changes in knowledge and skills needed for everyday life and jobs have shifted the attention from technology-centred skills and literacies to a broader set of generic competencies, such as creativity, analytical thinking, active self-driven learning, and global citizenship (World Economic Forum, 2018, 2020).

AI assessment ChatGPT CMM ed tech game based learning game design GPT4 higher education OPM

Ed tech must reads: column #76

First published in Campus Morning Mail 11th April, 2023

We tested a new ChatGPT-detector for teachers. It flagged an innocent student. From The Washington Post

Now that institutions (that didn’t opt out) have access to the new Generative AI reporting functionality in Turnitin, initial reports from testing aren’t exactly aligning with the vendor’s claims of accuracy. This (admittedly small) test against 16 real and fake samples generated by high school students by a WaPo journalist correctly identified 6, got 3 wrong and got mixed results on the remaining 7. This is certainly a space many people in education will be watching closely.

6 Tenets of Postplagiarism: Writing in the Age of Artificial Intelligence from Learning, Teaching and Leadership

I must admit, I had been surprised by the urgency with which the ed tech sector has moved into the new space of AI detection. I mean, clearly there is money to be made but given the looming arms race, having a product with a compelling level of accuracy seems essential. This post from Sarah Eaton (and conversations with colleagues during the week) highlights the idea that the ability to cheat with computers does have the potential to greatly reduce ‘traditional’ types of plagiarism. Eaton describes the prospective new landscape in an eye-opening way.  

2U Lawsuit Claims Looming Education Dept. Guidance Breaks the Law from The Chronicle of Higher Education.

I shared a story a bit over a month ago about the US Dept of Education announcing plans for stronger oversight over Online Program Managers (OPMs), third party partners that work with institutions to deliver online courses. One of the big players, 2U, has decided that they don’t care for this and are going to the courts. Clearly this doesn’t directly affect providers in Australia but I am sure that TEQSA and similar bodies are paying attention.

We Settled for Catan from The Atlantic (Paywall – free trial)

Ian Bogost is an academic, video game designer and provocateur, so I always have time for his hot takes. This article celebrates the work of recently departed German boardgame designer Klaus Teuber, creator of the renowned Settlers of Catan. Bogost’s angle is notable for pointing out that Catan isn’t a Great game but it’s ordinariness and simplicity makes it accessible and enjoyable to a far wider audience than most boardgames. He argues that this is in some ways more important than being the smartest game on the shelf and I think there are possibly lessons in that which can be applied to learning and teaching.

AI assessment ChatGPT CMM ed tech GPT4 higher education Lecture UX

Ed tech must reads: column #75

First published in Campus Morning Mail 4th April, 2023

Turnitin’s AI writing detector launches today – or does it?

Turnitin’s AI writing detection report is scheduled to launch today (April 4th). This will appear in the form of a single extra number in the standard similarity report interface representing the percentage of the document that Turnitin is 98% certain may have been created with Generative AI tools like ChatGPT. They claim a 1% false positive rate. This functionality is based on GPT 3 and GPT 3.5, not GPT 4. The simultaneous global release of this report has sparked wide discussion in the TEL community, captured in Twitter threads by Sian Bayne, Tim Fawns and Anna Mills, on a range of topics from the detectability of AI,  whether 1% false positives is acceptable and the need for alternative approaches to assessment.

While there was initially no option for institutions to opt out of the functionality – and there is no way for admins to disable it – a number of universities in the UK and some in Australia (Deakin, Sydney, Monash) have decided not to use the functionality just quite yet.  

Designs for our times: adapting assessment in an AI context – Webinar Wed 5th April 5pm AEST from ASCILITE Transforming Assessment, TELedvisors Network and Learning Design SIGs.

All of which makes this upcoming webinar about assessment and AI incredibly timely. Featuring a host of prominent academics including Thom Cochrane (UniMelb), Ruth Dimes (Auckland), Mitra Jayazeri (La Trobe) and Richard Hall (La Trobe), this session will look at practical approaches to assessment in this new AI age. (Or is that AIge?)

A Free, Open Source Course on Communicating with Artificial Intelligence from Learn Prompting

However things shake out, there is probably some value in learning to speak the language of our new digital overlords. This online course appears to be a good place to start, offering a tiered approach to developing skills in writing effective prompts for Gen AI tools.

The worst volume control interfaces in the world from Twitter

On a lighter note, I’m a sucker for deliberately bad user interface design. This thread of 22 competition entries from 2017 would offer the most annoying experiences imaginable for listening to audio.

AI ChatGPT CMM ed tech GPT4 higher education Learning design Lecture

Ed tech must reads: column #74

First published in Campus Morning Mail 28th March, 2023

AI is having an iPhone moment from Twitter

Last week I presented a small webinar about the latest in GenAI tools at work, which went well but would have been significantly easier if there hadn’t been major announcements in the field every day for the weeks leading up to it. ChatGPT, Google, Microsoft, Midjourney, Nvidia, Baidu, Adobe, Canva, and even Opera all having their little “oh and one more thing” moment. This thread from Lennart Nacke summarises the most recent updates nicely.

Will ChatGPT Kill the Student Essay? Universities Aren’t Ready for the Answer from The Walrus

This article published on Friday has popped up in my feed numerous times and is clearly resonating with people in this space. Irina Dumitrescu puts forward some thoughtful ideas about the nature and value of writing and what we stand to lose as we adapt to the GenAI behemoth by moving away from the idea of first drafts. She suggests that it is this (human) generative work that is some of the most powerful in terms of learning, even when academic writing assessments tend to be highly formulaic by their nature.

Chaos and calm in the lecture theatre: Transforming the lecture by creating and sustaining interactivity at scale part 3 from USyd Business Education Research Group

Meanwhile, in the physical world, educators continue to grapple with declining attendances in lectures and the need to consider what is next. This article from Peter Bryant is the third in a series considering the value of the lecture and it offers some concrete suggestions for transitioning to interactive experiences centred around active learning. (As a GenXer, it does my jaded heart proud to see a model drawing on the soft-loud-soft stylings of the Pixies and Nirvana)

A framework for quality standards in digital design from the UTS LX Lab

The Learner Experience (LX) Lab at UTS has been quietly chugging away for some years now doing great work in the digital learning design space (alongside its sister teams). This post from Anthony Burke, Matissa Strong and Rory Green describing their new framework makes the case that students deserve as good an online experience as they get in person and offers practical guidance on how to ensure this. The underlying principles are that learning is Authentic, Aligned, Active and Social.

AI ChatGPT CMM ed tech GPT4 higher education Lecture Metaverse Second life video

Ed tech must reads: column #73

First published in Campus Morning Mail 21st March, 2023

Can prompts improve self-explaining an online video lecture? Yes, but do not disturb! From International Journal of Educational Technology in Higher Education

While institutional leaders push for a move away from the lecture and the cool kids are all talking about ChatGPT, lecture videos still form a significant proportion of online content in Higher Ed. Commonly criticised as a passive and transmissive form of learning, researchers have been looking for pedagogical approaches that get the most out of the viewing experience. This article describes work around using prompts to signpost key concepts in these resources, with those that encourage students to explain concepts in their own language among the most effective.

The first open source text-to-video AI generation tool has been released from Huggingface

While it is still fairly primitive, the onward march of GenAI tools to increasingly complex content continues unabated. The HuggingFace Modelscope Text to Video tool works in a similar way to AI image generation tools but creates short (2 seconds) clips of animated images. The examples in the linked Twitter thread make me wonder what we might be seeing later in the year.

Can GPT-4 replace Reviewer 2 from Twitter

Ethan Mollick (@emollick) on Twitter has been one of the more interesting explorers (AIstronauts?) of the GenAI space in recent times and this tweet thread showcases what happened when he shared a previous academic article that he had written with GPT4 and asked for “harsh but fair review from an economic sociologist”. He notes that it raised many of the same things that he received in his human feedback.

20 years later, Second Life is launching on mobile from Ars Technica

As the Metaverse hype seems to recede into the distance with the latest shiny toy, the question that I never felt that was satisfactorily answered was “How is this better than/different to Second Life?” So it was interesting to see this story pop up recently that SL is still chugging along and they are soon to launch a mobile version. I have many fond memories of building weird things in this space and I wish them well.

Coming soon! Academics talk about Severance from Thesiswhisperer Pod

Some of you may remember the 2021 Netflix series “The Chair” and the equally delightful companion podcast “Academics talk about The Chair” from Inger (Thesis Whisper) Mewburn, Narelle Lemon, Anitra Nottingham and co. that dissected it and what it said about life in the academy. They have announced their next podcast season, exploring last year’s stylish and thinky corporate dystopia series Severance.

AI ChatGPT CMM ed tech GPT4 higher education Learning design UX

Ed tech must reads: column #72

First published in Campus Morning Mail 14th March, 2023

GPT4 is coming this week? From Heise Online

Speculation is rampant that a major upgrade to GPT, the language learning model behind ChatGPT, will be released this week following a comment at an event for partners and prospective clients of Microsoft Germany last week. No firm details are confirmed but it is believed that the update will increase the number of ‘parameters’ used by the tool from ~175 billion to 10-100 trillion and it may add multi-modal inputs and outputs (text, images, audio and possibly video). How will this impact learning and teaching? Outputs will probably be better, but it shouldn’t really alter the changes that are already occurring. The associated discussion on Reddit adds some useful surround details, including the fact that Microsoft does have an AI focused event scheduled for Thursday.

Engaging with students on the use of GenAI tools from Twitter (and USyd)

Coming back to Earth a little, student perspectives when it comes to the responsible use of GenAI tools like ChatGPT have been light on the ground in all the wider discussion. This twitter thread from Amanda White (UTS) captures the process she worked through with her students in deciding what usage is reasonable. Additionally, the Educational Innovation team at Sydney Uni recently held a couple of panel discussions with students covering their perspectives and the recordings are quite illuminating. While a certain type of student commonly appears in these sessions, it was interesting to note that they didn’t want to let the tools weaken their own writing skills.

Learning Designers as Expert Evaluators of Usability: Understanding Their Potential Contribution to Improving the Universality of Interface Design for Health Resources from International Journal of Environmental Research and Public Health

While learning content and activities may be vital elements in good online learning courses, the visual and structural design (the User Experience or UX) has a massive impact on their efficacy. This valuable research from Adams, Miller-Lewis and Tieman of UniSA and CQU compared the ability of Learning Designers, healthcare professionals and end-users to identify UX problems in resources based on previously identified end-user errors. They observed that Learning Designers correctly identified nearly three times as many design issues as the other evaluators, highlighting their value in assisting the development of these resources.

AI badges ChatGPT CMM collaboration ed tech higher education Learning design Microcredentials

Ed tech must reads: column #71

First published in Campus Morning Mail 7th March, 2023

We pitted ChatGPT against tools for detecting AI-written text, and the results are troubling from The Conversation

There is a cohort in any discussion about the AIpocalypse in Higher Ed whose first question is some variation on ‘how can we detect AI generated writing?’ Given the change that is needed in teaching practice to respond to these tools, it is understandable that a first response might be in the ‘shut it down’ vein. As with most things in the ed tech space though, there is no silver bullet, as this set of basic tests conducted by Armin Alimardani (UoW) and Emma Jane (UNSW) indicate. Detecting AI content is unlikely to ever be reliable and clever users will usually be able to find a workaround.

Towards a framework for designing and evaluating online assessments in business education from Assessment & Evaluation in Higher Education

This paper from Elaine Huber and a cadre of other heavy hitters in business education at USyd and UTS describes some very thoughtful work to develop an overarching framework for online assessment that holistically addresses learner, educator, institutional and disciplinary needs. While different discipline areas clearly have their own needs, the big picture takeaways from this work should be applicable to most educators, ranging across (but not limited to) authenticity, scale, quality feedback, resourcing, and accreditation

Micro-learning, Digital Badges and Micro-credentials: Definitions, Affordances and Design Considerations for application in Higher Education Institutions from All Ireland Journal of Higher Education

It has been interesting to see how all the GenAI talk recently has sucked the air out of a range of other important discussions in the technology enhanced learning space. I am not unhappy that the torrent of publications about remote emergency teaching has slowed to a trickle but things have also been quiet in micro-credentialling space. Happily this paper covers some rich work underway in Ireland, proposing some sensible models and describing some practical examples.

The Perceptions of Faculty and Instructional Designers Regarding the Impact of Professional Development to Teach Online Courses from Scott Mitchell January (Thesis)

Learning technology as contested terrain: Insights from teaching academics and learning designers in Australian higher education from AJET

I group these works together – a doctoral thesis from Abilene Christian University and an article from the Australasian Journal of Educational Technology – because they share some interesting overlaps from rather different perspectives. Both relate broadly to effective use of learning technologies by educators and the growing contribution that ‘Third Space’ workers in Higher Ed can/should make to this. The Australians (Tay et al.) note concerns about centralisation, surveillance, institutional homogenisation, responsibility and efficiency when it comes to the use and support of ed tech and both they and January flag a need for greater awareness of support from learning designers (and education technologists) and institutional supports for collaboration between them and educators.

AI ChatGPT CMM collaboration ed tech higher education Learning design OPM

Ed tech must reads: column #70

First published in Campus Morning Mail 28th Feb, 2023

Online reading lists: a mixed-method analysis of the academic perspective from International Journal on Digital Libraries

As semester kicks into gear, the perennial cry of students about the high price of textbooks can once again be heard throughout the land. Happily, institutional librarians are at least able to reduce the overall burden of supplementary readings through the use of digital reading list systems. This article from Kumara et al. explores current attitudes toward these platforms, notes different levels of use based on discipline area and the need to improve ease of use.

What’s behind the growth and interest in learning design? From Neil Mosley

Good teaching has always been challenging for individual practitioners and as technology and pedagogy grow more sophisticated, this is evermore the case. Neil Mosley discusses the growth of specialist advisors in Learning Design needed to support the evolution of teaching as a design process. Entry paths into this field are still poorly defined, with a smattering of post-grad qualifications emerging but nothing at the undergraduate level yet.

An evolving partnership model in higher education — a matter of inter-connections from Medium

Jenny Pesina reflects on the nature of working relationships between learning designers (and peers) and educators in Higher Education, considering some of the organisational structures that influence how these people can contribute to better learning and teaching. The way that relationships vary based on central vs faculty units and what might be done to strengthen bonds is noteworthy.

Education Dept. Shocks Ed-Tech Experts and Colleges With Expansion of Oversight from the Chronicle of Higher Education

This is American news, but these broad policy changes do seem to tend to flow on eventually. In a nutshell it sees third party providers of services to universities that are tied to recruitment and delivery of online programs facing great accountability in their activities. In Australia, this would include Online Program Managers (OPMs) like OES and Keypath, who operate online only programs in many Australian universities.

ChatGPT – how should educators respond? Webinar Wed 1st March 2-4 pm AEDT from CRADLE/TEQSA and Student-staff forum on generative artificial intelligence at Sydney Wed 1st March 1-2 pm from USyd

Two very interesting looking AI webinars on Wednesday this week, with CRADLE/TEQSA continuing their great series of deep dives with Margaret Bearman, Rola Ajjawi, Lucinda McKnight (Deakin), Simon Buckingham Shum (UTS), and Sarah Howard (UoW) considering educator responses and the Education Innovation team at USyd creating much needed space for the student voice in this discussion. (The recording of last week’s TELedvisors Webinar – the Two AIs – is now available on YouTube as well)

AI ChatGPT CMM collaboration ed tech higher education privacy

Ed tech must reads: column #69

First published in Campus Morning Mail 21st Feb, 2023

Perspective Chapter: The Learning Management System of 2028 and How we Start Planning for this Now from Higher Education – Reflections from the Field

With the myriad changes looming in the ed tech space, this insightful piece of crystal-ball gazing from Michael Sankey (CDU) and Stephen Marshall (VUW) about the current and likely future states of the LMS is well worth your time. The authors follow the steady progress of the LMS from single source of learning to the heart of a complex ed tech ecosystem. Along the way they raise interesting ideas about whether the future may look more like MS Teams or Slack (I’m unconvinced for now) and touch on necessary changes to teaching practice wrought by AI that these systems will need to accommodate.

From Cognitive Load Theory to Collaborative Cognitive Load Theory from International Journal of Computer-Supported Collaborative Learning

I stumbled up this 2018 article recently and with the discussion of ‘cognitive offloading’ and the need for new approaches to assessment that is occurring in the AI space, it seems like something worth revisiting. Kirschner et al. expand previous work on cognitive load in learners to collaborative learning activities, seeking to understand why some collaborative activities succeed while others fail. Broadly, they find that the transactional nature of collaborative learning and group dynamics should be considered in designing these kinds of tasks.

Australian privacy reform moves forward with new government report from International Association of Privacy Professionals

Privacy is often discussed more in principle than practice in education but it is worth being aware that the Australian government is currently reviewing the 1988 Privacy act and institutions and individual educators will need to consider how they treat student data once the work is done. A closer alignment with the very user-centred EU GDPR model, which gives rights to be deleted from systems, appears likely.

Is ChatGPT Smarter than a Student in Principles of Economics? From SSRN

Another day, another test of GenAI tools to see whether they could technically qualify as professional practitioners. This time we see ChatGPT making it into the 99th percentile for macroeconomists via the US Test of Understanding in College Economics.

The two ‘AIs’ – academic integrity and artificial intelligence – Webinar Thurs 23rd Feb, 12 noon AEDT from ASCILITE TELedvisors Network

Following on from the wildly popular AI (ChatGPT) Future webinar in early Feb, the TELedvisors Network presents another in the series, with a stronger focus on assessment and academic integrity questions. Alex Sims from Uni Auckland Business School explores these key issues in the first half, with a open discussion in the second.