As I noted in my last post, a student in my May-term class committed academic dishonesty in a reading journal I'd sought to make AI-resistant. Students were welcome to use AI to help them (today's undergrads are generally awful readers) understand themes in readings and connections between them. Some students in their evaluations noted that the graded work here "forced them" to do their readings.
Yes, readers, thank you for reading this post. Be advised that many college students no longer do any class readings, unless forced, even at selective institutions. To me, that's a gateway to a new Dark Age, nothing less.
My students, following the rules for multiple-entry journals, need one column with a quotation or summary of a key event, a second column with an analysis of the event, and a third with a question to bring to class. I also include a mandate to comment weekly on a peer's journal.
The student who misused AI had done good work for earlier assessments, yet for the final one asked an AI to find notable quotations. That was not against my policies. What was? In two instances, the AI invented direct quotations not in the readings. The writer, too harried to complacent to check, did not do a word search of the originals.
I gave the writer an F on that assessment, which pulled down the final course-grade. In my reasoning, I said that had that occurred on the job, the student would have likely been fired. Best to learn that ethical lesson now, while the stakes were relatively low (though to a perfection-obsessed undergrad, the stakes may have seemed high, indeed).
We discussed the matter in a cordial way; the writer had done well on earlier work but for reasons still fuzzy to me, failed on this final assessment. So in future classes, I'll change two things. First, my policy on AI hallucinations will be harsh; if the assessments are as frequent as in my recent class there will be no chance at revision. In ones where assessment is less frequent, the F will be applied but the writer will get to revise the journal and I will average the two grades.
Second, I'll add a new requirement for synthesis with earlier readings: yes, a fourth column! This skill is woefully lacking in students, who seem unable to construct consistent narratives across the work done in a class. This I attribute to a "taught to the test" mentality in high school as well as a disconnected learning experience and, for some, lack of passion for learning afterwards.
As for AI hallucination? Though Ethan Mollick claims that larger models hallucinate less now than in the early days, I'm not so sure. Mollick tends to use what I call "concierge" AIs that cost quite a bit; my students generally use a free model and do not engineer their prompts well.
You can read more about a comparison of different LLM models and hallucination here. I still feel that we remain a long way from knowing which LLM to trust and when, but this second article does provide good tests we humans can apply to check the AI's output.
Always check its work. My student did not, and paid a price, rightly so.
Image from the film Vertigo.
