Evaluating AI-Generated Student Work

Evaluating AI-Generated Student Work

Picture this: you’re teaching a 10-week undergraduate class, you assigned a weekly 1-paragraph response to the week’s readings, and your students are now submitting content that feels oddly correct and impersonal. You immediately suspect they wrote it with AI. Help! What do you do?

First of all, realize you’re not alone: It’s not the way your class is set up. A 2024 study has estimated that 86% of interviewed students used some form of AI to help with their homework. This new scenario is here to stay. Given that, what can you do as an instructor to still guide your students to engage in critical thinking, and what can you do when you suspect the work they submitted is (excessively) AI-generated?

Focus on your class goals

Think about the type of assignments and tests you designed for your class. What are you trying to test for, and what are you trying to assess when grading? This can help you start to shift your focus from the format of your assignment - which you might have to change- to the aim of your assignment - which you want to preserve.

To help you brainstorm, here are a few common points instructors test for, depending on course content:

  1. Memorization: facts, formulas, events, grammar structures

  2. Comprehension: contextualization and interpretation of written passages or historical situations

  3. Writing ability: fluency with different formats, writing styles, clarity in expressing ideas, precision in formulating arguments

  4. Critical thinking: commenting, critiquing, creating connections, noticing gaps

Ask your students openly

If you want to figure out how to go around the problem of heavy AI use by your students, you may want to first understand why they use it. Two of the most common reasons - and two different ways of looking at the problem - are for cheating and creativity. Depending on how you look at it, you might want to choose different paths to address the situation: find ways to curb the behavior or find ways to accompany it. To start, you could try to ask your students directly. The fact that AI use is common is an open secret at this point, so as the instructor, don’t be afraid to bring it up first. Is it because they are pressed for time? Because they have too much homework to get to? Because they don’t feel confident in their writing ability? Because the writing itself is too challenging? Because they don’t know where to start? Etc…

You could share an anonymous survey with your students in Week 1 of the quarter asking them what leads them to use AI (you can use an anonymous Google form or set up an anonymous survey on Bruin Learn). If you choose this approach, remind your students that you’re not trying to police their answers - you’re trying to understand how they function in your class to make sure you can help them take away something valuable from it.

Rework your assignments

Try to reshape some of your assignments in a way that focuses on feedback and improvement while making AI feel less necessary.

  1. Consider multimodal exercise formats: ask your students to submit audios or videos of themselves speaking, presenting on a topic, highlighting, or commenting on a paper they have to respond to.

  2. If your class size and topic permits it, break down your writing assignments into smaller-sized iterations throughout the quarter: this will give you a chance to get better acquainted with each student’s voice and style, and it will give them a chance to work on their writing while feeling less time and grading pressure on each individual written assignment.

  3. Focus on improvement rather than one-off performance: if your class calls for a longer paper or complex final project, try scaffolding your assignments throughout the quarter in such a way that individual homework assignments help your students build towards and get feedback for the final project.

  4. Include AI in your assignments: if your class is focused on analysis and critical thinking, encourage students to include an AI response among their sources and explicitly evaluate the type of information it provides, analyze its tone, positionality, and possible biases, or compare it to the type of information they gather from other sources.

  5. Encourage students to cite AI and treat it as they would another source: if your class is focused on writing or research, help students learn the differences between piecing knowledge together from different sources (each with different levels of validity and credibility) and gathering knowledge from one opaque source that presents a single possible answer to their questions.

  6. Consider giving your students some flexibility on their deadlines and choice of topics, to empower them to actively choose what to work on and to disincentivize AI use for assignments they may have less time for or less interest in. For example, if your class calls for 5 response papers, create 8 prompts for the quarter and let your students choose which 3 weeks or which 3 topics they don’t want to cover. 

Grading AI work

Regardless of the new types of assignments and class formats you may experiment with, you will likely still be confronted with the question of how to grade work that was supposed to be student-generated but feels like it was AI-generated. 

Here’s an ugly truth: There’s no surefire way to identify AI-generated writing from human-generated writing. Turnitin and Grammarly, among others, have an AI-identification feature, but they are not foolproof. Studies have shown that there is no truly reliable way to spot AI writing because AI-generated content has no intrinsic linguistic or technical markers distinguishing it from human language. 

As an instructor, try to be wary of implicit biases leading you to believe a student couldn’t possibly have come up with a specific way of phrasing or presenting content by themselves. Consider focusing on whether students seem to have gained a nuanced or improved understanding of your class content, rather than using your or your TA’s time to investigate AI use based on specific aspects of the students’ writing. 

If you suspect a student's assignment is AI-generated, you have a few options:

  1. Be clear from the start: address AI use in your course syllabus and focus on the educational, rather than punitive, role of grades in your class. Be clear in advance about what AI policies you will implement in your class, what you will consider cheating, and what you will consider as grounds for asking a student to redo or resubmit an assignment.

  2. Ask your student to redo an assignment during office hours under your or your TAs’ supervision.

  3. If you have doubts about specific parts of an assignment, an option you have is to grade the assignment as if certain parts were missing and ask students to rework specific portions of an assignment before you grade their submission in its entirety.

  4. If you’re concerned about specific passages in a student’s writing, ask them to verbally talk you through what they wrote, their thinking behind it, and their reasons for choosing certain arguments. Also, consider asking them to retrace their sources and provide a critique of what they read when writing a specific assignment. 

  5. In some cases, AI use is a form of cheating or plagiarism: while we have no strict guidelines to identify it, you can always refer to UCLA policies on academic integrity.

The bottom line is that AI is here to stay, and students are using it. The challenge is now to find new ways to guide students through difficult or time-consuming forms of writing and critical thinking without demonizing AI or being blind to its increasing presence in your students’ everyday lives.