How can humans detect AI writing? These Penn researchers have some tips

If a group of people is presented with a body of text, will they be able to spot the human-written passages over the AI-generated ones?

The answer is, well — maybe.

AI detection work is at the center of a University of Pennsylvania study in its School of Engineering and Applied Science. Chris Callison-Burch, an associate professor in the Department of Computer and Information Science, along with Liam Dugan and Daphne Ippolito, two Ph.D. students in the program, tested this question over the course of about a year.

The team drafted a test that asked users to identify which writing was done by a computer. They used language modeling AI tools GPT-2 and GPT-3, and tested two groups of people — some who were given tools and incentives to look for AI writing, and others who were not. A recently published paper “Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Text” summarizes their findings.

Take the test

Dugan told Technical.ly AI models have progressed immensely, even since just a few years ago in 2020.

“The models were good, but they weren’t like, scary good,” he said. “They could detect news articles, but they were easy to check. It’s progressed so quickly from there, from even just one year ago.”

The research mainly happened in 2021 and 2022, using GPT-2 and GPT-3. The team submitted their paper a few months before ChatGPT rolled out in November 2022. Though what ChatGPT can do is more advanced from GPT-2 and GPT-3, the same methods of spotting the AI mostly ring true, Dugan said.

The first thing the researchers found is that some people are just better at identifying the AI-written passages than others, Dugan said. Those who are close readers or advanced readers tended to spot some of the errors that clued them in to which writing was computer generated more than others, because they spotted factual errors or discontinuities.

The second thing they found was that those who were incentivized to identify the AI writing could be trained to get better at it.

“I think that’s really hopeful, Dugan said. “When people are pessimistic about if this is the end of creative writing, we point to this. Even when people couldn’t detect that well, understanding what these models can and can’t do, and the errors they tend to make, is important.”

So if you’re aiming to add some AI literacy to your tool box, Dugan and the other researchers advise three tactics:

• This stage of AI writing doesn’t tend to make grammatical errors, so don’t focus on that. Five years ago, AI chat bots were writing clunky, awkward sentences, but today’s bots are “good — almost too good,” Dugan said.
• Focus less on word choice and more on background knowledge and common sense errors. A chat bot tasked with writing an article about Ben Franklin might reference his time as president. Those factual inaccuracies or contradictions often showed up in AI writing, the researchers found.
• Take note of how the entire piece of writing comes together. Dugan explained that AI chat bots are focused on putting one word after another, rather than forming a complete, fleshed-out idea. They don’t do so well with long-term ideas.

“Does it waffle or say irrelevant things?” he said “We found they’d start to say things that were totally irrelevant to the original objective.”

Dugan credited his colleague Ippolito — currently a research assistant at Google — as a “pioneering figure” in detection. Dugan has a few years left on his Ph.D., and though second-year students are often still shaking out what their research might focus on, he’s feeling pretty narrowed in on AI detection. He’s excited about the ways language modeling will continue to evolve, and in the different career opportunities it may present in a few year’s time.

“The detection problem is interesting on its own as a question,” he said. “Is it possible for AI to be so good that it’s completely indistinguishable? Or are there ways of going around that?”