Software Development
AI / Universities

How can humans detect AI writing? These Penn researchers have some tips

Turns out artificial intelligence isn't the best at crafting long-form ideas.

Daphne Ippolito (left) and Liam Dugan. (Courtesy photos)

This guest post is a part of Universities Month 2023 in’s editorial calendar.

If a group of people is presented with a body of text, will they be able to spot the human-written passages over the AI-generated ones?

The answer is, well — maybe.

AI detection work is at the center of a University of Pennsylvania study in its School of Engineering and Applied Science. Chris Callison-Burch, an associate professor in the Department of Computer and Information Science, along with Liam Dugan and Daphne Ippolito, two Ph.D. students in the program, tested this question over the course of about a year.

The team drafted a test that asked users to identify which writing was done by a computer. They used language modeling AI tools GPT-2 and GPT-3, and tested two groups of people — some who were given tools and incentives to look for AI writing, and others who were not. A recently published paper “Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Text” summarizes their findings.

Take the test

Dugan told AI models have progressed immensely, even since just a few years ago in 2020.

“The models were good, but they weren’t like, scary good,” he said. “They could detect news articles, but they were easy to check. It’s progressed so quickly from there, from even just one year ago.”

The research mainly happened in 2021 and 2022, using GPT-2 and GPT-3. The team submitted their paper a few months before ChatGPT rolled out in November 2022. Though what ChatGPT can do is more advanced from GPT-2 and GPT-3, the same methods of spotting the AI mostly ring true, Dugan said.

The first thing the researchers found is that some people are just better at identifying the AI-written passages than others, Dugan said. Those who are close readers or advanced readers tended to spot some of the errors that clued them in to which writing was computer generated more than others, because they spotted factual errors or discontinuities.

The second thing they found was that those who were incentivized to identify the AI writing could be trained to get better at it.

“I think that’s really hopeful, Dugan said. “When people are pessimistic about if this is the end of creative writing, we point to this. Even when people couldn’t detect that well, understanding what these models can and can’t do, and the errors they tend to make, is important.”

So if you’re aiming to add some AI literacy to your tool box, Dugan and the other researchers advise three tactics:

  • This stage of AI writing doesn’t tend to make grammatical errors, so don’t focus on that. Five years ago, AI chat bots were writing clunky, awkward sentences, but today’s bots are “good — almost too good,” Dugan said.
  • Focus less on word choice and more on background knowledge and common sense errors. A chat bot tasked with writing an article about Ben Franklin might reference his time as president. Those factual inaccuracies or contradictions often showed up in AI writing, the researchers found.
  • Take note of how the entire piece of writing comes together. Dugan explained that AI chat bots are focused on putting one word after another, rather than forming a complete, fleshed-out idea. They don’t do so well with long-term ideas.

“Does it waffle or say irrelevant things?” he said “We found they’d start to say things that were totally irrelevant to the original objective.”

Dugan credited his colleague Ippolito — currently a research assistant at Google — as a “pioneering figure” in detection. Dugan has a few years left on his Ph.D., and though second-year students are often still shaking out what their research might focus on, he’s feeling pretty narrowed in on AI detection. He’s excited about the ways language modeling will continue to evolve, and in the different career opportunities it may present in a few year’s time.

“The detection problem is interesting on its own as a question,” he said. “Is it possible for AI to be so good that it’s completely indistinguishable? Or are there ways of going around that?”

Companies: University of Pennsylvania
Series: Universities Month 2023

Knowledge is power!

Subscribe for free today and stay up to date with news and tips you need to grow your career and connect with our vibrant tech community.


BarCamp Philly: Share your inner geek at this year's unconference

'We are just trying to survive, which is different from living': Afghan refugee Ghulam Danish on his journey toward thriving

As Guru turns 10, CEO Rick Nucci reflects on evolutions in AI and workplace culture

One uCity Square is officially open, housing life science companies and research labs

Technically Media