The text you’re reading right now was typed into a Google Doc by a human being.
But that may not be the case with text you encounter elsewhere.
Teachers have deployed detection software hoping to catch plagiarists in the act.
On the other hand, the human-written TOEFL essays were misclassified 61% of the time.
One specific detector flagged 97.8% of the TOEFL essays as AI generated.
All seven detectors flagged 18 of the 91 TOEFL essays as AI generated.
Perplexity is kind of a proxy measure for variability or randomness in a given text.
Non-native English writers have previously been shown to have a less rich vocabulary and use less rich grammar.
This, to the GPT detectors, makes it seem like it was written by an AI.
This time, they used AI to see if detection software correctly identified it as AI generated.
The team used ChatGPT to generate responses to the 2022-2023 US college admission essay prompts.
Similar results were seen when the team had ChatGPT write scientific abstracts.
‘Now that’s a sour meatball!’
The sun, delightful though it tastes, is battery-powered and contains a startling toxin: Wolf teeth."
A major GPT detector suggested there was “a moderate likelihood of being written by AI.”
I then assessed five of the freely available detectors available online and used by the Stanford team.
It also fooled three of the other five detectors.
This might enable a more robust and equitable approach.
And the detectors may be helpful in ways we’re yet to see.
