ChatGPT published a new classifier tool yesterday to detect AI-generated language, which proved to be, at best, imperfect within a few hours. It turns out that there may be no easy fix for recognizing generative AI, whether in text or graphics.
Sebastian Raschka, a Lightning AI artificial intelligence (AI) and machine learning (ML) researcher, began testing the OpenAI Text Classifier on ChatGPT with text fragments from a book he wrote in 2015. The tool reported that it was “unclear” whether the book’s preface was authored by AI; nevertheless, the introduction was “probably AI,” and a paragraph from the first chapter was “likely” AI.
Worryingly, the tool classified the opening page of Shakespeare’s Macbeth as follows:
“The classifier believes the text was most likely created by AI.”