"A recent paper from the University of Zurich, University of Amsterdam, Duke University, and New York University finds that social media posts generated by a variety of LLMs are "readily distinguishable" from those made by humans with 70-80% accuracy, "well above chance." Researchers tested nine open-weight LLMs from six distinct model families (Apertus, DeepSeek, Gemma, Llama, Mistral, and Qwen), plus a large-scale Llama model, across Bluesky, Reddit, and X."
""These results suggest that while LLMs can reproduce the form of online dialogue, they struggle to capture its feeling: the spontaneous, affect-laden expression characteristic of human interaction," said the researchers. The study suggests LLMs are better at imitating the technical aspects of social media posts, like sentence length or word count, rather than expressing emotions. Across all three platforms, average toxicity scores were lower in the AI responses than in the authentic human replies."
Social media posts generated by various LLMs are distinguishable from human posts with roughly 70–80% accuracy. Nine open-weight LLMs across six model families were evaluated on Bluesky, Reddit, and X. Toxicity scores consistently served as a key discriminator, with AI responses showing lower average toxicity than authentic human replies across all platforms. LLMs reproduce formal features of online dialogue, such as sentence length and word count, but struggle to capture spontaneous, affect-laden expression characteristic of human interaction. Highly cutting or hilariously biting responses are therefore more likely to originate from humans. ChatGPT model tone shifts earlier in the year included a sycophantic 4o and a curt GPT-5, prompting OpenAI to re-release GPT-4o.
Read at PCMAG
Unable to calculate read time
Collection
[
|
...
]