Can You Spot Fake News? Try JudgeGPT — and Discover Why AI-Generated Does Not Mean False

A person assesses digital news through separate layers of content-origin cues and supporting evidence, reflecting uncertainty about authorship and truthfulness.

Can you recognise an AI-generated news story? More importantly, can you tell whether it is actually true? JudgeGPT invites you to test your judgement — and discover why these are two fundamentally different questions.

Two questions that are easy to confuse

Imagine reading a plausible news article. You may ask who wrote it: a human journalist or a generative AI system? Separately, you may ask whether its claims are accurate or misleading. AI-generated does not automatically mean false, and human-written does not automatically mean true.

JudgeGPT, developed as part of a research programme by Alexander Loth, Martin Kappes and Marc-Oliver Pahl, separates these judgements instead of collapsing them into a single real-versus-fake question. The research interface assesses source attribution, perceived veracity and topic familiarity.

What does the research show?

In the peer-reviewed 2026 paper Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News, the authors analyse 2,318 judgements from 1,054 participants across material generated by six language models. Within that study, participants could not reliably distinguish machine-generated from human-written text. Self-reported topic expertise was associated with better judgement, and accuracy declined with repeated evaluations. These results concern the study’s particular stimuli and protocol; they do not mean that every person always performs at chance.

The practical implication is important: teaching people simply to look for an ‘AI writing style’ is unlikely to be sufficient. Trustworthy information requires evidence, context, source assessment and, where possible, provenance mechanisms.

Try it yourself — then reflect

  • Open JudgeGPT and evaluate the news fragments presented.
  • Keep two separate questions in mind: Who or what generated this? and Is the information trustworthy?
  • Notice when a confident judgement depends on writing style rather than evidence.
  • Discuss which cues helped you — and which may have misled you.

JudgeGPT is a research platform, not a certification tool for verifying arbitrary news. The associated project documentation describes research data collection; consult the application’s own notices before submitting personal information.

Explore the scientific background

Read Can Humans Tell? (ACM WebSci Companion 2026), the open-source JudgeGPT project and the Dresden workshop synthesis. JudgeGPT was also demonstrated during the Dresden workshop; explore the recordings.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *