ChatGPT broke the Turing test - the race is on for new ways to assess AI
Updated: Oct 21
ChatGPT Broke the Turing Test: The Race Is On For New Ways To Assess AI
Did ChatGPT break the Turing test? Not exactly. A 2025 pre-registered study found that GPT-4.5, when prompted to adopt a humanlike persona, was judged to be the human 73% of the time in a three-party Turing test. That is a striking result, but researchers still argue the test measures imitation, not understanding.
What is the Turing Test?
Understanding the Turing Test is like stepping back into the golden age of computer science.
Origins and Purpose
Conceived by Alan Turing, a British mathematician and computer scientist, the test was designed to evaluate a machine's ability to exhibit intelligent behavior that's indistinguishable from a human's. Could we, Turing wondered, create machines that think like us?
The Classic Turing Test
In the classic Turing Test, an individual engages in a natural language conversation with another human and a machine, without knowing which is which. If the individual can't reliably distinguish between the human and the machine, the machine passes the test. But that was yesterday. The future, it seems, belongs to the likes of ChatGPT.
The Rise of ChatGPT
Fast forward to the present day, where we are witnessing a revolution in artificial intelligence, courtesy of ChatGPT.
Creation and Purpose
Developed by OpenAI, ChatGPT is an AI model that utilizes machine learning to generate human-like text. It's designed to improve communication and efficiency in various fields, from customer service to entertainment.
Did ChatGPT Break the Turing Test?
ChatGPT's rise pushed the Turing test back into the headlines. A 2025 pre-registered study reported that GPT-4.5 was judged to be the human 73% of the time in a standard three-party test when it was prompted to adopt a humanlike persona. That study tested specific models, not ChatGPT as a product, and it is a single study.
What Does the Evidence Actually Show?
The most cited result comes from researchers Cameron Jones and Benjamin Bergen. In two randomised, controlled, pre-registered experiments, participants spent five minutes talking to a human and to one of four systems at the same time, then judged which partner was human. The paper on arXiv reports the following:
GPT-4.5, prompted to adopt a humanlike persona, was judged to be the human 73% of the time, significantly more often than the real human participants.
LLaMa-3.1-405B, with the same prompt, was judged human 56% of the time, which was not significantly different from the humans it was compared with.
The baseline systems, ELIZA and GPT-4o, scored well below chance at 23% and 21%.
The authors describe this as the first empirical evidence that an artificial system passes a standard three-party Turing test. Notice the conditions: a persona prompt, a short conversation and a specific set of models. The baseline GPT-4o score shows how different the results were without that setup.
This is one result from one research group, and the test itself has always been controversial. It measures how convincingly a machine imitates a person in conversation. It does not tell us whether the system understands what it says, whether its answers are accurate, or whether it is safe to rely on.
If you want to know how reliable ChatGPT is in everyday use, our guide to how accurate ChatGPT's information is is more useful than any imitation test. You can also compare assistants in our ranked guide to the best AI chatbots.
Implications of ChatGPT Passing the Turing Test
The claim that AI can now pass the Turing test opens up questions as well as possibilities.
The AI Evolution
AI is evolving rapidly, becoming more ingrained in our lives. From smart assistants to predictive algorithms, AI is reshaping the world around us. ChatGPT is a testament to this continuous evolution.
Potential Risks and Concerns
However, with great power comes great responsibility. There are legitimate concerns about AI, from data privacy issues to job displacement. If an AI can pass the Turing Test, where do we draw the line?
The Need for New Assessment Methods
As AI grows more sophisticated, it's becoming clear that the Turing Test may not be enough. We need new ways to assess AI.
Why the Turing Test Isn't Enough
The Turing Test measures whether an AI can impersonate a human, but it doesn't evaluate understanding, consciousness, or empathy. As AI becomes more integrated into society, we need benchmarks that measure these aspects.
New Frontiers in AI Assessment
The quest is on for new methods to assess AI, focusing on ethical use, transparency, and interpretability. The goal is not just to create intelligent machines, but also responsible and understandable ones.
What Lies Ahead
The future of AI, represented by ChatGPT and beyond, is both intriguing and challenging.
ChatGPT's Future and Beyond
ChatGPT's success heralds a future where AI could become a prevalent part of our lives. But it's also a reminder of the need for responsible AI development and deployment.
AI: An Uncharted Journey
As we move forward in the AI journey, we must navigate the balance between technological advancement and ethical considerations.
Final thoughts
The advent of ChatGPT breaking the Turing Test has ignited the race for new ways to assess AI. It's an exciting journey, filled with endless possibilities and challenges. As we navigate this new frontier, we must ensure we're creating AI that is not just intelligent but also ethical and beneficial for all.
FAQs: ChatGPT broke the Turing test
What is the Turing test? The Turing test is a measure of a machine's ability to exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human. Alan Turing proposed it in 1950 as an “imitation game”.
Did ChatGPT pass the Turing test? A 2025 pre-registered study by Cameron Jones and Benjamin Bergen found GPT-4.5 was judged to be the human 73% of the time in a three-party test when prompted to adopt a humanlike persona. It tested specific models, in five-minute conversations, not ChatGPT as a product.
Does passing the Turing test mean AI understands anything? No. The test measures whether a machine can imitate human conversation, not whether it understands, is conscious or has empathy. Critics have argued this since the test was proposed.
Why do we need new ways to assess AI? Because imitation is not the same as understanding, reliability or safety. Newer evaluations look at accuracy, reasoning, transparency and responsible use.
What lies ahead for AI like ChatGPT? Expect better tools, closer links to everyday software and more debate about trust and regulation. Treat claims about AI milestones as results from specific tests, not final verdicts.
Here are 10 facts about the Turing Test:
Origins: The Turing Test was proposed by British mathematician and computer scientist Alan Turing in 1950 as a way to measure a machine's ability to exhibit intelligent behavior.
Name: The test is named after Alan Turing, who is considered one of the founding figures of computer science.
Initial Concept: The original concept was introduced in Turing's paper, "Computing Machinery and Intelligence," where he proposed the idea of an "imitation game."
Test Design: In the Turing Test, a human judge engages in a natural language conversation with a human and a machine without knowing which is which. If the judge cannot reliably distinguish the machine from the human, the machine is said to have passed the test.
Turing’s Prediction: Turing predicted that by the year 2000, machines would be able to fool humans 30% of the time during a five-minute conversation.
First Machine to Pass: In 2014, a chatbot named Eugene Goostman reportedly became the first machine to pass the Turing Test by convincing 33% of human judges that it was a 13-year-old Ukrainian boy.
Controversy: Despite Eugene Goostman's achievement, there's an ongoing debate about whether it truly passed the Turing Test. Critics argue that the chatbot relied on tricks and deception rather than demonstrating genuine intelligence.
Purpose: The Turing Test is not a definitive test of intelligence. It is, rather, a test of a machine's ability to mimic human-like conversation.
Limitations: The Turing Test has limitations and has been criticized for being inadequate in truly measuring machine intelligence. It doesn't account for self-awareness, consciousness, or empathy, attributes often associated with intelligence.
Future of Testing AI: As AI continues to advance, there's an increasing need for new ways to test and measure AI capabilities that go beyond the Turing Test. This could include understanding context, exhibiting creativity, and more.
References
Last reviewed on 21 October 2026.
