What is the Turing test and is it still useful?

The Turing test is the trial Alan Turing proposed in 1950 to replace the question "can machines think?" with a verifiable one: if an interrogator conversing only in writing cannot tell a machine from a person, the original question becomes idle. In the paper Turing does not call it a test but the imitation game, and he describes it with three participants: "It is played with three people, a man (A), a woman (B), and an interrogator (C)". Three quarters of a century later that trial has become the wrong question to ask of a system headed for production, because it measures how human a model can seem, which is a property of style, and not how well it solves the task you are buying it for. For decision makers the right question is a different one, and it already has a technical name: evals.

What the 1950 paper actually says

The source is Computing Machinery and Intelligence, published in Mind, vol. LIX, no. 236, October 1950. The single numerical prediction Turing allows himself concerns the year 2000: in about fifty years' time, he writes, it will be possible to program computers to play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning. That is a prediction about how easy deception gets, not about thought. Searle's Chinese room, 1980, argues exactly this: manipulating symbols convincingly does not demonstrate understanding.

The test has been passed, and nothing changed

Cameron Jones and Benjamin Bergen ran two pre-registered tests in the original three-party setup, five minutes of conversation in parallel with a person and a system, published in PNAS in May 2026. The result: GPT-4.5 was judged to be the human 73% of the time, more often than the actual human. The condition matters more than the number: that 73% holds only with a prompt instructing the model to adopt a plausible humanlike persona, and without that prompt the same model drops to 38%. So the test also measures how good the prompt author is, and how long the conversation runs: a third study with fifteen-minute games brings the two best models back to 56% and 59%. Meanwhile the system cards of recent models report SWE-bench, GPQA Diamond and math reasoning evaluations: in those cards the Turing test is not among the metrics of progress.

An enterprise example

An insurance organization is evaluating a conversational assistant for first-level support on motor policies. The question "does it seem human?" produces an exciting demo and no decision. The criterion that ships it is an eval on its own cases: a hundred real requests already closed by agents, on which you measure answers correct against the policy text, cases handed to a human when data was missing, and how often the system invented a coverage that does not exist. The threshold is set beforehand. An assistant that writes stiffly but gets, say, 2% of answers wrong ships; one indistinguishable from an agent that gets 9% wrong does not, because here every error is a complaint.

Why it matters for decision makers

The Turing test remains history of ideas, but as a buying criterion it is misleading: it rewards fluency, which is precisely the quality that makes hallucinations expensive, because an error stated well passes review. The operational question is "how well does this system do my job, measured on my own cases", and the answer is built with evals on a company dataset, not with a public benchmark the model may already have seen in training. For the same reason the AGI debate does not help: it concerns breadth of capability in the abstract, while whoever invests is setting a threshold on one task.