Turing-like tests for language and vision demonstrate a narrowing gap between humans and AI

Researchers compared responses from 636 people and 37 AI systems across language and vision tasks, then ran 72,191 Turing-like judgments with 1,916 human judges. In these constrained tasks, current systems increasingly passed as human; simple AI judges distinguished human from machine responses better than people did.

The authors provide an open benchmark and data, while carefully stating that the tests are not equivalent to a general test of intelligence. The work raises concrete questions about people’s ability to identify synthetic communication.