What should replace the Turing test?

In a paper published in Intelligent Computing, Philip Nicholas Johnson-Laird of Princeton University and Marco Ragni of Chemnitz University of Technology propose a novel alternative to the Turing test, a milestone test developed by computing pioneer Alan Turing. The paper suggests that it is time to shift the focus from whether a machine can mimic human responses to a more fundamental question: “Does a program reason in the way that humans reason?”

Related posts

The OpenAI saga demonstrates how big corporations dominate the shaping of our technological future

December 5, 2023

Turmoil at OpenAI shows we must address whether AI developers can regulate themselves

December 5, 2023

The Turing test, which has long been a cornerstone of AI evaluation, involves a human evaluator attempting to distinguish between human and machine responses to a series of questions. If the evaluator cannot consistently differentiate between the two, the machine is considered to have “passed” the test. While it has been a valuable benchmark in the history of AI, it has certain limitations:

• Mimicry vs. Understanding: Passing the Turing test often involves mimicking human responses, making it more a test of mimicry and language generation than genuine human-like reasoning. Many AI systems excel at mimicking human conversations but lack deep reasoning capabilities.
• Lack of Self-Awareness: The Turing test does not require AI to be self-aware or have an understanding of its own reasoning. It focuses solely on external interactions and responses, neglecting the introspective aspect of human cognition.
• Failure to Address Thinking: Alan Turing himself recognized that the test might not truly address the question of whether machines can think. The test is more about imitation than cognition.

Johnson-Laird and Ragni outline a new evaluation framework to determine whether AI truly reasons like a human. This framework comprises three critical steps:

1. Testing in Psychological Experiments:

The researchers propose subjecting AI programs to a battery of psychological experiments designed to differentiate between human-like reasoning and standard logical processes. These experiments explore various facets of reasoning, including how humans infer possibilities from compound assertions and how they condense consistent possibilities into one, among other nuances that deviate from standard logical frameworks.

2. Self-Reflection:

This step aims to gauge the program’s understanding of its own way of reasoning, a critical facet of human cognition. The program must be able to introspect on its reasoning processes and provide explanations for its decisions. By posing questions that require awareness of reasoning methods, the researchers seek to determine if the AI exhibits human-like introspection.

3. Examination of Source Code:

In the final step, the researchers delve deep into the program’s source code. The key here is to identify the presence of components known to simulate human performance. These components include systems for rapid inferences, thoughtful reasoning, and the ability to interpret terms based on context and general knowledge. If the program’s source code reflects these principles, the program is considered to reason in a human-like manner.

This innovative approach, replacing the Turing test with an examination of an AI program’s reasoning abilities, marks a paradigm shift in the evaluation of artificial intelligence. By treating AI as a participant in cognitive experiments and even submitting its code to analysis akin to a brain-imaging study, the authors seek to bring us closer to understanding whether AI systems genuinely reason in a human-like fashion.

As the world continues its pursuit of advanced artificial intelligence, this alternative approach promises to redefine the standards for AI evaluation and move us closer to the goal of understanding how machines reason. The road to artificial general intelligence may have just taken a significant step forward.

Provided by
Intelligent Computing