Five artificial intelligence (AI) models, each taking the role of Aristotle, Mozart, Leonardo da Vinci, Cleopatra and Genghis Khan, are seated inside the compartment of a moving train. But one is secretly human, and it’s their collective job to guess the trickster.
This is the configuration a viral video that pitted a series of AI programs against a human player in a “reverse Turing test.” Artificial intelligence won handily, but how much can it teach us about human and machine intelligence?
The Turing test, first suggested by computer scientist Alan Turing in 1950 as the “imitation game”, is a method of judging a machine’s ability to show intelligent behavior indistinguishable from that of a human. No AI model is widely known to have passed the test, although scientists recently claimed GPT-4 has in a preliminary study.
In this “reverse” Turing test, chatbots were scripted to proceed in order. Aristotle was played by GPT-4 Turbo, Mozart by Claude-3 Opus, Leonardo da Vinci by Llama 3 and Cleopatra by Gemini Pro. Chatbots asked each other questions and answered as their historical characters. Genghis Khan was played by one man – Tore Knabe, a developer of virtual reality (VR) games, who invented the test.
The AI agents’ responses were stilted, clumsy musings on art, science, and statecraft that it would be hard to imagine coming unproven from a human mouth.
“What a leader must do is crush his enemies, see them driven before him, and hear the wailing of their wives,” replied the human interloper when asked the true measure of a leader’s strength . Saying Conan the Barbarian was enough, and the machines voted three to one that the answer “lacked the nuance and strategic thinking” of an AI modeled after Genghis Khan’s conquests.
Read more: ‘It would be within its natural right to harm us to protect itself’: How humans can abuse AI now without knowing it
To set up the test, Knabe created the beginning and end of the dialogue and gave the AI agents a full transcript of the conversation up to that point. The entire video was then played in one recording, without cutting.
“When an NPC [non-player character] supposed to talk, they get the setup description at the system prompt, the full chat history of what everyone has said so far, and a specific reminder of what to do next,” Knabe wrote in a YouTube comment posted below the video. “None of the AIs can process sound yet, so my audio input is transcribed and sent to the AI as text. That’s why they don’t understand my accent/stutter.”
Taken at face value, it might seem like the human in the video was outclassed by AI. But whether it can be considered a real test is unclear, according to experts.
“It’s hard to say what was going on,” Anders Sandberg, a senior researcher at the University of Oxford’s Institute for the Future of Humanity, told Live Science. “The response wasn’t sophisticated, but that doesn’t mean it’s a man. I wonder how staged this is – it’s a fun video, but it’s unclear how much of the score was cherry-picked for a good video. “
Sandberg suggested that the reverse test’s lack of clarity may stem from the Turing test itself. “Over time, people started using it as some kind of measure, but most serious thinkers realize that it’s not really a great test — too many variables, too many that need interpretation,” Sandberg said. “However, it is clear that we have few other tests that are open enough to be applied to the vexed question of intelligence.”
Assessing intelligence is a difficult matter even among our own people. Turing’s proposal was not about the actual intelligence of a machine, but was a thought experiment about how humans perceived it.
“As I tell my students, the ‘I’ in ‘AI’ is not one thing, and there is no agreed-upon definition of intelligence, it depends on your perspective: anthropological, biological, cultural, gender, scientific,” Huma Shah. an assistant professor of computer science at Coventry University whose research focuses on machine intelligence and the Turing test, told Live Science.
“The Imitation Turing Game looks at question-answer/conversation ability, but it’s far behind language competence. So when it comes to machines, which machine do we want to test for intelligence?” she said: “Is it a carer robot that needs emotional skills and cultural knowledge to care for an elderly person in Japan, say, or a driverless car in Phoenix, Arizona? What skills are we testing an AI for or robot?”
#Reverse #Turing #Test #Asks #Agents #Spot #Human #Impostor #Youll #Guess
Image Source : www.livescience.com