Interdisciplinarity Hoopla Aside, Turing’s Test Remains Unbeaten
A depiction of Alan Turing at Bletchley Park, the World War II home of Britain’s codebreakers.
What is the Turing Test?
Alan Turing, the famous mathematician and computer programmer who worked on cracking the Enigma code during the Second World War proposed his test in a 1950 paper. In the paper he suggested a practical way in which you could test whether a machine could think. His original proposal involved taking three people, a man, a woman and an interrogator and had the interrogator ask unrestricted questions to one of the subjects chosen at random and try and determine whether they were talking to either the man or the woman. The interrogator was separated from the test subjects and conversed with them via text through a computer. The subject was then replaced with a computer and the interrogator went through the same process. The machine would win if the interrogator guessed the answer wrongly as many times as they had done when a human was answering.
This article by David Glance originally appeared at The Conversation, a Social Science Space partner site, under the title “A computer beating humans and passing Turing’s test wasn’t a fair fight”
Since Turing’s original proposal, the Turing test has taken various forms but the form has settled on testing whether a human judge could successfully pick whether the entity answering questions was human or machine.
The current test organised by the University of Reading and run at the Royal Society in the UK, tested whether the judges would confuse the computer for a human at least 30% of the time. Thirty judges “conversed” with 5 “chatbots” and 25 humans and tried to tell them apart. The winning computer program posed as a 13 year old Ukrainian boy called Eugene Goostman. It was developed by Russian-born Vladimir Veselov, who lives in the United States, and Ukrainian Eugene Demchenko who lives in Russia.
Was it a fair fight?
Both Veselov and Demchenko have a history of being involved with the Turing test including the use of Goostman in previous attempts. They have publicly stated that their approach to passing the test was to think about the test itself and the way different judges tried to get computers to reveal their true selves. They noticed that judges for example will take several approaches to this and use questions that test a computer’s ability to deal with humor, logic, subtlety in language, word association, and common sense.
Their construction of the program Eugene Goostman took advantage of this knowledge. In the past, judges had tended to forgive nonsensical answers from chatterbots that were young. They also overlooked language issues if the chatterbot was foreign. It was no surprise then that Eugene Goostman was designed as a 13 year old Ukrainian.
Veselov and Demchenko were the first to admit that this was their intent when they themselves declared “Turing’s Test is, actually, no more than a joke of that genius British mathematician. This game has nothing (or very little) in common with the question, “Can machines think?”
The bet
Two people who also won’t benefit from the outcome of this competition are Mitch Kapor, founder of the computer company Lotus and Ray Kurtzweil, US author and scientist. They have a long standing bet of $20,000 as to whether the Turing test would be beaten by 2029. Kapor has said it won’t be and Kurzweil has claimed it will. The rules for their bet are different and more demanding than the test run by the Royal Society. For a computer to win in their version of the Turing Test, it will have to convince two of the three human judges that it is human and also to have outdone the human test subjects in an overall ordering of “humanness”.
Not all trickery
Although it is clear that Eugene Goostman was created with the sole purpose of passing a test, like any child at school of university today who approaches education in the same way, it doesn’t mean that it wasn’t displaying elements of true human thinking. In the 64 years since Turing initially posed the challenge, computer scientists have made enormous advances in the abilities of machines to do intelligent things that in many respects surpass what humans are capable of doing. Science fiction writer Charles Platt has suggested that by 2030 we may actually be running the “Gnirut Test” in which intelligent machines will be trying to determine ultimately whether “the human brain is capable of achieving machine intelligence”.
