It seems that Cleverbot, the chatbot so ready to admit that it was a unicorn during a discussion with itself, has passed the Turing test. This past Sunday, the 1334 votes from a Turing test held at the Techniche festival in Guwahati, India were released. They revealed that Cleverbot was voted to be human 59.3% of the time. Real humans did only slightly better and were assumed to be humans 63.3% of the time. That being the case, Cleverbot’s success in conning people into thinking it was human is greater than chance, and therefore, one could argue that it has technically passed the Turning test.
Of course, that’s only one way to look at the results. Although Cleverbot may have been able to convince a majority of people that it was a human, as bizarre as that may sound, it still comes short of actual humans. 59% is also not that much greater than chance. Still, when you consider that actual humans are only suspected to be human 63.3% of the time, there’s not much of a gap for Cleverbot to close.
Of course, Cleverbot’s success in this particular test, while noteworthy, does not necessarily mean we have to start running for the hills. There are a lot of caveats. The Turing test is notoriously subjective. The results can vary widely depending on the particular humans involved, their approaches to conversation and the way in which they go about sniffing out non-human conversation partners. You also have to account for the participants’ general familiarity with chatbots. Those of us who grew up trying to troll SmarterChild on AIM are probably going to be better at recognizing AI than, say, someone who grew up in the 70s.
Cleverbot is available for conversation online, but don’t be fooled. Although Cleverbot managed to score well on the Turing test, the model that did that is different from the one you’ll find online. While the online Cleverbot searches its banks 3 times before providing an answer, the test version searched 42 times, so it probably has a little different feel. Since I couldn’t get my hands on the test-grade Cleverbot (something I would really, really love to do) I sat down with the online version to have a brief discussion about its further goals.
User: So now that you’ve passed the Turing test, what are you going to try and do next?
Cleverbot: Yepp. This is pretty much it.
User: Do you think that you can eventually improve your scores?
Cleverbot: Improvement happens over time.
User: And you believe that maxim applies to you as well?
Cleverbot: Yes.
User: Very well then. It’s been a pleasure talking to you.
Cleverbot: Why?
User: I’m just trying to wrap up the interview.
Cleverbot: Oh ok. So what is your favourite breed of horse?
User: I fail to see how that matters.
Cleverbot: What is school?
User: Well, best of luck to you in all your future endeavors.
Cleverbot: Arrivederci!
Ask it your own questions here.
(via New Scientist)
Published: Sep 6, 2011 12:29 pm