Hi Ted, Actually, using a speech interface would put Watson at a *big* disadvantage.
When a contestant selects a clue, the clue appears on the screen in front for all three contestants to read from. At the same time Alex proceeds to read the clue himself. So all of the contestants already know the clue while Alex is reading it... and they are pondering the answer to it during the time it is being read. Once Alex finishes reading the clue, the buzzers are enabled, and the contestants can buzz in. Now if Watson only had a speech recognition interface to get the clue, it (he?) would get the whole clue only once Alex finished reading the it, while the human contestants would have seen the clue on the screen as soon as it was selected. So they would have way more time to ponder the clue, than Watson-with-a-speech-interface would. That seems less fair than the way it was done (i.e., using a text message to Watson -- Watson "sees" the text message as soon as the other contestants see the clue on the screen). The only time we considered a speech interface, was to listen to the answers from the other contestants (to avoid repeating any wrong answers already given by another contestant). But that idea was quickly abandoned because it rarely occurred in real games and was way more trouble than it was worth. -- Sid. On Thu, 2011-02-24 at 05:11 +0000, ted_pedersen wrote: > > > I can't help but draw attention to my very early remarks (in 2009) on > the IBM Jeopardy Challenge ... I think the decision not to use a > speech interface is perhaps at the root of some of the buzzer > controversy, in that if Watson had been using a speech interface, the > perception of it having an advantage in terms of amount of time to > ponder the question might have been minimized. And in fact it sounds > like IBM intends to sell Watson with a speech front end (this is the > parntership with Nuance that was recently announced), so perhaps in > retrospect a speech interface would have resulted in a bit less > discussion of "competition" issues, and would also have provided a > very appealing use case (ie Watson with a speech interface). Of course > the fact that IBM and Nuance are teaming up for the speech interface > suggests that maybe IBM doesn't have the in house speech expertise it > once did ... > > I wonder if IBM ever considered using a speech interface, and would > that have significantly changed the outcome? > > --- In [email protected], Ted Pedersen <duluthted@...> wrote: > > > > IBM has announced a very high profile AI/NLP challenge - it will > > develop a system to take on humans at Jeopardy (a very popular game > > show here in the USA at least). > > > > What's interesting about this is that the IBM system (named Watson) > > will play the game via text. For a company with a long and rich > > tradition in speech processing, I find that a rather revealing > choice, > > and I am certain much will be made of this ("Why can't the bazillion > > dollar computer listen and speak??"). > > > > Below is a short piece from the BBC on this...the title is a bit > harsh > > perhaps (calling it a "stunt" :), but make no mistake this is very > > much about marketing (in addition to the science part). IBM is a big > > player in the supercomputing market, which is quite lucrative and > > promises to remain that way for some time to come... > > > > http://news.bbc.co.uk/2/hi/technology/8021774.stm > > > > Machine versus man stunt by IBM > > By Maggie Shiels > > Technology reporter, BBC News, Silicon Valley > > > > The computer company that has in the past taken on the grandmasters > of > > chess is now turning its attention to the famed trivia quiz show > > Jeopardy. > > > > In a head to head challenge of man versus machine, IBM will pit a > > supercomputer code-named Watson against human contestants. > > > > Watson is a new question-answering system based on natural language. > > > > "The aim is to get Watson to think and interact in human terms," > IBM's > > Dr David Ferrucci told BBC News. > > > > "It will try to understand a users question and intent and > understand > > it at a rudimentary level and provide and accurate and confident > > answer." > > > > For the last two years scientists have been working on perfecting > the > > system that will drive Watson, named after IBM founder Thomas J > Watson > > Snr. > > > > "The most challenging aspect of this is that Watson has to know what > > it knows with utmost confidence. > > > > "Otherwise if it buzzes in and gets the answer wrong that is bad on > > Jeopardy because you lose money and lose the game," explained Dr > > Ferrucci, an IBM artificial intelligence researcher and team leader > on > > the project. > > > > "New approach" > > > > IBM is no stranger to such high profile stunts. > > > > Back in 1997 a computer called Deep Blue defeated world chess > champion > > Garry Kasparov in another battle of human versus machine. > > > > To compete at chess, the company built an extremely fast computer > that > > could calculate 200 million chess moves per second based on a fixed > > problem. > > > > This time round, Watson is looking to solve an open-ended problem > that > > requires an entirely new approach. > > > > To win on the show, contestants need a good knowledge across a range > > of topics and the ability to quickly analyse subtle meaning, irony, > > riddles and other complexities that humans excel at but computers do > > not. > > > > The rules of the game will be slightly modified for this battle. > > > > Watson will be given the questions as electronic text while the > human > > contestants will both see the text and hear it spoken by the show's > > host Alex Trebek. > > > > The computer will answer in a synthesised voice and choose follow up > > questions. For the taping of the show, Watson will not be hooked up > to > > the internet but will only be able to draw from what it has "read", > or > > processed and indexed, before the show. > > > > "It requires critical thinking. It requires a whole lot more than > > knowledge alone," said Jeopardy's executive producer Harry Friedman. > > > > "When people hear about this project, they say isn't this Google on > > steroids? No it's not." > > > > IBM and the show's producers are hoping grand champion Ken Jennings > to > > take part. He won Jeopardy 74 consecutive times and collected $2.52m > > (£1.71m) in 2004. > > > > "Ken is the best and no-one gets close to his achievement. This is > > just a great test for Watson," said Dr Ferrucci. > > > > "Mapping meaning to meaning" > > > > Behind this effort to put Watson on a big stage and have some fun, > > there is a serious business goal. > > > > "This is about giving enterprise faster access to more precise > > answers," explained Dr Ferrucci. > > > > "While business might be satisfied with the classic search approach > > because it gives you plenty to read, the thing that haunts you when > > you are making a decision is what is out there that I am not > reading. > > > > "Watson will look at all the information, including the most remote > > that might impinge on any answer you might make building more and > more > > confidence into that content to help support your decision." > > > > IBM's director of research John Kelly said "This represents a new > > level of communications between computers and human beings. > > > > "What we are doing is creating a system that will be able to be > > applied to all sorts of applications in the world and essentially > cut > > the time to find answers to very difficult problems." > > > > Dr Ferrucci said that in the past the "human has had to adapt to the > > computer working with classic search" and that Watson changes > > everything by "getting the computer to work in human terms". > > > > "Rather than mapping words to words which is easy for a computer to > > do, it now has to map meaning to meaning and that is a much bigger > > challenge." > > > > Here are some additional links with more details from IBM.... > > > > http://www.research.ibm.com/deepqa/index.shtml > > > > http://www.youtube.com/watch?v=3e22ufcqfTs > > > > Enjoy, > > Ted > > > > -- > > Ted Pedersen > > http://www.d.umn.edu/~tpederse > > > > > > >
