Ken Jennings has some recent thoughts on possible game modifications. He doesn't go so far as to suggest wearing a blindfold, but he does have an interesting suggestion for how to possibly correct the buzzer issues that involves the use of speech recognition...
http://ken-jennings.com/blog/?p=2554 Here's the relevant excerpt... But I can think of two better approaches to leveling the playing field buzzer-wise, should anyone want to try a Watson-like challenge in their basement or garage sometime. 1. *Remove the Extra Human.* *Jeopardy!* viewers know that there are four humans involved in each clue: the host, and the three players. But in fact, a fifth human is just as important: the anonymous staff member (I think it might still be researcher Ryan Haas) who manually activates the buzzer after Trebek finishes reading each clue. If you buzz in before Anonymous Human activates the buzzers, you get locked out for a fraction of a second and are pretty much doomed. When A.H. flips his switch, less than a syllable after Trebek gets done reading the clue, some lights around the game board turn on to let you know it’s time to buzz. In theory you’re supposed to wait for these lights; in practice, the really good contestants try to anticipate them. A.H. is very good and very consistent, but *he is human*, so there are tiny variations in his timing. These variations never affected Watson (who can pounce so fast it doesn’t need to anticipate) but could submarine its human opponents (who will never beat Watson if they don’t anticipate, and rarely even if they do). If the A.H. were replaced–say, by voice recognition software keying precisely off of Alex’s voice–then humans could try to master that now-fixed interval. We might even have a fighting chance against Watson. 2. *Make Watson “Anticipate the Lights.”* Alternately, you could have Watson anticipate as well, rather than pouncing. Get rid of the in-studio lights and the electronic “go” signal for the computer. Feed Watson the clue electronically, but also give it speech-recognition capabilities. It would be able to track Trebek’s voice as the end of the question approached, just like Brad and I were doing, and then try to time its buzz a split second after Trebek got done. Because of A.H.’s slight variability, it could buzz as optimally as possible but it would still be a little early sometimes and a little late at other times–just like a good human buzzer. On Thu, Feb 24, 2011 at 8:02 AM, Ted Pedersen <[email protected]> wrote: > Hi Sid, > > I left out the best part of my modified plan. The only people that see the > clues are the TV audience. The contestants only hear them. :) Imagine Ken > Jennings and Brad Rutter with blindfolds, if you will. So this whole issue > of when a contestant gets the question and has the complete question to > process seems to be equalized (it is when Alex finishes reading the clue > according to each contestant's speech processing mechanism). This might even > make the buzzer issue a bit more interesting, in that the contestants would > have to predict when the clue from Alex is actually finished....versus when > he takes one of his dramatic pauses. As I understand it now a light goes on > when he finishes reading the clue which opens up the buzzer window, but in > this case I guess in this scenario the light would still go on (in effect, > since someone needs to decide when Alex is done reading) but none of the > contestants would actually see it. But, as I understand it Watson doesn't > see the light anyway, so we sort of equalize things here by blinding the > humans (taking away vision) and forcing Watson to listen (requiring > hearing). So all contestants would hear and speak, but not see/read... > > This way we seem to have all contestants using the same channels of > information. I don't know what a good Jeopardy player would think of simply > relying on hearing the clues - that would probably be very challenging since > if you mis-hear a clue you have no chance to recover by reading it, but > since we know speech is likely to be hard for a system like Watson that > might actually equalize the "interface" issues a bit so that both kinds of > contestants are using the same channels.... > > This actually has me thinking about another issue - what about a human > Jeopardy contestant that is blind or deaf? How could you make the game > "fair" so that they could compete on an equal footing with other > contestants? I don't watch Jeopardy enough to know how or if they handle > this, but it seems like a really important thing to consider (from a human > participation point of view). > > BTW, as I wax philosophical I should say thanks to Sid for all the great > insights into Watson, that's really helpful in coming to understand these > issues and think about them a little more. > > Fun stuff, > Ted > > On Wed, Feb 23, 2011 at 11:51 PM, Siddharth Patwardhan < > [email protected]> wrote: > >> >> >> Hi Ted, >> >> Actually, using a speech interface would put Watson at a *big* >> disadvantage. >> >> When a contestant selects a clue, the clue appears on the screen in >> front for all three contestants to read from. At the same time Alex >> proceeds to read the clue himself. So all of the contestants already >> know the clue while Alex is reading it... and they are pondering the >> answer to it during the time it is being read. Once Alex finishes >> reading the clue, the buzzers are enabled, and the contestants can buzz >> in. >> >> Now if Watson only had a speech recognition interface to get the clue, >> it (he?) would get the whole clue only once Alex finished reading the >> it, while the human contestants would have seen the clue on the screen >> as soon as it was selected. So they would have way more time to ponder >> the clue, than Watson-with-a-speech-interface would. That seems less >> fair than the way it was done (i.e., using a text message to Watson -- >> Watson "sees" the text message as soon as the other contestants see >> the clue on the screen). >> >> The only time we considered a speech interface, was to listen to the >> answers from the other contestants (to avoid repeating any wrong answers >> already given by another contestant). But that idea was quickly >> abandoned because it rarely occurred in real games and was way more >> trouble than it was worth. >> >> -- Sid. >> >> >> On Thu, 2011-02-24 at 05:11 +0000, ted_pedersen wrote: >> > >> > >> > I can't help but draw attention to my very early remarks (in 2009) on >> > the IBM Jeopardy Challenge ... I think the decision not to use a >> > speech interface is perhaps at the root of some of the buzzer >> > controversy, in that if Watson had been using a speech interface, the >> > perception of it having an advantage in terms of amount of time to >> > ponder the question might have been minimized. And in fact it sounds >> > like IBM intends to sell Watson with a speech front end (this is the >> > parntership with Nuance that was recently announced), so perhaps in >> > retrospect a speech interface would have resulted in a bit less >> > discussion of "competition" issues, and would also have provided a >> > very appealing use case (ie Watson with a speech interface). Of course >> > the fact that IBM and Nuance are teaming up for the speech interface >> > suggests that maybe IBM doesn't have the in house speech expertise it >> > once did ... >> > >> > I wonder if IBM ever considered using a speech interface, and would >> > that have significantly changed the outcome? >> > >> > --- In [email protected], Ted Pedersen <duluthted@...> wrote: >> > > >> > > IBM has announced a very high profile AI/NLP challenge - it will >> > > develop a system to take on humans at Jeopardy (a very popular game >> > > show here in the USA at least). >> > > >> > > What's interesting about this is that the IBM system (named Watson) >> > > will play the game via text. For a company with a long and rich >> > > tradition in speech processing, I find that a rather revealing >> > choice, >> > > and I am certain much will be made of this ("Why can't the bazillion >> > > dollar computer listen and speak??"). >> > > >> > > Below is a short piece from the BBC on this...the title is a bit >> > harsh >> > > perhaps (calling it a "stunt" :), but make no mistake this is very >> > > much about marketing (in addition to the science part). IBM is a big >> > > player in the supercomputing market, which is quite lucrative and >> > > promises to remain that way for some time to come... >> > > >> > > http://news.bbc.co.uk/2/hi/technology/8021774.stm >> > > >> > > Machine versus man stunt by IBM >> > > By Maggie Shiels >> > > Technology reporter, BBC News, Silicon Valley >> > > >> > > The computer company that has in the past taken on the grandmasters >> > of >> > > chess is now turning its attention to the famed trivia quiz show >> > > Jeopardy. >> > > >> > > In a head to head challenge of man versus machine, IBM will pit a >> > > supercomputer code-named Watson against human contestants. >> > > >> > > Watson is a new question-answering system based on natural language. >> > > >> > > "The aim is to get Watson to think and interact in human terms," >> > IBM's >> > > Dr David Ferrucci told BBC News. >> > > >> > > "It will try to understand a users question and intent and >> > understand >> > > it at a rudimentary level and provide and accurate and confident >> > > answer." >> > > >> > > For the last two years scientists have been working on perfecting >> > the >> > > system that will drive Watson, named after IBM founder Thomas J >> > Watson >> > > Snr. >> > > >> > > "The most challenging aspect of this is that Watson has to know what >> > > it knows with utmost confidence. >> > > >> > > "Otherwise if it buzzes in and gets the answer wrong that is bad on >> > > Jeopardy because you lose money and lose the game," explained Dr >> > > Ferrucci, an IBM artificial intelligence researcher and team leader >> > on >> > > the project. >> > > >> > > "New approach" >> > > >> > > IBM is no stranger to such high profile stunts. >> > > >> > > Back in 1997 a computer called Deep Blue defeated world chess >> > champion >> > > Garry Kasparov in another battle of human versus machine. >> > > >> > > To compete at chess, the company built an extremely fast computer >> > that >> > > could calculate 200 million chess moves per second based on a fixed >> > > problem. >> > > >> > > This time round, Watson is looking to solve an open-ended problem >> > that >> > > requires an entirely new approach. >> > > >> > > To win on the show, contestants need a good knowledge across a range >> > > of topics and the ability to quickly analyse subtle meaning, irony, >> > > riddles and other complexities that humans excel at but computers do >> > > not. >> > > >> > > The rules of the game will be slightly modified for this battle. >> > > >> > > Watson will be given the questions as electronic text while the >> > human >> > > contestants will both see the text and hear it spoken by the show's >> > > host Alex Trebek. >> > > >> > > The computer will answer in a synthesised voice and choose follow up >> > > questions. For the taping of the show, Watson will not be hooked up >> > to >> > > the internet but will only be able to draw from what it has "read", >> > or >> > > processed and indexed, before the show. >> > > >> > > "It requires critical thinking. It requires a whole lot more than >> > > knowledge alone," said Jeopardy's executive producer Harry Friedman. >> > > >> > > "When people hear about this project, they say isn't this Google on >> > > steroids? No it's not." >> > > >> > > IBM and the show's producers are hoping grand champion Ken Jennings >> > to >> > > take part. He won Jeopardy 74 consecutive times and collected $2.52m >> > > (£1.71m) in 2004. >> > > >> > > "Ken is the best and no-one gets close to his achievement. This is >> > > just a great test for Watson," said Dr Ferrucci. >> > > >> > > "Mapping meaning to meaning" >> > > >> > > Behind this effort to put Watson on a big stage and have some fun, >> > > there is a serious business goal. >> > > >> > > "This is about giving enterprise faster access to more precise >> > > answers," explained Dr Ferrucci. >> > > >> > > "While business might be satisfied with the classic search approach >> > > because it gives you plenty to read, the thing that haunts you when >> > > you are making a decision is what is out there that I am not >> > reading. >> > > >> > > "Watson will look at all the information, including the most remote >> > > that might impinge on any answer you might make building more and >> > more >> > > confidence into that content to help support your decision." >> > > >> > > IBM's director of research John Kelly said "This represents a new >> > > level of communications between computers and human beings. >> > > >> > > "What we are doing is creating a system that will be able to be >> > > applied to all sorts of applications in the world and essentially >> > cut >> > > the time to find answers to very difficult problems." >> > > >> > > Dr Ferrucci said that in the past the "human has had to adapt to the >> > > computer working with classic search" and that Watson changes >> > > everything by "getting the computer to work in human terms". >> > > >> > > "Rather than mapping words to words which is easy for a computer to >> > > do, it now has to map meaning to meaning and that is a much bigger >> > > challenge." >> > > >> > > Here are some additional links with more details from IBM.... >> > > >> > > http://www.research.ibm.com/deepqa/index.shtml >> > > >> > > http://www.youtube.com/watch?v=3e22ufcqfTs >> > > >> > > Enjoy, >> > > Ted >> > > >> > > -- >> > > Ted Pedersen >> > > http://www.d.umn.edu/~tpederse >> > > >> > >> > >> > >> > >> > >> >> >> > > > > -- > Ted Pedersen > http://www.d.umn.edu/~tpederse > -- Ted Pedersen http://www.d.umn.edu/~tpederse
