Ken's written several books. His wrote his first book - Brainiac - after his 2004(?) run on Jeopardy. It's not only about his run, but also about the world of triva competition, including stuff like College Quiz Bowl. I bought the ebook version of it last week, and I'm about half-way through it.
It's a very fun book, chock-full of revelations about the world of competitive quizzing (they are revelations to me at least). In the collegiate Quiz Bowl, for example, you are allowed to interrupt the questioner as soon as you think you have the answer. There are situations when competitors buzz in after the first word(!) and get the answer right! Ken says that much of this culture is about methodically covering each major source of trivia questions one by one - US presidents, US states and their official symbols (flowers, animals, mottos, etc), Civil War, World War, wines, opera, etc, etc. There's like a baseline of information that any competitive trivia person would know cold. At the top-most level, the winners are often decided by who can be fastest in recall, fastest with his buzzer, and of course who happens to have covered the categories in question just a little bit better than the others. According to Ken, the reason this interrupt-the-questionner business isn't done on Jeopardy is that it wouldn't make for good TV - the audience would find it hard to hang on to what's going on. Ken also mentions that pretty often the people who come on Jeopardy are people who have been active on the trivia quiz circuit in the past. Ken himself apparently has been preparing for trivia since he was like 5 or 6. So now you know, Sid, what kind of competition you guys were up against :-). It's a good book - I recommend it. It isn't Watson- or AI-related of course, but I'm finding it a nice little tangent to go off on. -Bano On Thu, Feb 24, 2011 at 2:44 PM, Siddharth Patwardhan <[email protected]>wrote: > > > Ken Jennings is a good writer. Here's another article he wrote about > competing against Watson: > > http://www.slate.com/id/2284721/pagenum/all > > Good stuff! :) > > I believe he has even written a book. > > -- Sid. > > > On Thu, 2011-02-24 at 08:37 -0600, Ted Pedersen wrote: > > Ken Jennings has some recent thoughts on possible game modifications. > > He doesn't go so far as to suggest wearing a blindfold, but he does > > have an interesting suggestion for how to possibly correct the buzzer > > issues that involves the use of speech recognition... > > > > http://ken-jennings.com/blog/?p=2554 > > > > Here's the relevant excerpt... > > > > > > But I can think of two better approaches to leveling the playing field > > buzzer-wise, should anyone want to try a Watson-like challenge in > > their basement or garage sometime. > > > > 1. Remove the Extra Human. Jeopardy! viewers know that there are > > > four humans involved in each clue: the host, and the three > > players. But in fact, a fifth human is just as important: the > > anonymous staff member (I think it might still be researcher > > Ryan Haas) who manually activates the buzzer after Trebek > > finishes reading each clue. If you buzz in before Anonymous > > Human activates the buzzers, you get locked out for a fraction > > of a second and are pretty much doomed. When A.H. flips his > > switch, less than a syllable after Trebek gets done reading > > the clue, some lights around the game board turn on to let you > > know it’s time to buzz. In theory you’re supposed to wait for > > these lights; in practice, the really good contestants try to > > anticipate them. A.H. is very good and very consistent, but he > > is human, so there are tiny variations in his timing. These > > variations never affected Watson (who can pounce so fast it > > doesn’t need to anticipate) but could submarine its human > > opponents (who will never beat Watson if they don’t > > anticipate, and rarely even if they do). If the A.H. were > > replaced–say, by voice recognition software keying precisely > > off of Alex’s voice–then humans could try to master that > > now-fixed interval. We might even have a fighting chance > > against Watson. > > 2. Make Watson “Anticipate the Lights.” Alternately, you could > > > have Watson anticipate as well, rather than pouncing. Get rid > > of the in-studio lights and the electronic “go” signal for the > > computer. Feed Watson the clue electronically, but also give > > it speech-recognition capabilities. It would be able to track > > Trebek’s voice as the end of the question approached, just > > like Brad and I were doing, and then try to time its buzz a > > split second after Trebek got done. Because of A.H.’s slight > > variability, it could buzz as optimally as possible but it > > would still be a little early sometimes and a little late at > > other times–just like a good human buzzer. > > > > On Thu, Feb 24, 2011 at 8:02 AM, Ted Pedersen <[email protected]> > > wrote: > > Hi Sid, > > > > I left out the best part of my modified plan. The only people > > that see the clues are the TV audience. The contestants only > > hear them. :) Imagine Ken Jennings and Brad Rutter with > > blindfolds, if you will. So this whole issue of when a > > contestant gets the question and has the complete question to > > process seems to be equalized (it is when Alex finishes > > reading the clue according to each contestant's speech > > processing mechanism). This might even make the buzzer issue a > > bit more interesting, in that the contestants would have to > > predict when the clue from Alex is actually finished....versus > > when he takes one of his dramatic pauses. As I understand it > > now a light goes on when he finishes reading the clue which > > opens up the buzzer window, but in this case I guess in this > > scenario the light would still go on (in effect, since someone > > needs to decide when Alex is done reading) but none of the > > contestants would actually see it. But, as I understand it > > Watson doesn't see the light anyway, so we sort of equalize > > things here by blinding the humans (taking away vision) and > > forcing Watson to listen (requiring hearing). So all > > contestants would hear and speak, but not see/read... > > > > This way we seem to have all contestants using the same > > channels of information. I don't know what a good Jeopardy > > player would think of simply relying on hearing the clues - > > that would probably be very challenging since if you mis-hear > > a clue you have no chance to recover by reading it, but since > > we know speech is likely to be hard for a system like Watson > > that might actually equalize the "interface" issues a bit so > > that both kinds of contestants are using the same channels.... > > > > This actually has me thinking about another issue - what about > > a human Jeopardy contestant that is blind or deaf? How could > > you make the game "fair" so that they could compete on an > > equal footing with other contestants? I don't watch Jeopardy > > enough to know how or if they handle this, but it seems like a > > really important thing to consider (from a human participation > > point of view). > > > > BTW, as I wax philosophical I should say thanks to Sid for all > > the great insights into Watson, that's really helpful in > > coming to understand these issues and think about them a > > little more. > > > > Fun stuff, > > Ted > > > > On Wed, Feb 23, 2011 at 11:51 PM, Siddharth Patwardhan > > <[email protected]> wrote: > > > > > > > > Hi Ted, > > > > Actually, using a speech interface would put Watson at > > a *big* > > disadvantage. > > > > When a contestant selects a clue, the clue appears on > > the screen in > > front for all three contestants to read from. At the > > same time Alex > > proceeds to read the clue himself. So all of the > > contestants already > > know the clue while Alex is reading it... and they are > > pondering the > > answer to it during the time it is being read. Once > > Alex finishes > > reading the clue, the buzzers are enabled, and the > > contestants can buzz > > in. > > > > Now if Watson only had a speech recognition interface > > to get the clue, > > it (he?) would get the whole clue only once Alex > > finished reading the > > it, while the human contestants would have seen the > > clue on the screen > > as soon as it was selected. So they would have way > > more time to ponder > > the clue, than Watson-with-a-speech-interface would. > > That seems less > > fair than the way it was done (i.e., using a text > > message to Watson -- > > Watson "sees" the text message as soon as the other > > contestants see > > the clue on the screen). > > > > The only time we considered a speech interface, was to > > listen to the > > answers from the other contestants (to avoid repeating > > any wrong answers > > already given by another contestant). But that idea > > was quickly > > abandoned because it rarely occurred in real games and > > was way more > > trouble than it was worth. > > > > -- Sid. > > > > > > > > > > On Thu, 2011-02-24 at 05:11 +0000, ted_pedersen wrote: > > > > > > > > > I can't help but draw attention to my very early > > remarks (in 2009) on > > > the IBM Jeopardy Challenge ... I think the decision > > not to use a > > > speech interface is perhaps at the root of some of > > the buzzer > > > controversy, in that if Watson had been using a > > speech interface, the > > > perception of it having an advantage in terms of > > amount of time to > > > ponder the question might have been minimized. And > > in fact it sounds > > > like IBM intends to sell Watson with a speech front > > end (this is the > > > parntership with Nuance that was recently > > announced), so perhaps in > > > retrospect a speech interface would have resulted in > > a bit less > > > discussion of "competition" issues, and would also > > have provided a > > > very appealing use case (ie Watson with a speech > > interface). Of course > > > the fact that IBM and Nuance are teaming up for the > > speech interface > > > suggests that maybe IBM doesn't have the in house > > speech expertise it > > > once did ... > > > > > > I wonder if IBM ever considered using a speech > > interface, and would > > > that have significantly changed the outcome? > > > > > > --- In [email protected], Ted Pedersen > > <duluthted@...> wrote: > > > > > > > > IBM has announced a very high profile AI/NLP > > challenge - it will > > > > develop a system to take on humans at Jeopardy (a > > very popular game > > > > show here in the USA at least). > > > > > > > > What's interesting about this is that the IBM > > system (named Watson) > > > > will play the game via text. For a company with a > > long and rich > > > > tradition in speech processing, I find that a > > rather revealing > > > choice, > > > > and I am certain much will be made of this ("Why > > can't the bazillion > > > > dollar computer listen and speak??"). > > > > > > > > Below is a short piece from the BBC on this...the > > title is a bit > > > harsh > > > > perhaps (calling it a "stunt" :), but make no > > mistake this is very > > > > much about marketing (in addition to the science > > part). IBM is a big > > > > player in the supercomputing market, which is > > quite lucrative and > > > > promises to remain that way for some time to > > come... > > > > > > > > http://news.bbc.co.uk/2/hi/technology/8021774.stm > > > > > > > > Machine versus man stunt by IBM > > > > By Maggie Shiels > > > > Technology reporter, BBC News, Silicon Valley > > > > > > > > The computer company that has in the past taken on > > the grandmasters > > > of > > > > chess is now turning its attention to the famed > > trivia quiz show > > > > Jeopardy. > > > > > > > > In a head to head challenge of man versus machine, > > IBM will pit a > > > > supercomputer code-named Watson against human > > contestants. > > > > > > > > Watson is a new question-answering system based on > > natural language. > > > > > > > > "The aim is to get Watson to think and interact in > > human terms," > > > IBM's > > > > Dr David Ferrucci told BBC News. > > > > > > > > "It will try to understand a users question and > > intent and > > > understand > > > > it at a rudimentary level and provide and accurate > > and confident > > > > answer." > > > > > > > > For the last two years scientists have been > > working on perfecting > > > the > > > > system that will drive Watson, named after IBM > > founder Thomas J > > > Watson > > > > Snr. > > > > > > > > "The most challenging aspect of this is that > > Watson has to know what > > > > it knows with utmost confidence. > > > > > > > > "Otherwise if it buzzes in and gets the answer > > wrong that is bad on > > > > Jeopardy because you lose money and lose the > > game," explained Dr > > > > Ferrucci, an IBM artificial intelligence > > researcher and team leader > > > on > > > > the project. > > > > > > > > "New approach" > > > > > > > > IBM is no stranger to such high profile stunts. > > > > > > > > Back in 1997 a computer called Deep Blue defeated > > world chess > > > champion > > > > Garry Kasparov in another battle of human versus > > machine. > > > > > > > > To compete at chess, the company built an > > extremely fast computer > > > that > > > > could calculate 200 million chess moves per second > > based on a fixed > > > > problem. > > > > > > > > This time round, Watson is looking to solve an > > open-ended problem > > > that > > > > requires an entirely new approach. > > > > > > > > To win on the show, contestants need a good > > knowledge across a range > > > > of topics and the ability to quickly analyse > > subtle meaning, irony, > > > > riddles and other complexities that humans excel > > at but computers do > > > > not. > > > > > > > > The rules of the game will be slightly modified > > for this battle. > > > > > > > > Watson will be given the questions as electronic > > text while the > > > human > > > > contestants will both see the text and hear it > > spoken by the show's > > > > host Alex Trebek. > > > > > > > > The computer will answer in a synthesised voice > > and choose follow up > > > > questions. For the taping of the show, Watson will > > not be hooked up > > > to > > > > the internet but will only be able to draw from > > what it has "read", > > > or > > > > processed and indexed, before the show. > > > > > > > > "It requires critical thinking. It requires a > > whole lot more than > > > > knowledge alone," said Jeopardy's executive > > producer Harry Friedman. > > > > > > > > "When people hear about this project, they say > > isn't this Google on > > > > steroids? No it's not." > > > > > > > > IBM and the show's producers are hoping grand > > champion Ken Jennings > > > to > > > > take part. He won Jeopardy 74 consecutive times > > and collected $2.52m > > > > (£1.71m) in 2004. > > > > > > > > "Ken is the best and no-one gets close to his > > achievement. This is > > > > just a great test for Watson," said Dr Ferrucci. > > > > > > > > "Mapping meaning to meaning" > > > > > > > > Behind this effort to put Watson on a big stage > > and have some fun, > > > > there is a serious business goal. > > > > > > > > "This is about giving enterprise faster access to > > more precise > > > > answers," explained Dr Ferrucci. > > > > > > > > "While business might be satisfied with the > > classic search approach > > > > because it gives you plenty to read, the thing > > that haunts you when > > > > you are making a decision is what is out there > > that I am not > > > reading. > > > > > > > > "Watson will look at all the information, > > including the most remote > > > > that might impinge on any answer you might make > > building more and > > > more > > > > confidence into that content to help support your > > decision." > > > > > > > > IBM's director of research John Kelly said "This > > represents a new > > > > level of communications between computers and > > human beings. > > > > > > > > "What we are doing is creating a system that will > > be able to be > > > > applied to all sorts of applications in the world > > and essentially > > > cut > > > > the time to find answers to very difficult > > problems." > > > > > > > > Dr Ferrucci said that in the past the "human has > > had to adapt to the > > > > computer working with classic search" and that > > Watson changes > > > > everything by "getting the computer to work in > > human terms". > > > > > > > > "Rather than mapping words to words which is easy > > for a computer to > > > > do, it now has to map meaning to meaning and that > > is a much bigger > > > > challenge." > > > > > > > > Here are some additional links with more details > > from IBM.... > > > > > > > > http://www.research.ibm.com/deepqa/index.shtml > > > > > > > > http://www.youtube.com/watch?v=3e22ufcqfTs > > > > > > > > Enjoy, > > > > Ted > > > > > > > > -- > > > > Ted Pedersen > > > > http://www.d.umn.edu/~tpederse > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Ted Pedersen > > http://www.d.umn.edu/~tpederse > > > > > > > > > > -- > > Ted Pedersen > > http://www.d.umn.edu/~tpederse > > >
