----- Original Message ---- > From: Raymond Wold <[email protected]> > > On 23.07.2010 02:12, Stefan Kaitschick wrote: > > Why should the worst case be the most interesting? > > In a program of this complexity worst case isn't the "true" strength of > > the >program. > > Worst case is basically a bug. > > Given enough time, even an "AI" that chooses entirely randomly from the > legal >moves available will get a win against the best human.
"enough time" would be a rather long, long time. Let us know when you find a win of the random bots against even a modestly strong human. > > What's wrong with looking average play to judge the program? > > Well, it depends on how you measure the average. The typical, putting your > bot >on a go server and letting it play self-selected humans, is not very good, as >surprise at playing style, no knowledge of the program's fundamental flaws, >and >so on, (not to mention humans not necessarily taking a game against a bot as >serious) will bias it towards the program. As I observe on KGS, quite a few humans play more than one game against a given bot. The factor of surprise is not large. It is also possible to review games; by now, the weaknesses of the top bots on KGS are fairly well known. > I would not object to an average of, say, 100 games against one human > opponent >trying his best to win. With an even result under such a series I would >certainly consider the program as strong as the human. > > > And in terms of "interesting" I must say that I find the programs best > > play >much more interesting than it's worst play. > > With best play I don't mean some book play ofcourse, but a fine solution > > to >a tricky problem. > > "Tricky problems" is what a computer does best, a localized search for a >solution, possibly even brute forced. This isn't very impressive to me. > > > Granted, truly awesome play is currently mostly to be seen on 9*9. > > But I've seen some great kills on the big board that any top amateur could >be proud of. > > And how do you deal with confirmation bias? If you look for exceptional >results, do you also look for spectacular failures? What about if a program >gets an occasional brilliant win, but still loses most of the games? > _______________________________________________ > Computer-go mailing list > [email protected] > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > _______________________________________________ Computer-go mailing list [email protected] http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
