On 23.07.2010 02:12, Stefan Kaitschick wrote:
Why should the worst case be the most interesting?
In a program of this complexity worst case isn't the "true" strength
of the program.
Worst case is basically a bug.
Given enough time, even an "AI" that chooses entirely randomly from the
legal moves available will get a win against the best human.
What's wrong with looking average play to judge the program?
Well, it depends on how you measure the average. The typical, putting
your bot on a go server and letting it play self-selected humans, is not
very good, as surprise at playing style, no knowledge of the program's
fundamental flaws, and so on, (not to mention humans not necessarily
taking a game against a bot as serious) will bias it towards the program.
I would not object to an average of, say, 100 games against one human
opponent trying his best to win. With an even result under such a series
I would certainly consider the program as strong as the human.
And in terms of "interesting" I must say that I find the programs best
play much more interesting than it's worst play.
With best play I don't mean some book play ofcourse, but a fine
solution to a tricky problem.
"Tricky problems" is what a computer does best, a localized search for a
solution, possibly even brute forced. This isn't very impressive to me.
Granted, truly awesome play is currently mostly to be seen on 9*9.
But I've seen some great kills on the big board that any top amateur
could be proud of.
And how do you deal with confirmation bias? If you look for exceptional
results, do you also look for spectacular failures? What about if a
program gets an occasional brilliant win, but still loses most of the games?
_______________________________________________
Computer-go mailing list
[email protected]
http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
