>> I have done precisely this. The reports of scalability death are >> greatly exaggerated, as you can see from the attached graph. To >> avoid self play benchmarks which are misleading, I tested Pachi >> against Fuego 1.1. ... >> > Well this gives a biased solution. Wrong sample so to speak. Fuego > will not create complex semeais and hard-to-read ishi-no-shita nakade > shapes i.e opponent that puts no pressure to known problems . So you > prove that against an opponent who does not play like human you do > scale. But as you advance up the ladder of human players these small > issues tend to pop-up more often.
+1. This is exactly what was nagging at my mind as I read this thread. Hideki's hypothesis (as I understood it) is that increased CPU stops helping at a certain point because, against very strong human opponents: * there is systematic error in the evaluations; * doubling CPU doesn't help remove this error (I think Hideki is also saying that this point, where CPU stops helping, is rapidly approaching and that therefore this is a practical problem.) Playing a tournament between programs that all have this same problem just means you get some very high level games where they all avoid the types of games that will challenge them. Darren P.S. But, not wanting to be totally negative, here is one idea: * Collect a set of problem positions where the current strongest programs mess up. * Use each of these as the start position, instead of an empty board, in your scaling study. (In other words, don't let the programs play the style of game they want to play, but force them to start outside their comfort zone.) If Hideki is right, you should see very little improvement for each doubling. This has its own bias: the selection of problems. However, it could make for an interesting and educational grudge match: In the red corner, those who agree that scaling won't get you much further, hunt for some really great problem positions. In the blue corner, those who disagree, scrounge even more CPU power to see if just one more doubling will give a breakthrough. _______________________________________________ Computer-go mailing list [email protected] http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
