-------------------------------------------------- Darren Cook darren at dcook.org --------------------------------------------------
(I'd like to hijack Denis's mail; I've changed the subject) > My tests shows that even with as few as 10 amaf simulation per move, > the win rate against the pure-random strategy is almost 100%. I'd thought people were saying that AMAF only helped with weak bots. Once the playouts were cleverer, and/or you were doing a lot of them, it didn't help, or even became a liability. But that is just an impression I'd picked up from day to day reading of this list; and at CG2008 people were still talking about AMAF in their papers (*). Can someone with a strong bot confirm or deny that AMAF is still useful? *: But those papers may have been comparing programs with relatively few playouts (i.e. weak programs), due to the excessive CPU cycles needed to statistically-significantly compare programs with more playouts. ----------------------------------------------- Answers : ----------------------------------------------- My results attest that you have to be prudent whenever using AMAF. Still, i think that MOGO used it for a long time, calling it RAVE (i guess you'd have to look out for a paper :) I never really tried to understand what was going on exactly there. ) My personal tests were exclusively with light-simulations. I used amaf for biasing the score in the tree-phase, switching gradually to a more standard node-scoring policy. The result i had was great for low simulation counts. (I never really tryed to make the thing scale for i wanted something quick to test) Zoe used heavy playout (a project made by Ivan Dubois that i was distantly involved in). Using amaf for hard-pruning showed impressive results. Zoe was about 2000 on Cgos. Although it had some scaling trouble. It didn't used pondering nor did it used zobrist hashing. I think it remains to be shown what zobrist hashing really do for you in the context of go bots. So i really think that AMAF is still promising, although you have to be careful of what you try to do with it. And you can't avoid a lot of testing, including thorough scalability before trying to conclude anything. However i experienced wide issues with scaling. The scaling property proof is very very cpu-consuming, but it can hardly be avoided before stating that something work (or doesn't for that matter). For example MOGO team reported that with really huge number of simulation, they suddenly wanted more randomness in the playout. That makes one wondering if light-playout wouldn't be suddenly efficient with really high numbers of simulations. It'll probably not be known until we got more powerful hardware :) That's why i grew so interested in the "per light-playout" efficiency study. It was cpu-friendly. That's also why i believe that having a really fast light-playout implementation could be something good. But it would have to be rock-solid. That's why i'm so happy with the "standard light playout project". You can get excellent confidence that your conforming implementation is correct. (you can even test non-conforming for that matter :) ) -------------------------------------------------- Darren Cook darren at dcook.org -------------------------------------------------- > I used the following strategy ... though i have no hard-data to give > you, it gave me a fair result. Another thing I've picked up from this list is that when you get that hard, statistically significant data you can frequently be surprised ;-). Darren ----------------------------------------------- Answers : ----------------------------------------------- i think i got the < sign wrong anyway :) I did a lot of testing. BUT i lost the data :) Beside, i would really have trouble figuring what the thing really exactly did at the time, but i tried to give a simple description of something close enough. That's true of course. You need a lot of tests before your statement has any strength. I usually go for 1000 games before assuming that i get an approximate feeling of how things goes. Don seems to feel like 100 000 is more comfortable :) I think Hard-Data is really what matters for modern go programming. The more you have, the more you can be confident about your results. You then have to cross-check with somebody's else data. It's really hard to get convinced that a bot-implementation is conforming to what you think it should do. Idea are of little values by themselves, measures about them are the real treasure. It can hardly be done by a lonely team, and get high enough confidence that all is how it should. I have heard so many reports about something that proved to wrong, although it was thoroughly tested by one team. I have been in this situation so many times myself :) The main issue being bugs. _________________________________________________________________ Email envoyé avec Windows Live Hotmail. Dites adieux aux spam et virus, passez à Hotmail ! C'est gratuit ! http://www.windowslive.fr/hotmail/default.asp_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
