Re: [computer-go] Reference Montecarlo TreeDecision Bot.
I am not good at term definition, but I would say RAVE is the algorithm extending AMAF in MCTS, including how to accumulate the counts at each node (trivial extension even though implementation can be tricky), how to combine with UCT (or other move choice), and how to integrate with priors (based on expert knowledge or previous learning). Sylvain On Tue, Dec 15, 2009 at 10:31 PM, Mark Boon tesujisoftw...@gmail.comwrote: I took AMAF as the process to consider all the moves regardless when they were played in the sequence (although a slight discount for later in the sequence seems to help a little) whereas RAVE is using an undefined method to favour some nodes over others prior to expanding them. The reason (as far as I understood so far) they get confused is because a popular method to use in RAVE is in fact using AMAF values. Mark On Tue, Dec 15, 2009 at 2:12 AM, Magnus Persson magnus.pers...@phmp.se wrote: Quoting Petr Baudis pa...@ucw.cz: On Mon, Dec 14, 2009 at 10:37:24PM -0800, Peter Drake wrote: It's easy to get confused -- different researchers use the terms slightly differently. They both gather data on moves other than a move made from the current board configuration. I would say that AMAF stores statistics on every move played from any position, while RAVE only stores info on moves played from descendants of the current position. Consequently, AMAF uses a global table, whereas RAVE data must be stored at every node. I guess that is a good definition; I assume this difference to arise from the fact whether you use tree or flat MC, so for me, AMAF in tree always means from descendants of the current position. Instead, to me AMAF is the data collected, while RAVE is the way to apply the data in the node urgency computation (which furthermore splits to what I call for myself Sylvain Gelly's RAVE vs David Silver's RAVE, of course...). This also how I have interpreting AMAF and RAVE after being confused initially thinking it was just two names for one thing. I think it's because I haven't seen this approach evolve and I'm not too familiar with the pre-RAVE AMAF, so perhaps I underestimate how revolutionary the descendants only idea was. AMAF was first used with programs that did not build a tree. Perhaps this is why Peter Drake makes this interpretation. When I implemented AMAF in Valkyria it was always self evident that descendants only is only the only good way of making use of it, since search was so deep that the positions cannot be compared. Best Magnus ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
[computer-go] Biasing nodes according to pattern gammas
Petr Baudis wrote: I wonder now, do you use separate set of gammas for simulations and node biasing? Since I've found that the performance seems very bad if I don't include some time-expensive features, since the gammas are then very off; I will probably simply generate two gamma sets, but perhaps it's enough to do some trick like merging features by computing weighted (geometric?) averages? Rémi Coulom answered: I learn two sets of gammas separately for the two sets of features. I don't get it. Why do you need two sets one for the tree and one for the playouts? To learn gammas, I use a database of games. The patterns compete, some of them win. This is computed using a Bradley-Terry model. At that time moves are just moves, not tree moves or simulation moves. When that offline learned model 'best' fits the moves played (55000 games x 100 moves each in my case) I am done, I have a set of gamma values. I use these for playouts and biasing the tree. What else are you doing? How do you compute a set for the playouts and a set for the tree? Do you adjust gamma values one by one playing games? Of course, I guess this is not very useful for 9x9 that's why I took the (probably wrong) decision to work in 19x19 only. Jacques. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Biasing nodes according to pattern gammas
Jacques Basaldúa wrote: Petr Baudis wrote: I wonder now, do you use separate set of gammas for simulations and node biasing? Since I've found that the performance seems very bad if I don't include some time-expensive features, since the gammas are then very off; I will probably simply generate two gamma sets, but perhaps it's enough to do some trick like merging features by computing weighted (geometric?) averages? Rémi Coulom answered: I learn two sets of gammas separately for the two sets of features. I don't get it. Why do you need two sets one for the tree and one for the playouts? To learn gammas, I use a database of games. The patterns compete, some of them win. This is computed using a Bradley-Terry model. At that time moves are just moves, not tree moves or simulation moves. When that offline learned model 'best' fits the moves played (55000 games x 100 moves each in my case) I am done, I have a set of gamma values. I use these for playouts and biasing the tree. What else are you doing? How do you compute a set for the playouts and a set for the tree? Do you adjust gamma values one by one playing games? The problem is that I am not using the same set of features for biasing the tree, and for playouts. Playouts only use fast light features. The tree part uses slow complex features. In particular, I use patterns of radius 3 and 4 in the tree, and only radius 3 in the playouts. When 3x3 patterns are learnt together with r=4 patterns, they get different gammas. Rémi ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Biasing nodes according to pattern gammas
Since patterns are correlated with each other, the gamma sets are specific to the pattern set used. Since more patterns are used in the tree, itrequires a separate set of gammas than the in-tree search. Sent from my iPhone On Dec 16, 2009, at 2:50 PM, Jacques Basaldúa jacq...@dybot.com wrote: Petr Baudis wrote: I wonder now, do you use separate set of gammas for simulations and node biasing? Since I've found that the performance seems very bad if I don't include some time-expensive features, since the gammas are then very off; I will probably simply generate two gamma sets, but perhaps it's enough to do some trick like merging features by computing weighted (geometric?) averages? Rémi Coulom answered: I learn two sets of gammas separately for the two sets of features. I don't get it. Why do you need two sets one for the tree and one for the playouts? To learn gammas, I use a database of games. The patterns compete, some of them win. This is computed using a Bradley-Terry model. At that time moves are just moves, not tree moves or simulation moves. When that offline learned model 'best' fits the moves played (55000 games x 100 moves each in my case) I am done, I have a set of gamma values. I use these for playouts and biasing the tree. What else are you doing? How do you compute a set for the playouts and a set for the tree? Do you adjust gamma values one by one playing games? Of course, I guess this is not very useful for 9x9 that's why I took the (probably wrong) decision to work in 19x19 only. Jacques. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Gongo: Go in Go
Oops, you're right. Here it is with -server: Plug-and-Go refbot:17857 CRef bot (-O3) 12500 Gongo 1 Java bot: 1 CRef bot (no optimization) 5882 On Tue, Dec 15, 2009 at 1:40 PM, Mark Boon tesujisoftw...@gmail.com wrote: The relative values look about right. But I remember getting much higher numbers. Did you run the Java versions with or without the -server parameter? Mark On Mon, Dec 14, 2009 at 11:00 PM, Brian Slesinsky br...@slesinsky.org wrote: Okay, I added a few more timings (playouts / second, very rough): Plug-and-Go refbot: 14700 CRef bot (-O3) 12500 Gongo 1 Java bot: 6500 CRef bot (no optimization) 5882 Note that Gongo and Plug-and-Go are using different board data structures than the others. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Gongo: Go in Go
That's pretty impressive for the go language if this is an apples to apples comparison. Is it pretty much? On Wed, Dec 16, 2009 at 9:50 PM, Brian Slesinsky br...@slesinsky.orgwrote: Oops, you're right. Here it is with -server: Plug-and-Go refbot:17857 CRef bot (-O3) 12500 Gongo 1 Java bot: 1 CRef bot (no optimization) 5882 On Tue, Dec 15, 2009 at 1:40 PM, Mark Boon tesujisoftw...@gmail.com wrote: The relative values look about right. But I remember getting much higher numbers. Did you run the Java versions with or without the -server parameter? Mark On Mon, Dec 14, 2009 at 11:00 PM, Brian Slesinsky br...@slesinsky.org wrote: Okay, I added a few more timings (playouts / second, very rough): Plug-and-Go refbot:14700 CRef bot (-O3) 12500 Gongo 1 Java bot: 6500 CRef bot (no optimization) 5882 Note that Gongo and Plug-and-Go are using different board data structures than the others. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
