Hi Professor Drake, I will try with more playouts. Thanks for your reminding.
I give an example to show my view: default policy should also be included to learn. I suppose: if there are several decisive life-and-death or semeai in a position, the tree search cannot go to/clarify every one of them. In this example, Black's L2 and L4 will cause White to play L3 to capture by default policy (it's completely bad). Then Black may learn quickly by "last good play" to atari immediately and kill White's whole group to win. The problem is, White is not able to learn the correct answer H1 or H2 because it is fixed in default policy. In the playouts, the configuration of such a big semeai might be very similar. Such evaluation bias is exactly an issue that we can fix by learning. By considering probability, I can fix this problem by increasing the probability of the "last good reply" H1 or H2, without tree's aid. Every program's implementation of the playout is more or less different. But I think excluding default policy from the learning might limit the full power of "last good reply". Aja ----- Original Message ----- From: Peter Drake To: [email protected] Sent: Wednesday, January 26, 2011 2:27 AM Subject: Re: [Computer-go] The heuristic "last good reply" On Jan 25, 2011, at 10:19 AM, Aja wrote: Dear all, Today I have tried Professor Drake's "last good reply" in Erica. So far, I got at most 20-30 elo from it. I tested by self-play, with 3000 playouts/move on 19x19. The amount of playouts might be too few, but I would like to test more playouts IF the playing strength is not weaker with 3000 playouts. Yes -- the smallest experiments in the paper were with 8k playouts per move. There may not be time to fill up the reply tables with only 3k. From this preliminary experiments with 3000 playouts, I have some observations: 1. In Erica, it's better to consider probability for this heuristic. 2. In Prof. Drake's implementation, there is a weakness in learning. I think the main problem is that for a reply which is deterministically played by default policy, there is no room to learn a new reply. For example, if "save by capture" produces a lost game, then in the next simulation, it will still play "save by capture" by default policy. If I am wrong in this point, I am glad to be corrected by anyone. This is true, but only if the previous move (or previous two moves) come up again in exactly the same board configuration. When the configuration is exactly the same, we are probably still in the search tree, which overrides the policy. If we are beyond the tree, the configuration is almost always different. 3. This heuristic has potential to perform better in Erica. I hope this brief result would encourage other authors to try it. It's reassuring to see that you got some strength improvement out of it! Thanks, Peter Drake http://www.lclark.edu/~drake/ ------------------------------------------------------------------------------ _______________________________________________ Computer-go mailing list [email protected] http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
default_policy.sgf
Description: Binary data
_______________________________________________ Computer-go mailing list [email protected] http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
