On 9/21/07, Don Dailey <[EMAIL PROTECTED]> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jason, > > I noticed from several emails that you are probably doing a lot of > little things differently and assuming they make no difference.
This thread has certainly helped highlight them. I now have a list of things and a crude ordering of what may be affecting the quality of the simulations and the results. I plan to experiment with changes to the various things... but don't seem to get more than 20 minutes at a time to dedicate to the experimentation. For > instance you still haven't tried the same exact eye-rule we are using so > you can't really say with complete confidence that there is no difference. Huh? I thought my eye alternative #2 was the "exact eye rule". A center point is considered an eye if all 4 neighbors are friendly stones and no more than one diagonal is an enemy stone. An edge (corner) are eyes if all three (both) neighbors are friendly stones and none of the diagonals are enemy stones. I realize that alternative #1 wasn't correct, but I thought #2 was. Please let me know if that's an incorrect assumption. If you really want to get to the bottom of this, you should not assume > anything or make approximations that you believe shouldn't make much > difference even if you are right in most cases. There could be one > clearly wrong thing you are doing, or it could be many little things > that all make it take a hit. I'm systematically experimenting with these things. So far, this has been the eyes and playing a random game until one or two passes (in a row). housebot-xxx-amaf - original eye, one pass to end game hb-amaf-alteye - alternate eye rule #1, two passes to end game hb-amaf-alteye2 - alternate eye rule #2, two passes to end game hb-amaf-alt - original eye, two passes to end game I still have yet to experiment with random move generation. I'm 100% confident that this is what "effective go library" does, but I don't consider that evidence that it's the most correct method (only the fastest). I am curious myself what the difference is and I'm willing to help you > figure it out but we have to minimize the ambiguity. I appreciate the help. When we're all done, I'll take a crack at writing a few pages describing the experiments and the outcomes. I was going to suggest the random number generator next, but for these > simple bots there doesn't seem to be a great deal of sensitivity to the > quality of the random number generator if it's reasonable - at least for > a few games. Out of all things, I would suspect my PRNG the least. It's an open source Mersenne Twister implementation (Copyright (C) 1997 - 2002, Makoto Matsumoto and Takuji Nishimura). My understanding is considered a really good random number generator. I'll likely try alternatives to how random moves are selected based on the random number generator. Ogo (which is almost the same as AnchorMan) has a poor quality RNG and > if you play a few hundred games you will discover lot's of repeated > results. With a good quality generator there have NEVER been repeated > games that I have ever seen. So it could be a minor factor. > > One thing you mentioned earlier that bothers me is something about when > you end the random simulations. AnchorMan has a limit, but it's very > conservative - a game is rarely ended early and I would say 99.9% of > them get played to the bitter end. I don't use a mercy rule, and have tried out playing games to the bitter end (neither side has a legal non-eye-filling move). I assume this would resolve your concerns. Are you cheating here? I suggest you make the program as identical to > mine as you can - within reason. I agree and I'm slowly trying to do that. If you are doing little things wrong > they accumulate. I learned this from computer chess. Many > improvements are worth 5-20 ELO and you can't even measure them without > playing thousands of games - and yet if you put a few of them together > it can put your program in another class. I don't disagree with what you're saying. At 20 ELO per fix, 800 ELO is tough to overcome. I'd hope I can't have that many things wrong with a relatively pure monte carlo program ;) I'm hoping to find at least one really big flaw... something that'd put it close to ReadyFreddy. I want to try a breadth of changes to see if I can something akin to a magic bullet. As I go forward, I will also try various combos of the implemented hacks and see how they do. It's easier to put up a new combo and see how it does. One small problem I have is that I can only run two versions reliably and rankings at my bots' current level seem to fluctuate a lot by which bots are currently running. In my first marketable chess program I worked with my partner and I > obsessed every day on little tiny speedups - most of them less than 5% > speedups. We found 2 or 3 of these every day for weeks it seems. But > we kept finding them and when we were finished with had a program about > 300% faster than we started and it had a reputation at the time as being > a really fast chess program. > > This works with strength improvements too. So you might have one big > wrong thing, but you may have several little ones. > > So instead of continuing to explore eye rules and experiment, as much > fun as it is in it's own right, it's not a productive way to find the > performance problem. Start with the eye rule every program is using > with great success and if you want to find something better LATER, more > power to you. I think we're in agreement. I initially misunderstood that the typical eye rule considers empty diagonals to be equivalent to friendly stones in the diagonals. Even before this thread began, I had a similar rule where empty was considered to be like an enemy stone. What I was using at the start of this thread was better than that really restrictive eye rule (but not as good as the real one). I've started to experiment with other things such as when to end a game (and the side effect of how to score them), and will continue with other variations as time permits. If you want my help, I will try to make a very generic version of > AnchorMan that doesn't have any enhancements - just the clearly stated > basics which I believe will probably play around 1300-1400 ELO. That would be really helpful to see. I'd also be interested in seeing how other engines do as amaf... and hearing from the author of ego110_allfirst about how they did stuff. I know the normal ego allows suicides in random games, uses a faster random move selector (that is already suspect in this thread), uses a mercy rule, and likely doesn't use the 5/8 keep fraction for AMAF that you've recommended. By the way, Anchorman on the new server seems to be weaker than on the > old server - so that might explain some of the difference. > > > - - Don >
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
