I have heard this many times - but it doesn't always apply. In fact I have heard that IMPROVEMENTS always look better against your twin-brother but if that were true, I would always want to test against my twin since it makes improvements stand out. It's hard to measure small improvements so this would be like using a microscope to help me.
But unfortunately a change can help you beat your twin but make the program worse against other opponents - but I have only occasionally seen this be a big factor although I admit it does happen. I just think the effect is exaggerated by people. A general rule is that if you are better against Joe, you are "probably" better against Fred. I'm doing some experiments with automated tuning (with my chess program.) I have had a lot of success with this - in self test games I increased my win percentage significantly over a lot of games. When I play Toga, I actually achieved an even HIGHER win percentage. But I never trust an "improvement" without testing against a variety of opponents, at the minimum a self-test and a test against a different opponent. - Don On Wed, 2006-11-29 at 12:22 +0100, Chrilly wrote: > The paper mentions the relative comparison of 2 versions. This is > common > practice in the scientific literature, but it is a very poor choice if > one > wants to measure the effect of a new method. The effects of changes is > much > more pronounced than against another opponet. A method which is good > against > the twin-brother must not be good against other opponents at all. > Even > against other opponents it happens frequently that a method works > quite well > against opponent A but it fails against B. Its relative easy to make > a > version which crashes e.g. Rybka, but this version is poor against > Fritz and > Shredder. The really difficult task is to find a combination which > plays > good against all. _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
