On Fri, 2007-06-15 at 23:54 +0200, Erik van der Werf wrote: > So far, Steenvreter has never played on CGOS. I'm very busy with work, > so it will take a while before I have time to put it up for some > games. Also to be honest, I'm not really that interested. I guess CGOS > is nice if you have no other way to evaluate the strength of your > program, but I really like it much more to play in a tournament like > the Computer Olympiad where I can meet other programmers face to face.
It sounds almost like you are afraid to play on CGOS. After winning a tournament you may feel that you have a reputation to protect. I personally feel the results of hundreds of games on CGOS are more valid than 10 games no matter how much importance or prestige is attached to a particular tournament. 10 games just isn't statistically significant. Having said that, I agree with you 100 percent about your preference. There is nothing that compares to a real face to face tournament for the pure excitement of it. The stress of each move is powerful motivator for improving your program. I am also a strong supporter of ICGA having been a member off and on since it was ICCA in the early days. I really loved the experiences of playing in the Computer Chess tournaments they organized and the people I got to know. Your result was excellent, but the results were close, only a 1 game difference between the top 3 finishers doesn't "evaluate" a program. If you played that same tournament over and over you would very likely see Mogo and CrazyStone winning many of them too. There is no way to say who is better especially since the hardware isn't even equal, Steenvreter running on the most powerful of the 3 top finishers. When I played in the ICCA organized tournaments we used to privately laugh (even though we absolutely loved those well organized tournaments) that people actually believed it was about "proving who was the best computer player in the world." In 1993 my program won the International Computer Chess Championship and we privately joked about that too - we knew that we had played 2 programs that were at least a little better and one that was MUCH better. We "lucked out" by getting a draw against the program that was much better and wining against the 2 that we believed had a statistically better chance of beating us. We knew we were not the best, but we prepared the program to win. We knew it was about being good enough that we had a chance to win. A weak program has almost no chance, but to have a "reasonable" chance of winning a tournament you must be good enough to consistently beat the weaker players and not so weak that you have little chance of beating 1 or 2 strong players. In a 5 round tournament which was typically what we played in, you may only have to face 1 strong program and if you have a chance of beating it, you can win a tournament - or someone else may knock it out so that you don't have to. Winning a 10 round tournament of course if much more impressive. But it's still not a very valid indicator of who is the better player. We were in Hong Kong one year and I think Murray Campbell, one of the Deep Blue authors told me that they estimated their chances of winning to be about 50%. At first that sounded ludicrous because they were the heavy favorites and everyone knew their program was far better than any other program. But there calculations were roughly correct. Being best doesn't mean you will win. The problem is that you usually have to win almost every game. If you are statistically likely to win any particular game, you are much less statistically likely to put a string of wins together. On the other hand, out of many program playing it's not unlikely that one of them will get a bit lucky and put a string of wins together. The more programs in a tournament, the less likely the best one will win. As it turned out, they did NOT win that tournament. I found it pretty cool that this was a surprise to most of the players except for them. - Don _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
