On Sat, 2007-06-16 at 10:30 +0200, Erik van der Werf wrote: > On 6/16/07, Don Dailey <[EMAIL PROTECTED]> wrote: > > On Fri, 2007-06-15 at 23:54 +0200, Erik van der Werf wrote: > > > So far, Steenvreter has never played on CGOS. I'm very busy with work, > > > so it will take a while before I have time to put it up for some > > > games. Also to be honest, I'm not really that interested. I guess CGOS > > > is nice if you have no other way to evaluate the strength of your > > > program, but I really like it much more to play in a tournament like > > > the Computer Olympiad where I can meet other programmers face to face. > > > > It sounds almost like you are afraid to play on CGOS. After winning a > > tournament you may feel that you have a reputation to protect. > > I understand it may look that way, but really the primary problem is > lack of time. Coming week I have to be in Sweden, then a conference in > Germany, and further a lot of other small things to do in between. > These things were much easier when I was working full time at > university...
I'm just trying to goad you into playing on CGOS :-) > > I personally feel the results of hundreds of games on CGOS are more > > valid than 10 games no matter how much importance or prestige is > > attached to a particular tournament. 10 games just isn't statistically > > significant. > > The tournament was double round robin, so that's 18 games. 18 still isn't enough, but credit where credit is due - winning this is still a great result. You are very unlikely to win such a tournament if you are not at least one of the better programs. Did I say congratulations? > On the first day I lost one game because of a stupid bug, and won 2 > because I was lucky to meet a weak opponent. I was only able to > recover from the unnecessary loss against Go Intellect because Crazy > Stone managed to crush Mogo twice in the normal rounds. Also Mango > could probably have done much better if it hadn't suffered from some > weird bugs, so clearly the outcome could have been quite different... I would like to see a stronger top end on CGOS. Mogo is all by itself on the top (with an occasionally appearance by Crazy Stone.) Mogo cannot really get a solid rating when it is at least 200 ELO above everyone else. I lack the time to work on Lazarus and even if I get the time I don't know if I can get it up there. So we need some more competition. My real point is that CGOS is more useful to you than you think. - Don > > Having said that, I agree with you 100 percent about your preference. > > There is nothing that compares to a real face to face tournament for the > > pure excitement of it. The stress of each move is powerful motivator > > for improving your program. I am also a strong supporter of ICGA having > > been a member off and on since it was ICCA in the early days. I really > > loved the experiences of playing in the Computer Chess tournaments they > > organized and the people I got to know. > > > > Your result was excellent, but the results were close, only a 1 game > > difference between the top 3 finishers doesn't "evaluate" a program. > > If you played that same tournament over and over you would very likely > > see Mogo and CrazyStone winning many of them too. There is no way to > > say who is better especially since the hardware isn't even equal, > > Steenvreter running on the most powerful of the 3 top finishers. > > You're completely right. I think Mogo and Crazystone play stronger in > the beginning of the game, and my program has to be a bit lucky to get > them into a difficult fight. > > This is not something that can't be solved, I know exactly what I have > to do, but I just need time. Steenvreter was really a rush job, > hacking things together until the last day before the tournament and > no time to test properly. I was hoping to be able to catch up with the > stronger programs, but never expected it to win the tournament. > > > > When I played in the ICCA organized tournaments we used to privately > > laugh (even though we absolutely loved those well organized tournaments) > > that people actually believed it was about "proving who was the best > > computer player in the world." In 1993 my program won the > > International Computer Chess Championship and we privately joked about > > that too - we knew that we had played 2 programs that were at least a > > little better and one that was MUCH better. We "lucked out" by getting > > a draw against the program that was much better and wining against the 2 > > that we believed had a statistically better chance of beating us. We > > knew we were not the best, but we prepared the program to win. We knew > > it was about being good enough that we had a chance to win. A weak > > program has almost no chance, but to have a "reasonable" chance of > > winning a tournament you must be good enough to consistently beat the > > weaker players and not so weak that you have little chance of beating 1 > > or 2 strong players. In a 5 round tournament which was typically what > > we played in, you may only have to face 1 strong program and if you > > have a chance of beating it, you can win a tournament - or someone else > > may knock it out so that you don't have to. > > > > Winning a 10 round tournament of course if much more impressive. But > > it's still not a very valid indicator of who is the better player. > > > > We were in Hong Kong one year and I think Murray Campbell, one of the > > Deep Blue authors told me that they estimated their chances of winning > > to be about 50%. At first that sounded ludicrous because they were the > > heavy favorites and everyone knew their program was far better than any > > other program. But there calculations were roughly correct. Being > > best doesn't mean you will win. The problem is that you usually have to > > win almost every game. If you are statistically likely to win any > > particular game, you are much less statistically likely to put a string > > of wins together. On the other hand, out of many program playing it's > > not unlikely that one of them will get a bit lucky and put a string of > > wins together. The more programs in a tournament, the less likely the > > best one will win. As it turned out, they did NOT win that > > tournament. I found it pretty cool that this was a surprise to most of > > the players except for them. > > :-) > > Erik _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
