> > What I mean is that if human player H beats computer C1 65% of the > time, and computer C2 also beats computer C1 65% of the time, then I > would expect that H would be stronger than C2, especially if both C1 > and C2 are MC programs. If it is the case, then it would make it > difficult to compare human scale to computer scale. But that is just > my intuition. > > For instance, against computers, I estimate that Crazy Stone improved > about 3 stones between this summer and now. But it clearly did not > improve 3 stones on KGS. I vaguely remember that Sylvain also noticed > that MoGo could beat GNU go with a 4-stone handicap, but was only 2 > stones stronger than GNU on KGS. This happened in computer chess many years ago. There was a period of time when chess playing computers were relatively unknown but then suddenly became very common. This is anecdotal, but it appears that for a certain amount of time the computers continued to improve, but at the same time humans adapted very quickly to them. Humans quickly became educated. They didn't become educated until computers approached their playing strength.
So there is a fairly significant ELO advantage to the human that has experience playing computers. I think it's very possible that this is what you observed. The advantage of a human however has limits. You are not going to beat a player 1000 ELO stronger just because you know how he plays. Also, we don't see any serious discrepancy in computer vs human ratings in chess although it was always imagined. If you look in the sky you can imagine interesting shapes, but only because you are "looking" for them. Whenever we think we observe something based on a few data points it's extremely subject to error. Thus people often believed that some given program might be 200 ELO stronger than other programs but that it would translate into something very modest against humans. This NEVER turned out to be true - it was fantasy based on the continuing need to believe that the improvements were just too good to be true. A little common sense will tell you that this cannot be true but to a very limited extent if any. You would have to believe that the in-transitive rubber band stretches to infinity. Or you have to do what Einstein did and regretted, which is to impose an artificial explanation such as some kind of constant that pushes this back at higher strength levels. The main point is that in computer chess computers improved approximately 2000 ELO against humans over a couple of decades or so. But when it's claimed that ELO improvement against other computers is 4 times that, you imply 8000 ELO points for computer vs computer! This is clearly not the case. That was generally the claim I heard - 100 ELO improvement but it's someones "belief" that against humans it's only about 25 ELO. Nonsense. I could see one things possibly happening however. You might make a real improvement that doesn't hit a human weakness that hard and at the local limited horizon it really may not translate to the same improvement against humans. But intransitivity in playing strength is like a rubber band. You can only stretch it so far and it fights back. You are not going to get A beats B 99% of the time, B beats C 99% of the time, but A cannot beat C. Not unless you go way out of your way to construct programs that have this behavior. I don't believe multi-dimensional playing strength is much of an issue. It exists, but it's not severe. You are not going to have to worry that some 4 dan player will beat the world champion because he happens to have a playing style that the world champion cannot handle. Otherwise all rating/ranking systems would be useless. And it would be easy to construct some really ludicrous cases of intransitivity. - Don > > Rémi > _______________________________________________ > computer-go mailing list > [email protected] > http://www.computer-go.org/mailman/listinfo/computer-go/ > _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
