On Sat, 2007-03-17 at 18:45 -0500, Nick Apperson wrote: > one concern i have is that within a family of programs (such as MC) > the estimated skill differences are overestimated. I would really > like to see an anchor that uses a different technique. I'm not > offering a solution. Thoughts?
One idea is to measure this phenomenon to see how much we should be concerned by it. I'm actually performing a set of interesting scalability tests right now with Lazarus. I'm trying to measure the improvement trend as the number of play-outs increase. I'm interested in the curve produced. I can cross check my results with the performance of various levels against gnugo - perhaps estimate the amount of intransitivity. But these tests are based on self-play only. However, I can later individually test against gnugo with various levels of Lazarus. What I would be looking for is how much intransitivity exists. Since ELO predicts win percentage I can see if doubling the level of Lazarus gives me the expected improvement against gnugo. I don't know how well this will work since the range I can test in and still get meaningful samples is relatively low. Personally, I theorize that MC based programs are less susceptible to being taken advantage of by intransitivity and they might actually be the best choice for an Anchor. They don't play deterministically and compared to programs based heavily on hand crafted knoweldge, they should be signficantly less ideosyncratic. _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
