Quoting Heikki Levanto <[EMAIL PROTECTED]>:
Of course we can wait until we get more data. I just wanted to share my observation that the curves seem to change around the level where MC playouts tend to flatten out, and hear if anyone would have some insightful comments to that. Even with the risk that more data may invalidate my observation...
Nothing wrong with that, I do it myself all the time for my own tests. But I have tricked myself at lot of time plotting curves like this.
As a rule of thumb I want 300 games for each datapoint and at least 500 if I am going to make any conclusions.
Also note that this project collects data for a lot of datapoints at the same time. This means that by chance there must be several datapoints that are quite far off from where they should be, but the human eye will not correct for that. So making conclusions from details of these curves I would prefer even more games for each datapoint. Perhaps a 1000 games or so.
Interestingly the column of opponent rating indicate that many of the programs has played against harder or weaker opposition than expected, meaning that a lot of games played might have had a very predictable outcome. Thus the number of games played are probably to some extent misleading for how accurate the results are.
-Magnus _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
