Thanks for all these responses and discussions here! I'd like to try to gather pieces in this thread so far and to make the consensuses more clear to me (and others hopefully).
Consensus 1: given the searching/playout policies, more playouts will GENERALLY contribute to a stronger program. Here the term "generally" means that the observations are from statistics (or the sense we got empirically) upon large number of games between various GO bots, rather than refering to any specific cases/situations. So an example-based arguement is not valid here. However, two questions for this consensus: (1.a) Inhttp://cgos.boardspace.net/study/, the ELO improvement from doubling the number of playouts seems converge to 3000 score. Is this because versions with 3000+ ELO is "too strong" to be measured accurately? More importantly, seems these results are self-playing game records, which is known to have amplification effect on the ELO score improvement. (1.b) Generally speaking, we always want the progam to "show its best" in an important tournament like Olympaid, and assuming the consensus, more playouts will always helps this. As a consequence, there should be no serious player for the Olympaid who run their program on laptop (unless they are fully confident that the capability of their program shown in laptop is *already* strong enough to beat all others!). Note that "the authors prefer the algorithm-part improvement" or "software algorithm just matters much more" do not answer this question, since no matter what algorithmic improvement here, we just assume that more playouts give orthogonal contributions, right? Consensus 2: there is a trade-off between the quality of playout and the speed of playout. it's very interesting for me to find that consensus 2 actually has little necessary relation with consensus 1. First, the speed of playout translates to the number of playout only under fixed computation resources. So even the playout becomes slow due to a more complex playout policy, we can achieve the same number of playouts by simply using more CPU cores. Second, even though we do have the "quality vs number" trade-off given fixed CPU cores, this does not conflict with the fact that "number" brings additional contributions for given "quality". But seems all strong programs just favor the "quality" in this trade-off, because more playouts tend to make the same errors with previous playouts so they can hardly recover these errors in reasonable time. On the other hand, everyone admit that the random playout is clearly out of the sweet spot area of this trade-off. Finally, I'm trying to find a "lower bound quality" for any meaningful trade-off, but there seems no convincing answer. Bojun
_______________________________________________ Computer-go mailing list [email protected] http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
