>> Typically, how many parameters do you have to tune ? Real or two-level ? > > I guess I have 10 real valued and 10 binary ones. There are probably a lot > of stuff that are ahrd coded and could be parameterized. > > Here I am also completely ignoring playouts that have hundreds of handtuned > parameters.
There automatic techniques like those describes by Rémi Coulom are probably useful. >> If you consider a yet reasonable subset of parameters, an efficient way >> to estimate them is to use fractional factorial design for the linear >> part, and central composite design for quadratic part (once you know you >> are already in the right area). You are much more precise than with >> change-one-at-a-time strategies if there is no interaction between >> parameters, and you can detect interactions. > > I once met this guy: > > http://meche.mit.edu/people/faculty/index.html?id=27 > > his research is a mix of testing formal methods and how well they work in > practice and also studying how engineers (who often do not use these > methods) actually do in practice. > > He seemed to argue that doing parameter optimization intuitively by hand is > not as bad as one might think compared to fractional factorial design. So I > use that as an excuse for just doing it as I always did. For me it is > important to keep a careful record of what I do and plot the result with > confidence intervals to avoid tricking myself. Anyhow, it's always wiser for someone to use a method he understands; so best to keep it simple if you do not take/have the time to learn more sophisticated techniques. >> Alternatively, especially with a very high number of real parameters, >> derivatives of MC techniques can be efficient and easy to implement: >> particle filtering or swarm optimization in particular. > > That would be tempting (I once implemented a fitting method inspired by > simulating annealing and it was very efficient) but it would require a > completely different test setup than the one I use right now. > Thinking more about it, that's not completely obvious, and hence is interesting (for me): usually those methods are tailored for functions where they get the exact value, not a Bernoulli trial. > It also a matter of time and patience. I want new results every day. If I > would test all parameters at once using formal methods I would still have > to wait for weeks. Well, once you are in the right zone, if you want to check a change, you can look at this change only. Note that you can also try and get information on other real parameters even when focusing on a precise one: you make them vary very little. Then the results for those supplementary parameters nearly do not impact your main parameter, and you can get information through taking means for a longer time... Jonas _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
