2013/3/6 Chin-Chang Yang <[email protected]> > > Hi, > > I'm considering CLOP to be one of the compared optimizer in > RobustOptimizer https://github.com/ChinChangYang/RobustOptimizer/issues/68. > However, I have some questions to your experiment. > > The CLOP is for noisy black-box parameter tuning. However, your test > functions (LOG, FLAT, POWER, ANGLE, and STEP) are noise-free functions as > shown in Table 1. It is very difficult to prove that CLOP can work very > well on noisy functions. > > I suggest that the problem definition f(x) = 1/(1+exp(-r(x))) should be > perturbed with some random variables with a defined zero-mean distribution, > such as Gaussian distribution, uniform distribution, or any others. > Specifically, the problem definitions can be g(x) = 1/(1+exp(-r(x) + n(x))) > where n(x) is an additional noise. The performance of the algorithms can be > evaluated in terms of solution error measure, which is defined as f(x) - > g(x*) where x* is the global optimum of the noise-free function f. >
Sorry, the definition of solution error measure should be g(x) - f(x*). Chin-Chang Yang, 2013/03/06 > > BBOB 2012 defines some noisy functions > http://coco.gforge.inria.fr/doku.php?id=bbob-2012 which may also provide > confident performance evaluation for noisy optimization. > > There may exist more appropriate performance evaluation methods than > aforementioned ones for win/loss outcomes. Anyway, in this paper, the > experiment uses noise-free functions as test functions. It cannot prove > anything for noisy optimization. > > Best regards, > Chin-Chang Yang, 2013/03/06 > > 2011/9/1 Rémi Coulom <[email protected]> > >> Hi, >> >> This is a draft of the paper I will submit to ACG13. >> >> Title: CLOP: Confident Local Optimization for Noisy Black-Box Parameter >> Tuning >> >> Abstract: Artificial intelligence in games often leads to the problem of >> parameter tuning. Some heuristics may have coefficients, and they should be >> tuned to maximize the win rate of the program. A possible approach consists >> in building local quadratic models of the win rate as a function of program >> parameters. Many local regression algorithms have already been proposed for >> this task, but they are usually not robust enough to deal automatically and >> efficiently with very noisy outputs and non-negative Hessians. The CLOP >> principle, which stands >> for Confident Local OPtimization, is a new approach to local regression >> that overcomes all these problems in a simple and efficient way. It >> consists in discarding samples whose estimated value is confidently >> inferior to the mean of all samples. Experiments demonstrate that, when the >> function to be optimized is smooth, this method outperforms all other >> tested algorithms. >> >> pdf and source code: >> http://remi.coulom.free.fr/CLOP/ >> >> Comments, questions, and suggestions for improvement are welcome. >> >> Rémi >> _______________________________________________ >> Computer-go mailing list >> [email protected] >> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go >> > > > > -- > Chin-Chang Yang -- Chin-Chang Yang
_______________________________________________ Computer-go mailing list [email protected] http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
