Re: [Computer-go] dealing with multiple local optima

On Mon, Feb 27, 2017 at 8:05 AM, Erik van der Werf wrote: > On Mon, Feb 27, 2017 at 4:30 PM, Darren Cook wrote: > >> > But those video games have a very simple optimal policy. Consider Super >> Mario: >> > if you see an enemy, step on it; if you see

Re: [Computer-go] dealing with multiple local optima

On Mon, Feb 27, 2017 at 4:30 PM, Darren Cook wrote: > > But those video games have a very simple optimal policy. Consider Super > Mario: > > if you see an enemy, step on it; if you see a whole, jump over it; if > you see a > > pipe sticking up, also jump over it; etc. > > A bit

Re: [Computer-go] dealing with multiple local optima

> But those video games have a very simple optimal policy. Consider Super > Mario: > if you see an enemy, step on it; if you see a whole, jump over it; if you see > a > pipe sticking up, also jump over it; etc. A bit like go? If you see an unsettled group, make it live. If you have a ko, play

Re: [Computer-go] dealing with multiple local optima

On Sat, Feb 25, 2017 at 12:30 AM, Brian Sheppard via Computer-go < computer-go@computer-go.org> wrote: > In retrospect, I view Schradolph’s paper as evidence that neural networks > have always been surprisingly successful at Go. Like Brugmann’s paper about > Monte Carlo, which was underestimated

Re: [Computer-go] dealing with multiple local optima

com>; computer-go@computer-go.org Subject: Re: [Computer-go] dealing with multiple local optima TD-gammon is regarded as a special case from the stochastic characteristics of the backgammon game; it smoothens the search space for the value function and the value function itself to a great

Re: [Computer-go] dealing with multiple local optima

NEAT and hyperNEAT are awesome when "evolving" fairly simple networks with a very limited number of input and output dimensions. However, without access to some serious computational power, scaling the NEAT method up to the kind of level you would need for the current encoding methods for the

Re: [Computer-go] dealing with multiple local optima

"seeing" is complex when the input is just a bunch of pixels.  Terry McIntyre Unix/Linux Systems Administration Taking time to do it right saves having to do it twice. On Friday, February 24, 2017 12:32 PM, Minjae Kim wrote: But those video games have a very simple

Re: [Computer-go] dealing with multiple local optima

I should point out that Reinforcement Learning is a relatively unimportant part of AlphaGo, according to the paper. They only used it to turn the move-prediction network into a stronger player (presumably increasing the weights of the layer before SoftMax would do most of the job, by making the

Re: [Computer-go] dealing with multiple local optima

But those video games have a very simple optimal policy. Consider Super Mario: if you see an enemy, step on it; if you see a whole, jump over it; if you see a pipe sticking up, also jump over it; etc. On Sat, Feb 25, 2017 at 12:36 AM, Darren Cook wrote: > > ...if it is hard to

Re: [Computer-go] dealing with multiple local optima

> ...if it is hard to have "the good starting point" such as a trained > policy from human expert game records, what is a way to devise one. My first thought was to look at the DeepMind research on learning to play video games (which I think either pre-dates the AlphaGo research, or was done in

Re: [Computer-go] dealing with multiple local optima

Neural networks always have a lot of local optima. Simply because they have a high degree of internal symmetry. That is, you can “permute” sets of coefficients and get the same function. Don’t think of starting with expert training as a way to avoid local optima. It is a way to start