Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi Oliver Reinforcement learning is different to unsupervised learning. We used reinforcement learning to train the Atari games. Also we published a more recent paper (www.nature.com/articles/nature14236) that applied the same network to 50 different Atari games (achieving human level in around half). Similar neural network architectures can indeed be applied to Go (indeed that was one of the motivations for our recent ICLR paper). However, training by reinforcement learning from self-play is perhaps more challenging than for Atari: our method (DQN) was applied to single-player Atari games, whereas in Go there is also an opponent. I could not guarantee that DQN will be stable in this setting. Cheers Dave On 16 March 2015 at 22:21, Oliver Lewis ojfle...@yahoo.co.uk wrote: Can you say anything about whether you think their approach to unsupervised learning could be applied to networks similar to those you trained? Any practical or theoretical constraints we should be aware of? On Monday, 16 March 2015, Aja Huang ajahu...@gmail.com wrote: Hello Oliver, 2015-03-16 11:58 GMT+00:00 Oliver Lewis ojfle...@yahoo.co.uk: It's impressive that the same network learned to play seven games with just a win/lose signal. It's also interesting that both these teams are in different parts of Google. I assume they are aware of each other's work, but maybe Aja can confirm. The authors are my colleagues at Google DeepMind as on the paper they list DeepMind as their affiliation. Yes we are aware of each other's work. Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Can you say anything about whether you think their approach to unsupervised learning could be applied to networks similar to those you trained? Any practical or theoretical constraints we should be aware of? On Monday, 16 March 2015, Aja Huang ajahu...@gmail.com wrote: Hello Oliver, 2015-03-16 11:58 GMT+00:00 Oliver Lewis ojfle...@yahoo.co.uk javascript:_e(%7B%7D,'cvml','ojfle...@yahoo.co.uk');: It's impressive that the same network learned to play seven games with just a win/lose signal. It's also interesting that both these teams are in different parts of Google. I assume they are aware of each other's work, but maybe Aja can confirm. The authors are my colleagues at Google DeepMind as on the paper they list DeepMind as their affiliation. Yes we are aware of each other's work. Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
For many faces, moves like, any Atari, fill a liberty in a losing semeai, attack a group that is alive but doesn’t have two clear eyes yet. From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Stefan Kaitschick Sent: Saturday, January 10, 2015 1:13 AM To: computer-go@computer-go.org Subject: Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go But, I imagine this is more fuss than it is worth; the NN will be integrated into MCTS search, and I think the strong programs already have ways to generate ko threat candidates. Darren Do they? What would look like? Playing 2 moves in a row for the same side? I thought the programs naively discovered ko threats. Stefan ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
But, I imagine this is more fuss than it is worth; the NN will be integrated into MCTS search, and I think the strong programs already have ways to generate ko threat candidates. Darren Do they? What would look like? Playing 2 moves in a row for the same side? I thought the programs naively discovered ko threats. Stefan ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
On 2014-12-19 15:25, Hiroshi Yamashita wrote: Ko fight is weak. Ko threat is simpley good pattern move. I suppose you could train on a subset of data: only positions where there was a ko-illegal move on the board. Then you could learn ko threats. And then use this alternative NN when meeting a ko-illegal position in a game. But, I imagine this is more fuss than it is worth; the NN will be integrated into MCTS search, and I think the strong programs already have ways to generate ko threat candidates. Darren ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi, I am just trying to reproduce the data from page 7 with all features disabled. I do not reach the accuracy (I stay below 20%). Now I wonder about a short statement in the paper, I did not really understand: On page 4 top right they state In our experience using the rectifier function was slightly more effective then using the tanh function Where do they put this functions in? I use caffe, and as far as I understood it, I would have to add extra layers to get a function like this. Does this mean: before every layer there should be a tanh or rectifier layer? I would be glad to share my sources if somebody is trying the same, Detlef Am 15.12.2014 um 00:53 schrieb Hiroshi Yamashita: Hi, This paper looks very cool. Teaching Deep Convolutional Neural Networks to Play Go http://arxiv.org/pdf/1412.3409v1.pdf Thier move prediction got 91% winrate against GNU Go and 14% against Fuego in 19x19. Regards, Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi! On Wed, Dec 31, 2014 at 11:16:57AM +0100, Detlef Schmicker wrote: I am just trying to reproduce the data from page 7 with all features disabled. I do not reach the accuracy (I stay below 20%). Now I wonder about a short statement in the paper, I did not really understand: On page 4 top right they state In our experience using the rectifier function was slightly more effective then using the tanh function Where do they put this functions in? I use caffe, and as far as I understood it, I would have to add extra layers to get a function like this. Does this mean: before every layer there should be a tanh or rectifier layer? I think this is talking about the non-linear transformation function. Basically, each neuron output y = f(wx) for weight vector w and input vector x and transfer function f. Traditionally, f is a sigmoid (the logistic function 1/(1+e^-x)), but tanh is also popular and with deep learning, rectifier and such functions are very popular IIRC because they allow much better propagation of error to deep layers. I would be glad to share my sources if somebody is trying the same, I hope to be able to start dedicating time to this starting the end of January (when I'll be moving to Japan for three months! I'll be glad to meet up with fellow Go developers some time, and see you at the UEC if it's in 2015 too :-). I would very much appreciate an open source implementation of this - or rather, I'd rather spend my time using one to do interesting things rather than building one, I do plan to open source my implementation if I have to make one and can bring myself to build one from scratch... -- Petr Baudis If you do not work on an important problem, it's unlikely you'll do important work. -- R. Hamming http://www.cs.virginia.edu/~robins/YouAndYourResearch.html ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Am 31.12.2014 um 14:05 schrieb Petr Baudis: Hi! On Wed, Dec 31, 2014 at 11:16:57AM +0100, Detlef Schmicker wrote: I am just trying to reproduce the data from page 7 with all features disabled. I do not reach the accuracy (I stay below 20%). Now I wonder about a short statement in the paper, I did not really understand: On page 4 top right they state In our experience using the rectifier function was slightly more effective then using the tanh function Where do they put this functions in? I use caffe, and as far as I understood it, I would have to add extra layers to get a function like this. Does this mean: before every layer there should be a tanh or rectifier layer? I think this is talking about the non-linear transformation function. Basically, each neuron output y = f(wx) for weight vector w and input vector x and transfer function f. Traditionally, f is a sigmoid (the logistic function 1/(1+e^-x)), but tanh is also popular and with deep learning, rectifier and such functions are very popular IIRC because they allow much better propagation of error to deep layers. Thanks a lot. I was struggling with the traditionally, and expected this to be the case for the standard convolutional layers in caffe. This seems not to be the case, so now I added layers for f(x): Now I reach 50% accuracy for a small dataset (285000 positions). Of cause this data set is too small (therefore the number is overestimated), but I only had 15% on this before introducing f(x) :) I would be glad to share my sources if somebody is trying the same, I hope to be able to start dedicating time to this starting the end of January (when I'll be moving to Japan for three months! I'll be glad to meet up with fellow Go developers some time, and see you at the UEC if it's in 2015 too :-). I would very much appreciate an open source implementation of this - or rather, I'd rather spend my time using one to do interesting things rather than building one, I do plan to open source my implementation if I have to make one and can bring myself to build one from scratch... oakfoam is open source anyway. In my branch my caffe based implementation is available. My branch is not so clean as Francois's one, and we did not merge for quite a time:( At the moment the CNN part is in a very early state, you have to produce the database by different scripts... But I would be happy to assist! Detlef ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
I would very much appreciate an open source implementation of this - or rather, I'd rather spend my time using one to do interesting things rather than building one, I do plan to open source my implementation if I have to make one and can bring myself to build one from scratch... I started building a convolution network library for OpenCL at https://github.com/hughperkins/ClConvolve/ - tanh, relu, linear activations - OpenCL - fully connected and convolutional layers OpenCL you might see as good or bad, depending on your point of view. It's certainly unique. eg, caffe uses CUDA I believe, as does Theano, and so on. OpenCL has the advantage of being an open standard, and you can run it on many CPUs, eg Intel Ivy Bridge integrated graphics cards. I intend to implement 'pizza-slice' symmetry, or maybe 'kaleidoscope'-symmetry is a better name. Either way, the 4-way symmetry, for w: vertically, horizontally, and across both diagonals. It's currently a work in progress. It can get 83% accuracy on mnist, using a single convolutional layer, and no other layers at all. Fully-connected layer also seems to be working. forward prop and backward prop are both in gpu, for convolutional layers. fully-connected layers are still 100% on cpu, but you only would have one such layer, right, so not a high priority? I'm currently building test cases to ensure that multiple, deep, topologies work correctly. Hugh ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi! On Fri, Dec 19, 2014 at 10:50:30AM +0900, Hiroshi Yamashita wrote: One question: Is there a place where I can find sgf Paper author, Christopher Clark kindly sent me sgf and let me share on ML. That's great, thanks for negotiating that. :-) This is a copy of sgf. http://www.yss-aya.com/dcnn_games_20141218.tar.gz For those impatient, here is an example SGF of a game won by the game predictor against Fuego: http://eidogo.com/#CjfDX1zV The predictor is white. It really does just play shapes, but evidently it's plenty enough sometimes or against weaker opponents. -- Petr Baudis If you do not work on an important problem, it's unlikely you'll do important work. -- R. Hamming http://www.cs.virginia.edu/~robins/YouAndYourResearch.html ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
That's pretty good looking for a pure predictor. Considering it has no specific knowledge about semeais, ladders, or ko threat situations... Switching out the pattern matcher (not the whole move generator) in an existing mc program, should be pretty straightforward. Even if the nn is a lot slower than the orig. pattern matcher, it could still make sense high up in the tree. If this does prove fruitful, the question of what transforms of the raw position should be fed to the nn could be the next big thing. After the almost annoying success of general purpose mc, this would be a more poetic line of research. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi, The predictor is white. It really does just play shapes, but evidently it's plenty enough sometimes or against weaker opponents. I saw some games, and my impression are DCNN sees board widely. Without previous move info, DCNN can answer opponent move. It knows well corner life and death shape. It does not understand two eyes, and ladder. Tactical fight is weak. Ko fight is weak. Ko threat is simpley good pattern move. It does not understand semeai which has many libs, like 4 vs 5. So it will not help to generate semeai moves. This is Aya's move predictor(W) vs GNU Go(B). http://eidogo.com/#3BNw8ez0R I think previous move effect is too strong. Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
This is Aya's move predictor(W) vs GNU Go(B). http://eidogo.com/#3BNw8ez0R I think previous move effect is too strong. This is a good example of why a good playout engine will not necessarily play well. The purpose of the playout policy is to *balance* errors. Following your opponent's last play is very helpful in that regard. -Original Message- From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Hiroshi Yamashita Sent: Friday, December 19, 2014 10:25 AM To: computer-go@computer-go.org Subject: Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go Hi, The predictor is white. It really does just play shapes, but evidently it's plenty enough sometimes or against weaker opponents. I saw some games, and my impression are DCNN sees board widely. Without previous move info, DCNN can answer opponent move. It knows well corner life and death shape. It does not understand two eyes, and ladder. Tactical fight is weak. Ko fight is weak. Ko threat is simpley good pattern move. It does not understand semeai which has many libs, like 4 vs 5. So it will not help to generate semeai moves. This is Aya's move predictor(W) vs GNU Go(B). http://eidogo.com/#3BNw8ez0R I think previous move effect is too strong. Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
I put two commented games on http://webdocs.cs.ualberta.ca/~mmueller/fuego/Convolutional-Neural-Network.html http://webdocs.cs.ualberta.ca/~mmueller/fuego/Convolutional-Neural-Network.html Enjoy! Martin ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
[Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
On Sun Dec 14 23:53:45 UTC 201, Hiroshi Yamashita wrote: Teaching Deep Convolutional Neural Networks to Play Go http://arxiv.org/pdf/1412.3409v1.pdf Wow, this resembles somewhat what I was hoping to do! But now I should look for some other avenue :-) But I'm surprised it's only published on arxiv, I was hoping that something like this could be conference-worthy? ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi, One question: Is there a place where I can find sgf Paper author, Christopher Clark kindly sent me sgf and let me share on ML. This is a copy of sgf. http://www.yss-aya.com/dcnn_games_20141218.tar.gz His notes is as follows. --- Some notes: The names of the 'players' are fuego, gnugo, gnugo_j (gnugo with Japanese rules), clf_gogod and clf_kgs for networks trained on the respective dataset, and clf_small for the smaller network trained on the gogod dataset. In the archive, each prefix (0_xxx) corresponds to one round of games between two of these players. The metadata should contain all the details. Games against GNU Go and Fuego are in separate directories. The networks were being run on a CPU and using some rather inefficient python code to convert the raw positions into the liberty encoding style needed as input, you can process positions much faster with a GPU and pre-encoded positions I am hoping to open source this work soon, but it will take some time so that will not be ready until at least next year. A detail left out of the paper, for a few games against Fuego, Fuego was unable to score the final position. We have used GNU Go to rescore these games, I believe Fuego was the winner in all of these cases. Best Regard, Christopher Clark --- Regards, Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi HIroshi, thanks for giving the link. One question: Is there a place where I can find sgf of the(some test games against Fuego1.1 and GnuGo? I want to understand the playing style of this CNN approach. Ingo. Gesendet: Montag, 15. Dezember 2014 um 00:53 Uhr Von: Hiroshi Yamashita y...@bd.mbn.or.jp An: computer-go@computer-go.org Betreff: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go Hi, This paper looks very cool. Teaching Deep Convolutional Neural Networks to Play Go http://arxiv.org/pdf/1412.3409v1.pdf Thier move prediction got 91% winrate against GNU Go and 14% against Fuego in 19x19. Regards, Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi Ingo, One question: Is there a place where I can find sgf I could not find. I also want to see sgf. Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
My impression is that each feature gets a single weight in Crazy-stone. The team-of-features aspect arises because a single point can match several patterns, so you need a model to assign credit when tuning. The paper that I remember used fixed receptive fields to define the patterns. (E.g., from 3x3 through 5x10, or some such.) The easiest way to match those is to use a hash function. My impression is that both NN and DT are capable of asymptotically learning the entire game. (Also true if you use fixed receptive fields.) They should be equally powerful, though they differ in terms of the degree of understanding required by the programmer. IMO, MCTS should always be the outermost loop in the system. MCTS provides asymptotic optimality guarantees under remarkably general conditions. From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of René van de Veerdonk Sent: Monday, December 15, 2014 11:47 PM To: computer-go Subject: Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go Correct me if I am wrong, but I believe that the CrazyStone approach of team-of-features can be cast in terms of a shallow neural network. The inputs are matched patterns on the board and other local information on atari, previous moves, ko situation, and such. Remi alluded as much on this list sometime after his paper got published. Without having studied the Deep Learning papers in detail, it seems that these are the types of smart features that could be learned by a Deep Neural Net in the first few layers if the input is restricted to just the raw board, but could equally well be provided as domain specific features in order to improve computational efficiency (and perhaps enforce correctness). These approaches may not be all that far apart, other than the depth of the net and the domain specific knowledge used directly. Remi recently mentioned that the number of patterns in more recent versions of CrazyStone also number in the millions. I think the prediction rates for these two approaches are also pretty close. Compare the Deep Learning result to the other recent study of a German group quoted in the Deep Learning paper. The bigger questions to me are related to engine architecture. Are you going to use this as an input to a search? Or are you going to use this directly to play? If the former, it had better be reasonably fast. The latter approach can be far slower, but requires the predictions to be of much higher quality. And the biggest question, how can you make these two approaches interact efficiently? René On Mon, Dec 15, 2014 at 8:00 PM, Brian Sheppard sheppar...@aol.com wrote: Is it really such a burden? Well, I have to place my bets on some things and not on others. It seems to me that the costs of a NN must be higher than a system based on decision trees. The convolution NN has a very large parameter space if my reading of the paper is correct. Specifically, it can represent all patterns translated and rotated and matched against all points in parallel. To me, that seems like a good way to mimic the visual cortex, but an inefficient way to match patterns on a Go board. So my bet is on decision trees. The published research on NN will help me to understand the opportunities much better, and I have every expectation that the performance of decision trees should be = NN in every way. E.g., faster, more accurate, easier and faster to tune. I recognize that my approach is full of challenges. E.g., a NN would automatically infer soft qualities such as wall, influence that would have to be provided to a DT as inputs. No free lunch, but again, this is about betting that one technology is (overall) more suitable than another. From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Stefan Kaitschick Sent: Monday, December 15, 2014 6:37 PM To: computer-go@computer-go.org Subject: Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go Finally, I am not a fan of NN in the MCTS architecture. The NN architecture imposes a high CPU burden (e.g., compared to decision trees), and this study didn't produce such a breakthrough in accuracy that I would give away performance. Is it really such a burden? Supporting the move generator with the NN result high up in the decision tree can't be that expensive. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi Aja, That being said, Hiroshi, are you sure there was no problem in your experiment? 6% winning rate against GnuGo on 19x19 seems too low for a predictor of 38.8% accuracy. And yes, in the paper we will show a game that I tried without resign, but result is similar. winrate games 19x19 0.0612416 13x13 0.1701808 9x9 0.1621000 As far as I remember, about 30% accuracy got 3% winrate, so I don't think 6% is too low. I think DCNN 41% accuracy can get 91% winrate is because DCNN maybe understand whole board life and death, like I have one eye and need another one. You can see the sgf. Is it simply playing too good shape? Regards, Hiroshi Yamashita winrate games 19x19 0.059 607 13x13 0.170 545 9x9 0.1411020 ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
I am very much looking forward to your paper. I do see the CNN research as a new idea that has great potential. Linear models like what MM is using are ... far less powerful than CNN My mathematical objection is that this cannot be. The no free lunch theorem applies, and besides, both representations are provably asymptotically perfect given sufficiently accurate training data. I am skeptical about the practical claims, but maybe your paper will sway me. Best, Brian. From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Aja Huang Sent: Tuesday, December 16, 2014 10:23 AM To: computer-go@computer-go.org Subject: Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go Hi Brian, I understand your points, but deep convolutional neural networks are very powerful in the sense that they can represent very complicated functions and are still trainable. Linear models like what MM is using are much simpler and easier to train, but far less powerful than CNN. In our paper we have presented the theoretical background and very good results using CNN on Go. Once approved I'll post the paper and I hope you will enjoy reading it. Regards, Aja On Tue, Dec 16, 2014 at 2:03 PM, Brian Sheppard sheppar...@aol.com wrote: My impression is that each feature gets a single weight in Crazy-stone. The team-of-features aspect arises because a single point can match several patterns, so you need a model to assign credit when tuning. The paper that I remember used fixed receptive fields to define the patterns. (E.g., from 3x3 through 5x10, or some such.) The easiest way to match those is to use a hash function. My impression is that both NN and DT are capable of asymptotically learning the entire game. (Also true if you use fixed receptive fields.) They should be equally powerful, though they differ in terms of the degree of understanding required by the programmer. IMO, MCTS should always be the outermost loop in the system. MCTS provides asymptotic optimality guarantees under remarkably general conditions. From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of René van de Veerdonk Sent: Monday, December 15, 2014 11:47 PM To: computer-go Subject: Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go Correct me if I am wrong, but I believe that the CrazyStone approach of team-of-features can be cast in terms of a shallow neural network. The inputs are matched patterns on the board and other local information on atari, previous moves, ko situation, and such. Remi alluded as much on this list sometime after his paper got published. Without having studied the Deep Learning papers in detail, it seems that these are the types of smart features that could be learned by a Deep Neural Net in the first few layers if the input is restricted to just the raw board, but could equally well be provided as domain specific features in order to improve computational efficiency (and perhaps enforce correctness). These approaches may not be all that far apart, other than the depth of the net and the domain specific knowledge used directly. Remi recently mentioned that the number of patterns in more recent versions of CrazyStone also number in the millions. I think the prediction rates for these two approaches are also pretty close. Compare the Deep Learning result to the other recent study of a German group quoted in the Deep Learning paper. The bigger questions to me are related to engine architecture. Are you going to use this as an input to a search? Or are you going to use this directly to play? If the former, it had better be reasonably fast. The latter approach can be far slower, but requires the predictions to be of much higher quality. And the biggest question, how can you make these two approaches interact efficiently? René On Mon, Dec 15, 2014 at 8:00 PM, Brian Sheppard sheppar...@aol.com wrote: Is it really such a burden? Well, I have to place my bets on some things and not on others. It seems to me that the costs of a NN must be higher than a system based on decision trees. The convolution NN has a very large parameter space if my reading of the paper is correct. Specifically, it can represent all patterns translated and rotated and matched against all points in parallel. To me, that seems like a good way to mimic the visual cortex, but an inefficient way to match patterns on a Go board. So my bet is on decision trees. The published research on NN will help me to understand the opportunities much better, and I have every expectation that the performance of decision trees should be = NN in every way. E.g., faster, more accurate, easier and faster to tune. I recognize that my approach is full of challenges. E.g., a NN would automatically infer soft qualities such as wall, influence that would have
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
A move generator, that always plays it's first choice, that can win games against Fuego? That smells like a possible game changer.(pardon the pun). Surely, programmers will take this workhorse, and put it before the MC cart. Stefan ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
When I had an opportunity to talk to Yann LeCun about a month ago, I asked him if anybody had used convolutional neural networks to play go and he wasn't aware of any efforts in that direction. There was work using neural networks in the mid 1990s, when I first started with computer go. I think the problem, at that time, came down to if you use just a few features it was terrible quality, but if you used more interesting inputs the training times increased exponentially, so much so that it became utterly impractical. I suppose this might be another idea, like monte carlo, that just needed enough computing power for it to become practical for go; it'll be interesting to see how their attempts to scale it turn out. I've added the paper in my Christmas Reading list :-) Darren Teaching Deep Convolutional Neural Networks to Play Go http://arxiv.org/pdf/1412.3409v1.pdf Thier move prediction got 91% winrate against GNU Go and 14% against Fuego in 19x19. -- Darren Cook, Software Researcher/Developer My new book: Data Push Apps with HTML5 SSE Published by O'Reilly: (ask me for a discount code!) http://shop.oreilly.com/product/0636920030928.do Also on Amazon and at all good booksellers! ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Álvaro, this is exactly something that I have been thinking about as well (the last part about MC+NN and feedback between the two). It seems like the authors of that paper are also thinking about something similar. I currently have the very basics of an implementation as well but performance is bad and I don't have access to any (reasonable) GPUs and currently only a very slow CPU. It would be nice to try to figure something out together if you're interested. Regards, Mikael Simberg On Mon, Dec 15, 2014, at 02:54, Álvaro Begué wrote: When I had an opportunity to talk to Yann LeCun about a month ago, I asked him if anybody had used convolutional neural networks to play go and he wasn't aware of any efforts in that direction. This is precisely what I had in mind. Thank you very much for the link, Hiroshi! I have been learning about neural networks recently, I have some basic implementation (e.g., no convolutional layers yet), and I am working hard on getting GPU acceleration. One of the things I want to do with NNs is almost exactly what's described in this paper (but including number of liberties and chain length as inputs). I also want to try to use a similar NN to initialize win/loss counts in new nodes in the tree in a MCTS program. One last thing I want to experiment with is providing the results of MC simulations as inputs to the network to improve its performance. I think there is potential for MC to help NNs and viceversa. Does anyone else in this list have an interest in applying these techniques to computer go? Álvaro. On Sun, Dec 14, 2014 at 6:53 PM, Hiroshi Yamashita y...@bd.mbn.or.jp wrote: Hi, This paper looks very cool. Teaching Deep Convolutional Neural Networks to Play Go http://arxiv.org/pdf/1412.__3409v1.pdf Thier move prediction got 91% winrate against GNU Go and 14% against Fuego in 19x19. Regards, Hiroshi Yamashita _ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/__mailman/listinfo/computer-go _ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi! On Mon, Dec 15, 2014 at 08:53:45AM +0900, Hiroshi Yamashita wrote: This paper looks very cool. Teaching Deep Convolutional Neural Networks to Play Go http://arxiv.org/pdf/1412.3409v1.pdf Thier move prediction got 91% winrate against GNU Go and 14% against Fuego in 19x19. That's awesome! I was hoping to spend the evenings in the first quarter of 2015 playing with exactly this, it seems I've been far from the only one with this idea - but also that the prediction task really is as easy as I suspected. :-) Thanks a lot for pointing this out. It's a pity they didn't make their predictor open source - I don't look forward to implementing reflectional preservation. Petr Baudis ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Thanks for posting this Hiroshi! Nice to see this neural network revival. It is mostly old ideas, and it is not really surprising to me, but with modern compute power everyone can now see that it works really well. BTW for some related work (not cited), people might be interested to read up on the 90s work of Stoutamire, Enderton, Schraudolph and Enzenberger. Comparing results to old publications is a bit tricky. For example, the things I did in 2001/2002 are reported to achieve around 25% prediction accuracy, which at the time seemed good but is now considered unimpressive. However, in hindsight, an important reason for that number was time pressure and lack of compute power, which is not really related to anything fundamental. Nowadays using nearly the same training mechanism, but with more data and more capacity to learn (i.e., a bigger network), I also get pro-results around 40%. In case you're interested, this paper http://arxiv.org/pdf/1108.4220.pdf by Thomas Wolf has a figure with more recent results (the latest version of Steenvreter is still a little bit better though). Another problem with comparing results is the difficulty to obtain independent test data. I don't think that was done optimally in this case. The problem is that, especially for amateur games, there are a lot of people memorizing and repeating the popular sequences. Also, if you're not careful, it is quite easy to get duplicate games in you dataset (I've had cases where one game was annotated in chinese, and the other (duplicate) in English, or where the board was simply rotated). My solution around this was to always test on games from the most recent pro-tournaments, for which I was certain they could not yet be in the training database. However, even that may not be perfect, because also pro's play popular joseki, which means there will at least be lots of duplicate opening positions. I'm not surprised these systems now work very well as stand alone players against weak opponents. Some years ago David and Thore's move predictors managed to beat me once in a 9-stones handicap game, which indicates that also their system was already stronger than GNU Go. Further, the version of Steenvreter in my Android app at its lowest level is mostly just a move predictor, yet it still wins well over 80% of its games. In my experience, when the strength difference is big, and the game is even, it is usually enough for the strong player to only play good shape moves. The move predictors only break down in complex tactical situations where some form of look-ahead is critical, and the typical shape-related proverbs provide wrong answers. Erik On Mon, Dec 15, 2014 at 12:53 AM, Hiroshi Yamashita y...@bd.mbn.or.jp wrote: Hi, This paper looks very cool. Teaching Deep Convolutional Neural Networks to Play Go http://arxiv.org/pdf/1412.3409v1.pdf Thier move prediction got 91% winrate against GNU Go and 14% against Fuego in 19x19. Regards, Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Chris Maddison also produced very good (in fact much better) results using a deep convolutional network during his internship at Google. Currently waiting for publication approval, I will post the paper once it is passed. Aja On Mon, Dec 15, 2014 at 2:59 PM, Erik van der Werf erikvanderw...@gmail.com wrote: Thanks for posting this Hiroshi! Nice to see this neural network revival. It is mostly old ideas, and it is not really surprising to me, but with modern compute power everyone can now see that it works really well. BTW for some related work (not cited), people might be interested to read up on the 90s work of Stoutamire, Enderton, Schraudolph and Enzenberger. Comparing results to old publications is a bit tricky. For example, the things I did in 2001/2002 are reported to achieve around 25% prediction accuracy, which at the time seemed good but is now considered unimpressive. However, in hindsight, an important reason for that number was time pressure and lack of compute power, which is not really related to anything fundamental. Nowadays using nearly the same training mechanism, but with more data and more capacity to learn (i.e., a bigger network), I also get pro-results around 40%. In case you're interested, this paper http://arxiv.org/pdf/1108.4220.pdf by Thomas Wolf has a figure with more recent results (the latest version of Steenvreter is still a little bit better though). Another problem with comparing results is the difficulty to obtain independent test data. I don't think that was done optimally in this case. The problem is that, especially for amateur games, there are a lot of people memorizing and repeating the popular sequences. Also, if you're not careful, it is quite easy to get duplicate games in you dataset (I've had cases where one game was annotated in chinese, and the other (duplicate) in English, or where the board was simply rotated). My solution around this was to always test on games from the most recent pro-tournaments, for which I was certain they could not yet be in the training database. However, even that may not be perfect, because also pro's play popular joseki, which means there will at least be lots of duplicate opening positions. I'm not surprised these systems now work very well as stand alone players against weak opponents. Some years ago David and Thore's move predictors managed to beat me once in a 9-stones handicap game, which indicates that also their system was already stronger than GNU Go. Further, the version of Steenvreter in my Android app at its lowest level is mostly just a move predictor, yet it still wins well over 80% of its games. In my experience, when the strength difference is big, and the game is even, it is usually enough for the strong player to only play good shape moves. The move predictors only break down in complex tactical situations where some form of look-ahead is critical, and the typical shape-related proverbs provide wrong answers. Erik On Mon, Dec 15, 2014 at 12:53 AM, Hiroshi Yamashita y...@bd.mbn.or.jp wrote: Hi, This paper looks very cool. Teaching Deep Convolutional Neural Networks to Play Go http://arxiv.org/pdf/1412.3409v1.pdf Thier move prediction got 91% winrate against GNU Go and 14% against Fuego in 19x19. Regards, Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
I tested Aya's move prediction strength. Prediction rate is 38.8% (first choice is same as pro's move) against GNU Go 3.7.10 Level 10 winrate games 19x19 0.059 607 13x13 0.170 545 9x9 0.1411020 I was bit surprised there is no big difference from 9x9 to 19x19. But 6% in 19x19 is still low, paper's 91% winrate is really high. It must understand whole board life and death. I'd like to see their sgf vs GNU Go and Fuego. Aya's prediction includes local string capture search. So this result maybe include some look-ahead. Aya uses this move prediction in UCT, playout uses another prediction. Aya gets 50% against GNU Go with 300 playout in 19x19 and 100 in 9x9. Regards, Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
So I read this kind of study with some skepticism. My guess is that the large-scale pattern systems in use by leading programs are already pretty good for their purpose (i.e., progressive bias). Rampant personal and unverified speculation follows... -- I found the 14% win rate against Fuego is potentially impressive, but I didn't get a sense for Fuego's effort level in those games. E.g., Elo ratings. MCTS actually doesn't play particularly well until a sufficient investment is made. I am not sure what to think about winning 91% against Gnu Go. Gnu Go makes a lot of moves based on rules, so it replays games. I found that many of Pebbles games against Gnu Go were move-for-move repeats of previous games, so much so that I had to randomize Pebbles if I wanted to use Gnu Go for calibrating parameters. My guess is that the 91% rate is substantially attributable to the way that Gnu Go's rule set interacts with the positions that the NN likes. This could be a measure of strength, but not necessarily. My impression is that the progressive bias systems in MCTS programs should prioritize interesting moves to search. A good progressive bias system might have a high move prediction rate, but that will be a side-effect of tuning it for its intended purpose. E.g., it is important to search a lot of bad moves because you need to know for *certain* that they are bad. Similarly, it is my impression is that a good progressive bias engine does not have to be a strong stand-alone player. Strong play implies a degree of tactical pattern matching that is not necessary when the system's responsibility is to prioritize moves. Tactical accuracy should be delegated to the search engine. The theoretical prediction is that MCTS search will be (asymptotically) a better judge of tactical results. Finally, I am not a fan of NN in the MCTS architecture. The NN architecture imposes a high CPU burden (e.g., compared to decision trees), and this study didn't produce such a breakthrough in accuracy that I would give away performance. -Original Message- From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Hiroshi Yamashita Sent: Monday, December 15, 2014 10:27 AM To: computer-go@computer-go.org Subject: Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go I tested Aya's move prediction strength. Prediction rate is 38.8% (first choice is same as pro's move) against GNU Go 3.7.10 Level 10 winrate games 19x19 0.059 607 13x13 0.170 545 9x9 0.1411020 I was bit surprised there is no big difference from 9x9 to 19x19. But 6% in 19x19 is still low, paper's 91% winrate is really high. It must understand whole board life and death. I'd like to see their sgf vs GNU Go and Fuego. Aya's prediction includes local string capture search. So this result maybe include some look-ahead. Aya uses this move prediction in UCT, playout uses another prediction. Aya gets 50% against GNU Go with 300 playout in 19x19 and 100 in 9x9. Regards, Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
On Mon, Dec 15, 2014 at 02:57:32PM -0500, Brian Sheppard wrote: I found the 14% win rate against Fuego is potentially impressive, but I didn't get a sense for Fuego's effort level in those games. E.g., Elo ratings. MCTS actually doesn't play particularly well until a sufficient investment is made. Generally I'd expect Fuego in the described hardware configurations and time seetings to be in 2k-1d KGS range. I am not sure what to think about winning 91% against Gnu Go. Gnu Go makes a lot of moves based on rules, so it replays games. I found that many of Pebbles games against Gnu Go were move-for-move repeats of previous games, so much so that I had to randomize Pebbles if I wanted to use Gnu Go for calibrating parameters. My guess is that the 91% rate is substantially attributable to the way that Gnu Go's rule set interacts with the positions that the NN likes. This could be a measure of strength, but not necessarily. That's an excellent point! My impression is that the progressive bias systems in MCTS programs should prioritize interesting moves to search. A good progressive bias system might have a high move prediction rate, but that will be a side-effect of tuning it for its intended purpose. E.g., it is important to search a lot of bad moves because you need to know for *certain* that they are bad. That sounds a bit backwards; it's enough to find a single good move, you don't need to confirm that all other moves are worse. Of course sometimes this collapses to the same problem, but not nearly all the time. Similarly, it is my impression is that a good progressive bias engine does not have to be a strong stand-alone player. Strong play implies a degree of tactical pattern matching that is not necessary when the system's responsibility is to prioritize moves. Tactical accuracy should be delegated to the search engine. The theoretical prediction is that MCTS search will be (asymptotically) a better judge of tactical results. I don't think anyone would *aim* to make the move predictor as strong as possible, just that everyone is surprised that it is so strong coincidentally. :-) Still, strong play makes sense for a strong predictor. I believe I can also beat GNUGo 90% of time in blitz settings without doing pretty much *any* concious sequence reading. So I would expect a module that's supposed to mirror my intuition to do the same. Finally, I am not a fan of NN in the MCTS architecture. The NN architecture imposes a high CPU burden (e.g., compared to decision trees), and this study didn't produce such a breakthrough in accuracy that I would give away performance. ...so maybe it is MCTS that has to go! We could be in for more surprises. Don't be emotionally attached to your groups. -- Petr Baudis If you do not work on an important problem, it's unlikely you'll do important work. -- R. Hamming http://www.cs.virginia.edu/~robins/YouAndYourResearch.html ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
You don't need a neural net to predict pro moves at this level. My measurement metric was slightly different, I counted how far down the list of moves the pro move appeared, so matching the pro move scored as 100% and being tenth on a list of 100 moves scored 90%. Combining simple metrics such as 3x3 neighborhood, position on the board, and proximity to previous play, you can easily get to an average score of 85%, without producing noticeably good play, at least without a search to back it up. http://real-me.net/ddyer/go/global-eval.html ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
On 12/15/2014 01:39 PM, Dave Dyer wrote: You don't need a neural net to predict pro moves at this level. My measurement metric was slightly different, I counted how far down the list of moves the pro move appeared, so matching the pro move scored as 100% and being tenth on a list of 100 moves scored 90%. There is a huge difference between matching a pro move or have it #10 on a list of 100 Combining simple metrics such as 3x3 neighborhood, position on the board, and proximity to previous play, you can easily get to an average score of 85%, without producing noticeably good play, at least without a search to back it up. 85% is basically meaningless, I am sure even a mid-kyu player can put a pro-move in the top 15% of 100 moves. Christoph ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
2014-12-15 21:31 GMT+00:00 Petr Baudis pa...@ucw.cz: Still, strong play makes sense for a strong predictor. I believe I can also beat GNUGo 90% of time in blitz settings without doing pretty much *any* concious sequence reading. So I would expect a module that's supposed to mirror my intuition to do the same. I'm very surprised you are so confident in beating GnuGo over 90% of time in *blitz settings*. There are even people complaining he couldn't beat GnuGo http://www.lifein19x19.com/forum/viewtopic.php?f=18t=170 Finally, I am not a fan of NN in the MCTS architecture. The NN architecture imposes a high CPU burden (e.g., compared to decision trees), and this study didn't produce such a breakthrough in accuracy that I would give away performance. ...so maybe it is MCTS that has to go! We could be in for more surprises. Don't be emotionally attached to your groups. Fair enough. :) Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
On Mon, Dec 15, 2014 at 11:03:35PM +, Aja Huang wrote: 2014-12-15 21:31 GMT+00:00 Petr Baudis pa...@ucw.cz: Still, strong play makes sense for a strong predictor. I believe I can also beat GNUGo 90% of time in blitz settings without doing pretty much *any* concious sequence reading. So I would expect a module that's supposed to mirror my intuition to do the same. I'm very surprised you are so confident in beating GnuGo over 90% of time in *blitz settings*. There are even people complaining he couldn't beat GnuGo http://www.lifein19x19.com/forum/viewtopic.php?f=18t=170 Huh, aren't you? I just played quick two games GnuGoBot39 where I tried very hard not to read anything at all, and had no trouble winning. (Well, one of my groups had some trouble but mindless clicking saved it anyway.) Petr Baudis ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Finally, I am not a fan of NN in the MCTS architecture. The NN architecture imposes a high CPU burden (e.g., compared to decision trees), and this study didn't produce such a breakthrough in accuracy that I would give away performance. Is it really such a burden? Supporting the move generator with the NN result high up in the decision tree can't be that expensive. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
2014-12-15 23:29 GMT+00:00 Petr Baudis pa...@ucw.cz: Huh, aren't you? I just played quick two games GnuGoBot39 where I tried very hard not to read anything at all, and had no trouble winning. (Well, one of my groups had some trouble but mindless clicking saved it anyway.) That well explains your level is far beyond GnuGo, probably at least 3k on KGS. That being said, Hiroshi, are you sure there was no problem in your experiment? 6% winning rate against GnuGo on 19x19 seems too low for a predictor of 38.8% accuracy. And yes, in the paper we will show a game that the neural network beat Fuego (or pachi) at 100k playouts / move. Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
RE: MC + NN feedback: One area I'm particularly interested in is using NN to apply knowledge from the tree during the playout. I expect that NNs will have difficulty learning strong tactical play, but a combination of a pre-trained network with re-training based on the MCTS results might be able to apply the knowledge gained in MCTS during the playout to correctly resolve LD situations, semeai, and maybe ko fights. Does anyone else have interest in this? -Mark On Mon, Dec 15, 2014 at 3:58 PM, Aja Huang ajahu...@gmail.com wrote: 2014-12-15 23:29 GMT+00:00 Petr Baudis pa...@ucw.cz: Huh, aren't you? I just played quick two games GnuGoBot39 where I tried very hard not to read anything at all, and had no trouble winning. (Well, one of my groups had some trouble but mindless clicking saved it anyway.) That well explains your level is far beyond GnuGo, probably at least 3k on KGS. That being said, Hiroshi, are you sure there was no problem in your experiment? 6% winning rate against GnuGo on 19x19 seems too low for a predictor of 38.8% accuracy. And yes, in the paper we will show a game that the neural network beat Fuego (or pachi) at 100k playouts / move. Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Is it really such a burden? Well, I have to place my bets on some things and not on others. It seems to me that the costs of a NN must be higher than a system based on decision trees. The convolution NN has a very large parameter space if my reading of the paper is correct. Specifically, it can represent all patterns translated and rotated and matched against all points in parallel. To me, that seems like a good way to mimic the visual cortex, but an inefficient way to match patterns on a Go board. So my bet is on decision trees. The published research on NN will help me to understand the opportunities much better, and I have every expectation that the performance of decision trees should be = NN in every way. E.g., faster, more accurate, easier and faster to tune. I recognize that my approach is full of challenges. E.g., a NN would automatically infer soft qualities such as wall, influence that would have to be provided to a DT as inputs. No free lunch, but again, this is about betting that one technology is (overall) more suitable than another. From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Stefan Kaitschick Sent: Monday, December 15, 2014 6:37 PM To: computer-go@computer-go.org Subject: Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go Finally, I am not a fan of NN in the MCTS architecture. The NN architecture imposes a high CPU burden (e.g., compared to decision trees), and this study didn't produce such a breakthrough in accuracy that I would give away performance. Is it really such a burden? Supporting the move generator with the NN result high up in the decision tree can't be that expensive. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Correct me if I am wrong, but I believe that the CrazyStone approach of team-of-features can be cast in terms of a shallow neural network. The inputs are matched patterns on the board and other local information on atari, previous moves, ko situation, and such. Remi alluded as much on this list sometime after his paper got published. Without having studied the Deep Learning papers in detail, it seems that these are the types of smart features that could be learned by a Deep Neural Net in the first few layers if the input is restricted to just the raw board, but could equally well be provided as domain specific features in order to improve computational efficiency (and perhaps enforce correctness). These approaches may not be all that far apart, other than the depth of the net and the domain specific knowledge used directly. Remi recently mentioned that the number of patterns in more recent versions of CrazyStone also number in the millions. I think the prediction rates for these two approaches are also pretty close. Compare the Deep Learning result to the other recent study of a German group quoted in the Deep Learning paper. The bigger questions to me are related to engine architecture. Are you going to use this as an input to a search? Or are you going to use this directly to play? If the former, it had better be reasonably fast. The latter approach can be far slower, but requires the predictions to be of much higher quality. And the biggest question, how can you make these two approaches interact efficiently? René On Mon, Dec 15, 2014 at 8:00 PM, Brian Sheppard sheppar...@aol.com wrote: Is it really such a burden? Well, I have to place my bets on some things and not on others. It seems to me that the costs of a NN must be higher than a system based on decision trees. The convolution NN has a very large parameter space if my reading of the paper is correct. Specifically, it can represent all patterns translated and rotated and matched against all points in parallel. To me, that seems like a good way to mimic the visual cortex, but an inefficient way to match patterns on a Go board. So my bet is on decision trees. The published research on NN will help me to understand the opportunities much better, and I have every expectation that the performance of decision trees should be = NN in every way. E.g., faster, more accurate, easier and faster to tune. I recognize that my approach is full of challenges. E.g., a NN would automatically infer soft qualities such as wall, influence that would have to be provided to a DT as inputs. No free lunch, but again, this is about betting that one technology is (overall) more suitable than another. *From:* Computer-go [mailto:computer-go-boun...@computer-go.org] *On Behalf Of *Stefan Kaitschick *Sent:* Monday, December 15, 2014 6:37 PM *To:* computer-go@computer-go.org *Subject:* Re: [Computer-go] Teaching Deep Convolutional Neural Networks to Play Go Finally, I am not a fan of NN in the MCTS architecture. The NN architecture imposes a high CPU burden (e.g., compared to decision trees), and this study didn't produce such a breakthrough in accuracy that I would give away performance. Is it really such a burden? Supporting the move generator with the NN result high up in the decision tree can't be that expensive. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
[Computer-go] Teaching Deep Convolutional Neural Networks to Play Go
Hi, This paper looks very cool. Teaching Deep Convolutional Neural Networks to Play Go http://arxiv.org/pdf/1412.3409v1.pdf Thier move prediction got 91% winrate against GNU Go and 14% against Fuego in 19x19. Regards, Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
