Re: [Computer-go] Converging to 57%
I've been working on my own AlphaGo replication (code on github https://github.com/brilee/MuGo), and I've found it reasonably easy to hit 45% prediction rate with basic features (stone locations, liberty counts, and turns since last move), and a relatively small network (6 intermediate layers, 32 filters in each layer), using Adam / 10e-4 learning rate. This took ~2 hours on a GTX 960. As others have mentioned, learning shoots up sharply at the start, and there is an extremely slow but steady improvement over time. So I'll experiment with fleshing out more features, increasing size of network, and longer training periods. Brian ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Converging to 57%
There are situations where carefully crafting the minibatches makes sense. For instance, if you are training an image classifier it is good to build the minibatches so the classes are evenly represented. In the case of predicting the next move in go I don't expect this kind of thing will make much of a difference. I got to around 52% on a subset of GoGoD using ideas from the ResNet paper ( https://arxiv.org/abs/1512.03385). I used 128x20 followed by 64x20 and finally 32x20, with skip connections every two layers. I started the training with Adam(1e-4) and later on I lowered it to 1e-5 and eventually 1e-6. The only inputs I use are the signed count of liberties (positive for black, negative for white), the age of each stone capped at 8, and a block of ones indicating where the board is. I'll be happy to share some code if people are interested. Álvaro. On Tue, Aug 23, 2016 at 7:29 AM, Gian-Carlo Pascuttowrote: > On 23/08/2016 11:26, Brian Sheppard wrote: > > The learning rate seems much too high. My experience (which is from > > backgammon rather than Go, among other caveats) is that you need tiny > > learning rates. Tiny, as in 1/TrainingSetSize. > > I think that's overkill, as in you effectively end up doing batch > gradient descent instead of mini-batch/SGD. > > But yes, 0.01 is rather high with momentum. Try 0.001 for methods with > momentum, and with the default Adam parameters you have to go even lower > and try 0.0001. > > > Neural networks are dark magic. Be prepared to spend many weeks just > > trying to figure things out. You can bet that the Google & FB results > > are just their final runs. > > As always it's sad nobody publishes what didn't work saving us the time > of trying it all over again :-) > > > Changing batching to match DarkForest style (making sure that a > > minibatch contains samples from game phases... for example > > beginning, middle and end-game). > > This sounds a bit suspicious. The entries in your minibatch should be > randomly selected from your entire training set, so statistically having > positions from all phases would be guaranteed. (Or you can shuffle the > entire training set before the epoch, instead of randomly picking during > it). > > Don't feed the positions in in-order or from the same game... > > -- > GCP > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
[Computer-go] codecentric Challenge 2016
Hello, in the lasst two years best-of-5 matches were played between a strong human (FJ Dickhut, German 6-dan) and strong bots (CrazyStone in 2014, Zen in 2015). This Fall will see the next edition: Lukas Kraemer (6-dan) - Zen Lukas has won the German Championships in 2013, 2014, and 2015. Zen got the gold medals in 19x19-Go in the last four Computer Olympiads (2011, 2013, 2015, 2016). Playing time will be always Sunday evening (for Central Europe), with start at 18:00 CEST. Venue is the computer room of KGS. Thinking times: 60 minutes basic time, plus 3 times 60 seounds Byoyomi. Japanese rules, komi 6.5 points. Start is soon already. The first game will be played on Sunday, August 28. Sponsor of the event is like in the years before the software company codecentric.de Ingo. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Converging to 57%
On 23/08/2016 11:26, Brian Sheppard wrote: > The learning rate seems much too high. My experience (which is from > backgammon rather than Go, among other caveats) is that you need tiny > learning rates. Tiny, as in 1/TrainingSetSize. I think that's overkill, as in you effectively end up doing batch gradient descent instead of mini-batch/SGD. But yes, 0.01 is rather high with momentum. Try 0.001 for methods with momentum, and with the default Adam parameters you have to go even lower and try 0.0001. > Neural networks are dark magic. Be prepared to spend many weeks just > trying to figure things out. You can bet that the Google & FB results > are just their final runs. As always it's sad nobody publishes what didn't work saving us the time of trying it all over again :-) > Changing batching to match DarkForest style (making sure that a > minibatch contains samples from game phases... for example > beginning, middle and end-game). This sounds a bit suspicious. The entries in your minibatch should be randomly selected from your entire training set, so statistically having positions from all phases would be guaranteed. (Or you can shuffle the entire training set before the epoch, instead of randomly picking during it). Don't feed the positions in in-order or from the same game... -- GCP ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Converging to 57%
The learning rate seems much too high. My experience (which is from backgammon rather than Go, among other caveats) is that you need tiny learning rates. Tiny, as in 1/TrainingSetSize. Neural networks are dark magic. Be prepared to spend many weeks just trying to figure things out. You can bet that the Google & FB results are just their final runs. From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Robert Waite Sent: Tuesday, August 23, 2016 2:40 AM To: computer-go@computer-go.org Subject: [Computer-go] Converging to 57% I had subscribed to this mailing list back with MoGo... and remember probably arguing that the game of go wasn't going to be beat for years and years. I am a little late to the game now but was curious if anyone here has worked with supervised learning networks like in the AlphaGo paper. I have been training some networks along the lines of the AlphaGo paper and the DarkForest paper.. and a couple others... and am working with a single 660gtx. I know... laugh... but its a fair benchmark and i'm being cheap for the moment. Breaking 50% accuracy is quite challenging... I have attempted many permutations of learning algorithms... and can hit 40% accuracy in perhaps 4-12 hours... depending on the parameters set. Some things I have tried are using default AlphaGo but wtih 128 filters, 32 minibatch size and .01 learning rate, changing the optimizer from vanilla SGD to Adam or RMSProp. Changing batching to match DarkForest style (making sure that a minibatch contains samples from game phases... for example beginning, middle and end-game).Pretty much everything seems to converge at a rate that will really stretch out. I am planning on picking a line and going with it for an extended training but was wondering if anyone has ever gotten close to the convergence rates implied by the DarkForest paper. For comparison... Google team had 50 gpus, spend 3 weeks.. and processed 5440M state/action pairs. The FB team had 4 gpus, spent 2 weeks and processed 150M-200M state/action pairs. Both seemed to get to around 57% accuracy with their networks. I have also been testing them against GnuGo as a baseline.. and find that GnuGo can be beaten rather easily with very little network training... my eye is on Pachi... but have to break 50% accuracy i think to even worry about that. Have also played with reinforcement learning phase... started with learning rate of .01... which i think was too high that does take quite a bit of time on my machine.. so didnt play too much with it yet. Anyway does anyone have any tales of how long it took to break 50%? What is the magic bullet that will help me converge there quickly! Here is a long-view graph of various attempts: https://drive.google.com/file/d/0B0BbrXeL6VyCUFRkMlNPbzV2QTQ/view Red and Blue lines are from another member that ran 32 in a minibatch, .01 learning rate and 128 filters in the middle layers vs. 192. They had 4 k40 gpus I believe. They also used 4 training pairs to 4 validation pairs... so I imagine that is whey they had such a spread. There is a jump in the accuracy which was when learning rate was decreased to .001 I believe. Closer shot: https://drive.google.com/file/d/0B0BbrXeL6VyCRVUxUFJaWVJBdEE/view Most stay between the lines... but looking at both graphs makes me wonder if any of the lines are approaching the convergence of DarkForest. My gut tells me they were onto something... and am rather curious of the playing strength of the DarkForest SL network and the AG SL network. Also... a picture of the network's view on a position... this one was trained to 41% accuracy and played itself greedily. https://drive.google.com/file/d/0B0BbrXeL6VyCNkRmVDBIYldraWs/view Oh... and another thing AG used KGS amateur data... FB and my networks have been trained on pro games only. At one point I tested the 41% network in the image (trained on pro data) and a 44% network trained on amateur (KGS games) against GnuGo... and the pro data network soundly won... and the amateur network soundly lost... so I stuck with pro since. Not sure if the end result is the same... and kinda glad AG team used amateur as that removes the argument that it somehow learned Le Sedol's style. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Converging to 57%
On 23-08-16 08:57, Detlef Schmicker wrote: > So, if somebody is sure, it is measured against GoGod, I think a > number of other go programmers have to think again. I heard them > reaching 51% (e. g. posts by Hiroshi in this list) I trained a 128 x 14 network for Leela 0.7.0 and this gets 51.1% on GoGoD. Something I noticed from the papers is that the prediction percentage keeps going upwards with more epochs, even if slowly, but still clearly up. In my experience my networks converge rather quickly (like >0.5% per epoch after the first), get stuck, get one more 0.5% gain if I lower the learning rate (by a factor 5 or 10) and don't gain any more regardless of what I do thereafter. I do use momentum. IIRC I tested without momentum once and it was worse, and much slower. I did not find any improvement in playing strength from doing Facebook's 3 move prediction. Perhaps it needs much bigger networks than 128 x 12. Adding ladder features also isn't good enough to (consistently) keep the network from playing into them. (And once it's played the first move, you're totally SOL because the resulting positions aren't in the training set and you'll get 99% confidence for continuing the losing ladder moves) I'm currently doing a more systematic comparison of all methods (and GoGoD vs KGS+GoGoD) on 128 x 12, and testing the resulting strength (rather than looking at prediction %). I'll post the results here, if anything definite comes out of it. -- GCP ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Converging to 57%
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, good to start this discussion here. I had the discussion some times, and we (discussion partner and me) were not sure, against which test set the 57% was measured. If trained and tested with kgs 6d+ dataset, it seems reasonable to reach 57% (I reached >55% within 2 weaks, but than changed to GoGod), with test against GoGod I do not get near that value too (around >51% within 3 weeks on gtx 970). By the way, both get roughly the same playing strength in my case. So, if somebody is sure, it is measured against GoGod, I think a number of other go programmers have to think again. I heard them reaching 51% (e. g. posts by Hiroshi in this list) Detlef Am 23.08.2016 um 08:40 schrieb Robert Waite: > I had subscribed to this mailing list back with MoGo... and > remember probably arguing that the game of go wasn't going to be > beat for years and years. I am a little late to the game now but > was curious if anyone here has worked with supervised learning > networks like in the AlphaGo paper. > > I have been training some networks along the lines of the AlphaGo > paper and the DarkForest paper.. and a couple others... and am > working with a single 660gtx. I know... laugh... but its a fair > benchmark and i'm being cheap for the moment. > > Breaking 50% accuracy is quite challenging... I have attempted > many permutations of learning algorithms... and can hit 40% > accuracy in perhaps 4-12 hours... depending on the parameters set. > Some things I have tried are using default AlphaGo but wtih 128 > filters, 32 minibatch size and .01 learning rate, changing the > optimizer from vanilla SGD to Adam or RMSProp. Changing batching to > match DarkForest style (making sure that a minibatch contains > samples from game phases... for example beginning, middle and > end-game).Pretty much everything seems to converge at a rate that > will really stretch out. I am planning on picking a line and going > with it for an extended training but was wondering if anyone has > ever gotten close to the convergence rates implied by the > DarkForest paper. > > For comparison... Google team had 50 gpus, spend 3 weeks.. and > processed 5440M state/action pairs. The FB team had 4 gpus, spent 2 > weeks and processed 150M-200M state/action pairs. Both seemed to > get to around 57% accuracy with their networks. > > I have also been testing them against GnuGo as a baseline.. and > find that GnuGo can be beaten rather easily with very little > network training... my eye is on Pachi... but have to break 50% > accuracy i think to even worry about that. > > Have also played with reinforcement learning phase... started with > learning rate of .01... which i think was too high that does > take quite a bit of time on my machine.. so didnt play too much > with it yet. > > Anyway does anyone have any tales of how long it took to break > 50%? What is the magic bullet that will help me converge there > quickly! > > Here is a long-view graph of various attempts: > > https://drive.google.com/file/d/0B0BbrXeL6VyCUFRkMlNPbzV2QTQ/view > > Red and Blue lines are from another member that ran 32 in a > minibatch, .01 learning rate and 128 filters in the middle layers > vs. 192. They had 4 k40 gpus I believe. They also used 4 > training pairs to 4 validation pairs... so I imagine that is > whey they had such a spread. There is a jump in the accuracy which > was when learning rate was decreased to .001 I believe. > > Closer shot: > > https://drive.google.com/file/d/0B0BbrXeL6VyCRVUxUFJaWVJBdEE/view > > Most stay between the lines... but looking at both graphs makes me > wonder if any of the lines are approaching the convergence of > DarkForest. My gut tells me they were onto something... and am > rather curious of the playing strength of the DarkForest SL network > and the AG SL network. > > Also... a picture of the network's view on a position... this one > was trained to 41% accuracy and played itself greedily. > > https://drive.google.com/file/d/0B0BbrXeL6VyCNkRmVDBIYldraWs/view > > Oh... and another thing AG used KGS amateur data... FB and my > networks have been trained on pro games only. At one point I tested > the 41% network in the image (trained on pro data) and a 44% > network trained on amateur (KGS games) against GnuGo... and the pro > data network soundly won... and the amateur network soundly lost... > so I stuck with pro since. Not sure if the end result is the > same... and kinda glad AG team used amateur as that removes the > argument that it somehow learned Le Sedol's style. > > > > ___ Computer-go mailing > list Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJXu/PCAAoJEInWdHg+Znf4WmkP/iMpRo/jI/ZYh83C0CTOh2HB 1nq4cb9CG1fubaOIfH5T7veu3pqQWTxPLUI5FOIN99BkESc0NbuSP5geoyLy0hVY
[Computer-go] Converging to 57%
I had subscribed to this mailing list back with MoGo... and remember probably arguing that the game of go wasn't going to be beat for years and years. I am a little late to the game now but was curious if anyone here has worked with supervised learning networks like in the AlphaGo paper. I have been training some networks along the lines of the AlphaGo paper and the DarkForest paper.. and a couple others... and am working with a single 660gtx. I know... laugh... but its a fair benchmark and i'm being cheap for the moment. Breaking 50% accuracy is quite challenging... I have attempted many permutations of learning algorithms... and can hit 40% accuracy in perhaps 4-12 hours... depending on the parameters set. Some things I have tried are using default AlphaGo but wtih 128 filters, 32 minibatch size and .01 learning rate, changing the optimizer from vanilla SGD to Adam or RMSProp. Changing batching to match DarkForest style (making sure that a minibatch contains samples from game phases... for example beginning, middle and end-game).Pretty much everything seems to converge at a rate that will really stretch out. I am planning on picking a line and going with it for an extended training but was wondering if anyone has ever gotten close to the convergence rates implied by the DarkForest paper. For comparison... Google team had 50 gpus, spend 3 weeks.. and processed 5440M state/action pairs. The FB team had 4 gpus, spent 2 weeks and processed 150M-200M state/action pairs. Both seemed to get to around 57% accuracy with their networks. I have also been testing them against GnuGo as a baseline.. and find that GnuGo can be beaten rather easily with very little network training... my eye is on Pachi... but have to break 50% accuracy i think to even worry about that. Have also played with reinforcement learning phase... started with learning rate of .01... which i think was too high that does take quite a bit of time on my machine.. so didnt play too much with it yet. Anyway does anyone have any tales of how long it took to break 50%? What is the magic bullet that will help me converge there quickly! Here is a long-view graph of various attempts: https://drive.google.com/file/d/0B0BbrXeL6VyCUFRkMlNPbzV2QTQ/view Red and Blue lines are from another member that ran 32 in a minibatch, .01 learning rate and 128 filters in the middle layers vs. 192. They had 4 k40 gpus I believe. They also used 4 training pairs to 4 validation pairs... so I imagine that is whey they had such a spread. There is a jump in the accuracy which was when learning rate was decreased to .001 I believe. Closer shot: https://drive.google.com/file/d/0B0BbrXeL6VyCRVUxUFJaWVJBdEE/view Most stay between the lines... but looking at both graphs makes me wonder if any of the lines are approaching the convergence of DarkForest. My gut tells me they were onto something... and am rather curious of the playing strength of the DarkForest SL network and the AG SL network. Also... a picture of the network's view on a position... this one was trained to 41% accuracy and played itself greedily. https://drive.google.com/file/d/0B0BbrXeL6VyCNkRmVDBIYldraWs/view Oh... and another thing AG used KGS amateur data... FB and my networks have been trained on pro games only. At one point I tested the 41% network in the image (trained on pro data) and a 44% network trained on amateur (KGS games) against GnuGo... and the pro data network soundly won... and the amateur network soundly lost... so I stuck with pro since. Not sure if the end result is the same... and kinda glad AG team used amateur as that removes the argument that it somehow learned Le Sedol's style. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go