Re: [Computer-go] Fwd: Teaching Deep Convolutional Neural Networks to Play Go
Hello Oliver, 2015-03-16 11:58 GMT+00:00 Oliver Lewis ojfle...@yahoo.co.uk: It's impressive that the same network learned to play seven games with just a win/lose signal. It's also interesting that both these teams are in different parts of Google. I assume they are aware of each other's work, but maybe Aja can confirm. The authors are my colleagues at Google DeepMind as on the paper they list DeepMind as their affiliation. Yes we are aware of each other's work. Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Fwd: Teaching Deep Convolutional Neural Networks to Play Go
The important thing is that the games don't have to be played perfectly: They just need to be significantly better than your current model, so you can tweak the model to learn from them. Thats an important incite. I hadnt thought of that. Maybe could combine with some concept of forgetting, eg weight decay, so the net gradually unlearns some of the original, more naive, associations? The important thing is that the games don't have to be played perfectly: They just need to be significantly better than your current model, so you can tweak the model to learn from them. Thats an important incite. I hadnt thought of that. Maybe could combine with some concept of forgetting, eg weight decay, so the net gradually unlearns some of the original, more naive, associations? could combine with some concept of forgetting, eg weight decay, so the net gradually unlearns some of the original, more naive, associations? ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Fwd: Teaching Deep Convolutional Neural Networks to Play Go
To be honest, what I really want is for it to self-learn,... I wonder if even the world's most powerful AI (i.e. the human brain) could self-learn go to, say, strong dan level? I.e. Give a boy genius a go board, the rules, and two years, but don't give him any books, hints, or the chance to play against anyone who has had access to books/teaching. Darren ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Fwd: Teaching Deep Convolutional Neural Networks to Play Go
The human brain is not the most powerful AI, because it fails the A test. I suspect bootstrapping is not very hard. I have recently written a Spanish checkers program starting with no knowledge and I got it to play top-human level checkers within a few weeks. You can build a database of games as you go. The important thing is that the games don't have to be played perfectly: They just need to be significantly better than your current model, so you can tweak the model to learn from them. In the case of checkers the model is the evaluation function, for which I used a small neural network. In the case of go the model could be an evaluation function or it could be something that assigns probabilities to moves. In either case, a database with hundreds of thousands of games played by the current version of the program is very useful to refine the model. A few iterations of (1. build database using current model, 2. train model using database) should figure out how to play the game well without any human knowledge. Álvaro. On Mon, Mar 16, 2015 at 4:52 AM, Darren Cook dar...@dcook.org wrote: To be honest, what I really want is for it to self-learn,... I wonder if even the world's most powerful AI (i.e. the human brain) could self-learn go to, say, strong dan level? I.e. Give a boy genius a go board, the rules, and two years, but don't give him any books, hints, or the chance to play against anyone who has had access to books/teaching. Darren ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Fwd: Teaching Deep Convolutional Neural Networks to Play Go
I was thinking about bootstrapping possibilities, and wondered whether it would be possible to use a shallower mimic net for positional evaluation playouts from a specific depth on after having generated positions with a certain branching factor that typically allows the actual pro move to be included, hopefully finding even stronger moves, which then are fed back as targets for the primary function/net. Perhaps even apply different amounts of shallowness in mimic function NN configuration as well as depth/branching for move tree generation. No idea if there are kind of depth/branching configurations that would make sense or seem promising, given the existing hardware options. On Sun, Mar 15, 2015 at 2:56 AM, Hugh Perkins hughperk...@gmail.com wrote: To be honest, what I really want is for it to self-learn, like David Silver's TreeStrap did for chess, but on the one hand I guess I should start by reproducing the existent, and on the other hand if we need millions of moves to train the net, that's going to make for very slow self-play... Also, David Silver was associated with Aja Huang's paper, and I'm guessing therefore that it is very non-trivial to do, otherwise David Silver would have done it already :-) ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
[Computer-go] Fwd: Teaching Deep Convolutional Neural Networks to Play Go
On Wed, Dec 31, 2014 at 9:29 PM, Hugh Perkins hughperk...@gmail.com wrote:
- finally, started to get a signal, on the kgsgo data :-) Not a very strong
signal, but a signal :-) :
test accuracy: 364/1 3.64%
Up to 35.1% test accuracy for next-move-prediction task now, still 9%
lower than Clarke and Storkey, but gradually moving forward...
- https://github.com/hughperkins/kgsgo-dataset-preprocessor probably
works ok now
- three datasets available:
- test: moves from 100 randomly selected (but fixed) games,
totalling 18860 moves
- train10k: moves from 10,000 randomly selected (but fixed)
games, non-overlapping with test games, about 1.8 million moves
- trainall: moves from all games up to end of 2014,
non-overlapping with test games, about 32 million moves (I think; I've
only used 16 millon of them so far)
- Using 16 million moves from trainall, and 6 layers of 32 5x5
filters, results after 3 epochs:
after epoch 3 6.05158e+07 ms
annealed learning rate: 0.0001 training loss: 4.11055e+07
train accuracy: 5752089/1600 35.9506%
test accuracy: 6623/18860 35.1166%
6.05 * 10^7 milliseconds is about 14 hours. Note that no obvious
overtraining here, so can probably add more layers (eg the 12 clarke
and storkey use), to get more test accuracy.
commandline used for this:
./clconvolve1 dataset=kgsgoall
netdef=32c5{z}-32c5{z}-32c5{z}-32c5{z}-32c5{z}-32c5{z}-500n-361n
numepochs=15 learningrate=0.0001 numtrain=1600
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
