[Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
On Fri Dec 19 23:17:23 UTC 2014, Aja Huang wrote: We've just submitted our paper to ICLR. We made the draft available at http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf Cool... just out of curiosity, did a back-of-an-envelope estimation of the cost of training your and Clark and Storkey's network, if renting time on AWS GPU instances and came up with: - Clark and Storkey: 125 usd (4 days * 2 instances * 0.65usd/hour) - Yours: 2025usd(cost of Clark and Storkey * 25/7 epochs * 29.4/14.7 action-pairs * 12/8 layers) Probably a bit high for me personally to just spend one weekend for fun, but not outrageous at all in fact, if the same technique was being used by an organization. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
Great work. Looks like the age of nn is here. How does this compare in computation time to a heavy MC move generator? One very minor quibble, I feel like a nag for even mentioning it: You write The most frequently cited reason for the difficulty of Go, compared to games such as Chess, Scrabble or Shogi, is the difficulty of constructing an evaluation function that can differentiate good moves from bad in a given position. If MC has shown anything, it's that computationally, it's much easier to suggest a good move, than to evaluate the position. This is still true with your paper, it's just that the move suggestion has become even better. Stefan ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
On 20.12.2014 09:43, Stefan Kaitschick wrote: If MC has shown anything, it's that computationally, it's much easier to suggest a good move, than to evaluate the position. Such can only mean an improper understanding of positional judgement. Positional judgement depends on reading (or MC simulation of reading) but the reading has a much smaller computational complexity because localisation and quiescience apply. The major aspects of positional judgement are territory and influence. Evaluating influence is much easier than evaluating territory if one uses a partial influence concept: influence stone difference. Its major difficulty is the knowledge of which stones are alive or not, however, MC simulations applied to outside stones should be able to assess such with reasonable certainty fairly quickly. Hence, the major work of positional judgement is assessment of territory. See my book Positional Judgement 1 - Territory for that. By designing (heuristically or using a low level expert system) MC for its methods, territorial positional judgement by MC should be much faster than ordinary MC because much fewer simulations should do. However, it is not as elegant as ordinary MC because some expert knowledge is necessary or must be approximated heuristically. Needless to say, keep the computational complexity of this expert knowledge low. -- robert jasiek ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
Am Samstag, den 20.12.2014, 09:43 +0100 schrieb Stefan Kaitschick: Great work. Looks like the age of nn is here. How does this compare in computation time to a heavy MC move generator? One very minor quibble, I feel like a nag for even mentioning it: You write The most frequently cited reason for the difficulty of Go, compared to games such as Chess, Scrabble or Shogi, is the difficulty of constructing an evaluation function that can differentiate good moves from bad in a given position. If MC has shown anything, it's that computationally, it's much easier to suggest a good move, than to evaluate the position. This is still true with your paper, it's just that the move suggestion has become even better. It is, but I do not think, that this is necessarily a feature of NN. NNs might be a good evaluators, but it is much easier to train them for a move predictor, as it is not easy to get training data sets for an evaluation function?! Detlef P.S.: As we all might be trying to start incorporating NN into our engines, we might bundle our resources, at least for the first start?! Maybe exchanging open source software links for NN. I personally would have started trying NN some time ago, if iOS had OpenCL support, as my aim is to get a strong iPad go program Stefan ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional NeuralNetworks
Hi Aja, I hope you enjoy our work. Comments and questions are welcome. I have three questions. I don't understand minibatch. Does CNN need 0.15sec for a positon, or 0.15sec for 128 positions? ABCDEFGHJ 9. White(O) to move. 8...OO Previous Black move is H5(X) 7..XXXOO.. 6.XXO. 5...X. 4. 3XO... 2OX... 1. ABCDEFGHJ Liberties after move means H7(O) is 5, F8(O) is 6. Liberties means H5(X) is 3, H6(O) is 2. Ladder move means G2(O), not E6(O). Is this correct? Is KGS rank set 9 dan when it plays against Fuego? Regards, Hiroshi Yamashita ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
On 20.12.2014 11:21, Detlef Schmicker wrote: it is not easy to get training data sets for an evaluation function?! You seem be asking for abundant data sets, e.g., with triples Position, Territory, Influence. Indeed, only dozens are available in the literature and need a bit of extra work. Hundreds of available local joseki positions do not fit your purpose, e.g., because also the Stone Difference matters there. However, I suggest a different approach: 1) One strong player (strong enough to be accurate +-1 point of territory when using his known judgement methods) creates a few examples, e.g., by taking the existing examples for territory and adding the influence stone difference. It should be only one player so that the values are created consistently. (If several players are involved, they should discuss and agree on their application of known methods.) 2) Code is implemented and produces sample data sets. 3) The same player judges how far off the sample data are from his own judgement. Thereby, training does not require many thousands of data sets. Instead it requires much of a strong player's time to accurately judge dozens of data sets. In theory, the player could be replaced by program judgement, but I wish happy development of the then necessary additional theory and algorithms! ;) As you see, I suggest human/program collaboration to accelerate program playing strength. Maybe 9p programs can be created without strong players' help, but then we will not understand much in terms of go theory why the programs will excel. For getting much understanding of go theory from programs, human/program collaboration will be necessary anyway. -- robert jasiek ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Computer-go Digest, Vol 59, Issue 25
in Go Using Deep Convolutional Neural Networks Message-ID: CAJRqE7xjy5r=rwxq9pjmoxsv3baca1j1qiq3a-cgfzajnre...@mail.gmail.com Content-Type: text/plain; charset=utf-8 Great work. Looks like the age of nn is here. How does this compare in computation time to a heavy MC move generator? One very minor quibble, I feel like a nag for even mentioning it: You write The most frequently cited reason for the difficulty of Go, compared to games such as Chess, Scrabble or Shogi, is the difficulty of constructing an evaluation function that can differentiate good moves from bad in a given position. If MC has shown anything, it's that computationally, it's much easier to suggest a good move, than to evaluate the position. This is still true with your paper, it's just that the move suggestion has become even better. Stefan -- next part -- An HTML attachment was scrubbed... URL: http://computer-go.org/pipermail/computer-go/attachments/20141220/27c1afe7/attachment-0001.html -- Message: 5 Date: Sat, 20 Dec 2014 10:24:03 +0100 From: Robert Jasiek jas...@snafu.de To: computer-go@computer-go.org Subject: Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks Message-ID: 54954033.8090...@snafu.de Content-Type: text/plain; charset=UTF-8; format=flowed On 20.12.2014 09:43, Stefan Kaitschick wrote: If MC has shown anything, it's that computationally, it's much easier to suggest a good move, than to evaluate the position. Such can only mean an improper understanding of positional judgement. Positional judgement depends on reading (or MC simulation of reading) but the reading has a much smaller computational complexity because localisation and quiescience apply. The major aspects of positional judgement are territory and influence. Evaluating influence is much easier than evaluating territory if one uses a partial influence concept: influence stone difference. Its major difficulty is the knowledge of which stones are alive or not, however, MC simulations applied to outside stones should be able to assess such with reasonable certainty fairly quickly. Hence, the major work of positional judgement is assessment of territory. See my book Positional Judgement 1 - Territory for that. By designing (heuristically or using a low level expert system) MC for its methods, territorial positional judgement by MC should be much faster than ordinary MC because much fewer simulations should do. However, it is not as elegant as ordinary MC because some expert knowledge is necessary or must be approximated heuristically. Needless to say, keep the computational complexity of this expert knowledge low. -- robert jasiek -- Message: 6 Date: Sat, 20 Dec 2014 11:21:16 +0100 From: Detlef Schmicker d...@physik.de To: computer-go@computer-go.org Subject: Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks Message-ID: 1419070876.13427.7.ca...@physik.de Content-Type: text/plain; charset=UTF-8 Am Samstag, den 20.12.2014, 09:43 +0100 schrieb Stefan Kaitschick: Great work. Looks like the age of nn is here. How does this compare in computation time to a heavy MC move generator? One very minor quibble, I feel like a nag for even mentioning it: You write The most frequently cited reason for the difficulty of Go, compared to games such as Chess, Scrabble or Shogi, is the difficulty of constructing an evaluation function that can differentiate good moves from bad in a given position. If MC has shown anything, it's that computationally, it's much easier to suggest a good move, than to evaluate the position. This is still true with your paper, it's just that the move suggestion has become even better. It is, but I do not think, that this is necessarily a feature of NN. NNs might be a good evaluators, but it is much easier to train them for a move predictor, as it is not easy to get training data sets for an evaluation function?! Detlef P.S.: As we all might be trying to start incorporating NN into our engines, we might bundle our resources, at least for the first start?! Maybe exchanging open source software links for NN. I personally would have started trying NN some time ago, if iOS had OpenCL support, as my aim is to get a strong iPad go program Stefan ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go -- Message: 7 Date: Sat, 20 Dec 2014 20:33:30 +0900 From: Hiroshi Yamashita y...@bd.mbn.or.jp To: computer-go@computer-go.org Subject: Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional NeuralNetworks Message-ID: BC600A945BD9499E9D0B86865BA36BD5@i3540 Content-Type: text/plain; format=flowed; charset=UTF-8; reply-type
Re: [Computer-go] Computer-go Digest, Vol 59, Issue 25
(sorry for forgetting to delete the digest before replying :-( ) ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
[Computer-go] Building a database for training CNNs
Hi, There are things a CNN could probably do well, if only we had the right database to train it. I have in mind these two possibilities: * using a CNN as an evaluation function, * using a CNN to estimate ownership for each point (i.e., a number between -1 and 1 that is an estimate of who is going to end up scoring it). So we need a large set of positions labelled with a final score for the game and who ended up scoring each point. I believe the right database to use for this purpose would consist of positions from games played by strong computer opponents which play to maximize score and which play to the bitter end, passing only when the opponent has no dead stones left on the board. I would like to know if you think this would be an interesting resource to have, if you have any recommendations on what engine(s) to use and if you would be willing to collaborate in creating it. Any other comments are welcome too, of course. Cheers, Álvaro. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional NeuralNetworks
2014-12-20 11:33 GMT+00:00 Hiroshi Yamashita y...@bd.mbn.or.jp: I don't understand minibatch. Does CNN need 0.15sec for a positon, or 0.15sec for 128 positions? 0.15 sec for 128 positions. ABCDEFGHJ 9. White(O) to move. 8...OO Previous Black move is H5(X) 7..XXXOO.. 6.XXO. 5...X. 4. 3XO... 2OX... 1. ABCDEFGHJ Liberties after move means H7(O) is 5, F8(O) is 6. Liberties means H5(X) is 3, H6(O) is 2. Ladder move means G2(O), not E6(O). Is this correct? Yes, all correct. Is KGS rank set 9 dan when it plays against Fuego? Yes. Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional NeuralNetworks
Hi Hiroshi, 2014-12-20 3:31 GMT+00:00 Hiroshi Yamashita y...@bd.mbn.or.jp: But it looks playing strength is similar to Clark's CNN. Against GnuGo our 12-layer CNN is about 300 Elo stronger (97% winning rate against 86% based one the same KGS games). Against Fuego using their time setting (10 sec per move on 2 threads) our CNN scored about 30%. To precisely compare with their results, we also ran 6 sec per move (since our CPU is faster) and got 20-25% (against their 12%). So, our network is clearly much stronger. MCTS with CNN is interesting. But CNN (0 playout) vs 1 playout is 67% seems small. Maybe playout is weak? I'm curious if playout uses CNN. MCTS + CNN (10k playouts) scored 67% against CNN alone. Yes the playout is still very simple. In page 6, approximately 5,000 rollouts per move Christopher Clark's CNN used Fuego with 10 seconds a move, 2 threads on an Intel i7-4702MQ. So maybe it is about 40,000 rollouts per move. Thanks for the information. I'll verify that again. Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Building a database for training CNNs
Hi Álvaro, I've done things like that, except I didn't use games by strong computer opponents (none existed at the time), so just human amateur games. In my experience the critical part is in learning about life death. Once you have that, estimating ownership is fairly easy, asymptotically reaching near 100% prediction accuracy toward the end of the game. See the following papers for more details: http://erikvanderwerf.tengen.nl/pubdown/learning_to_score_extended.pdf http://erikvanderwerf.tengen.nl/pubdown/learningLifeandDeath.pdf http://erikvanderwerf.tengen.nl/pubdown/predicting_territory.pdf or just read the second half of my PhD thesis :) http://erikvanderwerf.tengen.nl/pubdown/thesis_erikvanderwerf.pdf BTW it's great to see this list come back from the dead. Thanks Petr Michael! Best, Erik On Sat, Dec 20, 2014 at 4:43 PM, Álvaro Begué alvaro.be...@gmail.com wrote: Hi, There are things a CNN could probably do well, if only we had the right database to train it. I have in mind these two possibilities: * using a CNN as an evaluation function, * using a CNN to estimate ownership for each point (i.e., a number between -1 and 1 that is an estimate of who is going to end up scoring it). So we need a large set of positions labelled with a final score for the game and who ended up scoring each point. I believe the right database to use for this purpose would consist of positions from games played by strong computer opponents which play to maximize score and which play to the bitter end, passing only when the opponent has no dead stones left on the board. I would like to know if you think this would be an interesting resource to have, if you have any recommendations on what engine(s) to use and if you would be willing to collaborate in creating it. Any other comments are welcome too, of course. Cheers, Álvaro. ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
Hi, I am still fighting with the NN slang, but why do you zero-padd the output (page 3: 4 Architecture Training)? From all I read up to now, most are zero-padding the input to make the output fit 19x19?! Thanks for the great work Detlef Am Freitag, den 19.12.2014, 23:17 + schrieb Aja Huang: Hi all, We've just submitted our paper to ICLR. We made the draft available at http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf I hope you enjoy our work. Comments and questions are welcome. Regards, Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
If you start with a 19x19 grid and you take convolutional filters of size 5x5 (as an example), you'll end up with a board of size 15x15, because a 5x5 box can be placed inside a 19x19 board in 15x15 different locations. We can get 19x19 outputs if we allow the 5x5 box to be centered on any point, but then you need to do multiply by values outside of the original 19x19 board. Zero-padding just means you'll use 0 as the value coming from outside the board. You can either prepare a 23x23 matrix with two rows of zeros along the edges, or you can just keep the 19x19 input and do your math carefully so terms outside the board are ignored. On Sat, Dec 20, 2014 at 12:01 PM, Detlef Schmicker d...@physik.de wrote: Hi, I am still fighting with the NN slang, but why do you zero-padd the output (page 3: 4 Architecture Training)? From all I read up to now, most are zero-padding the input to make the output fit 19x19?! Thanks for the great work Detlef Am Freitag, den 19.12.2014, 23:17 + schrieb Aja Huang: Hi all, We've just submitted our paper to ICLR. We made the draft available at http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf I hope you enjoy our work. Comments and questions are welcome. Regards, Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
Thanks for sharing. I'm intrigued by your strategy for integrating with MCTS. It's clear that latency is a challenge for integration. Do you have any statistics on how many searches new nodes had been through by the time the predictor comes back with an estimation? Did you try any prefetching techniques? Because the CNN will guide much of the search at the frontier of the tree, prefetching should be tractable. Did you do any comparisons between your MCTS with and w/o CNN? That's the direction that many of us will be attempting over the next few months it seems :) - Mark On Sat, Dec 20, 2014 at 10:43 AM, Álvaro Begué alvaro.be...@gmail.com wrote: If you start with a 19x19 grid and you take convolutional filters of size 5x5 (as an example), you'll end up with a board of size 15x15, because a 5x5 box can be placed inside a 19x19 board in 15x15 different locations. We can get 19x19 outputs if we allow the 5x5 box to be centered on any point, but then you need to do multiply by values outside of the original 19x19 board. Zero-padding just means you'll use 0 as the value coming from outside the board. You can either prepare a 23x23 matrix with two rows of zeros along the edges, or you can just keep the 19x19 input and do your math carefully so terms outside the board are ignored. On Sat, Dec 20, 2014 at 12:01 PM, Detlef Schmicker d...@physik.de wrote: Hi, I am still fighting with the NN slang, but why do you zero-padd the output (page 3: 4 Architecture Training)? From all I read up to now, most are zero-padding the input to make the output fit 19x19?! Thanks for the great work Detlef Am Freitag, den 19.12.2014, 23:17 + schrieb Aja Huang: Hi all, We've just submitted our paper to ICLR. We made the draft available at http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf I hope you enjoy our work. Comments and questions are welcome. Regards, Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
This would be very similar to the integration I do in Many Faces of Go. The old engine provides a bias to move selection in the tree, but the old engine is single threaded and only does a few hundred evaluations per second. I typically get between 40 and 200 playouts through a node before Old Many Faces adjusts the biases. David -Original Message- From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Mark Wagner Sent: Saturday, December 20, 2014 11:18 AM To: computer-go@computer-go.org Subject: Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks Thanks for sharing. I'm intrigued by your strategy for integrating with MCTS. It's clear that latency is a challenge for integration. Do you have any statistics on how many searches new nodes had been through by the time the predictor comes back with an estimation? Did you try any prefetching techniques? Because the CNN will guide much of the search at the frontier of the tree, prefetching should be tractable. Did you do any comparisons between your MCTS with and w/o CNN? That's the direction that many of us will be attempting over the next few months it seems :) - Mark On Sat, Dec 20, 2014 at 10:43 AM, lvaro Begu alvaro.be...@gmail.com wrote: If you start with a 19x19 grid and you take convolutional filters of size 5x5 (as an example), you'll end up with a board of size 15x15, because a 5x5 box can be placed inside a 19x19 board in 15x15 different locations. We can get 19x19 outputs if we allow the 5x5 box to be centered on any point, but then you need to do multiply by values outside of the original 19x19 board. Zero-padding just means you'll use 0 as the value coming from outside the board. You can either prepare a 23x23 matrix with two rows of zeros along the edges, or you can just keep the 19x19 input and do your math carefully so terms outside the board are ignored. On Sat, Dec 20, 2014 at 12:01 PM, Detlef Schmicker d...@physik.de wrote: Hi, I am still fighting with the NN slang, but why do you zero-padd the output (page 3: 4 Architecture Training)? From all I read up to now, most are zero-padding the input to make the output fit 19x19?! Thanks for the great work Detlef Am Freitag, den 19.12.2014, 23:17 + schrieb Aja Huang: Hi all, We've just submitted our paper to ICLR. We made the draft available at http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf I hope you enjoy our work. Comments and questions are welcome. Regards, Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Building a database for training CNNs
On Sat, Dec 20, 2014 at 9:35 PM, Robert Jasiek jas...@snafu.de wrote: On 20.12.2014 17:04, Erik van der Werf wrote: the critical part is in learning about life death. Once you have that, estimating ownership is fairly easy [...] See the following papers for more details: [...] http://erikvanderwerf.tengen.nl/pubdown/predicting_territory.pdf Estimating ownership or evaluation functions to predict final scores of already played games are other things than estimating potential territory. Therefore I dislike the title of your paper. Apart from lots of simplistic heuristics without relation to human understanding of territorial positional judgement, one thing has become clear to me from your paper: There are two fundamentally different ways of assessing potential territory: 1) So far mainly human go: count territory, do not equate influence as additional territory. 2) So far mainly computer go: count territory, equate influence as additional territory. Human players might think as follows: The player leads by T points. Therefore the opponent has to use his superior influence to make T more new points than the player. Computers think like this: One value is simpler than two values, therefore I combine territory and influence in just one number, the predicted score. Both methods have their advantages and disadvantages, but it does not mean that computers would always have to use (2); they can as well learn to use (1). (1) has the advantage that counting territory (or intersections that are almost territory) is easy for quiet positions. Minor note on your paper: influence and thickness are defined now (see Joseki 2 - Strategy) and influence stone difference and mobility are related concepts if one wants simpler tools. aji has approached a mathematical definition a bit but still some definition work remains. Sure, I tried lots of simple heuristics to ease the learning task for the networks. One might hope that 'deep' networks would be able to learn advanced concepts more easily, perhaps more on par with human understanding, but for the near future that might still just be wishful thinking. At the time I didn't really care much for a fundamental distinction between territory and influence; I just wanted to have a function to predict the outcome of the game for every intersection as well as possible (because it seemed useful as an evaluation function). Intersections colored with high probability for one side tend to coincide with what human players call territory, while mediocre probabilities tend to coincide more with influence. I know there are non-probabilistic ways to define the two, but I'm not sure it really matters. Perhaps the more effective approach is to just go directly for the probability of winning (like MC does). Best, Erik ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
I think many of the programs have a mechanism for dealing with “slow” knowledge. For example in Fuego, you can call a knowledge function for each node that reaches some threshold T of playouts. The new technical challenge is dealing with the GPU. I know nothing about it myself, but from what I read it seems to work best in batch mode - you don’t want to send single positions for GPU evaluation back and forth. My impression is that we will see a combination of both in the future - “normal”, fast knowledge which can be called as initialization in every node, and can be learned by Remi Coulom’s method (e.g. Crazy Stone, Aya, Erica) or by Wistuba’s (e.g. Fuego). And then, on top of that a mechanism to improve the bias using the slower deep networks running on the GPU. It would be wonderful if some of us could work on an open source network evaluator to integrate with Fuego (or pachi or oakfoam). I know that Clark and Storkey are planning to open source theirs, but not in the very near future. I do not know about the plans of the Google DeepMind group, but they do mention something about a strong Go program in their paper :) Martin Thanks for sharing. I'm intrigued by your strategy for integrating with MCTS. It's clear that latency is a challenge for integration. Do you have any statistics on how many searches new nodes had been through by the time the predictor comes back with an estimation? Did you try any prefetching techniques? Because the CNN will guide much of the search at the frontier of the tree, prefetching should be tractable. Did you do any comparisons between your MCTS with and w/o CNN? That's the direction that many of us will be attempting over the next few months it seems :) ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Teaching Deep Convolutional Neural Networks toPlay Go
On Sat, Dec 20, 2014 at 6:16 AM, Hiroshi Yamashita y...@bd.mbn.or.jp wrote: I put two commented games on http://webdocs.cs.ualberta.ca/~mmueller/fuego/Convolutional-Neural-Network.html Thank you for the report. It was fun. I'm also surprised CNN can play move 185 in Game 1. CNN uses 1, 2, or 3 or more liberties info. B libs changed from 4 to 3. And W libs was 3. It looks CNN can not understand this difference, but he could. Indeed surprising, but maybe it just got lucky. With some sense of move history this one would have been more easily explained... Erik ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
Aja wrote: We haven't measured that but I think move history is an important feature since Go is very much about answering the opponent's last move locally (that's also why in Go we have the term tenuki for not answering the last move). I guess you could get some measure of the importance by looking at the weights? ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
[Computer-go] Fuego 1.1 vs current Fuego
I ran Fuego (latest svn version) on our machine (Intel(R) Xeon(R) CPU Aja, the 1.1 release of Fuego is from 2011 and much different from current svn Fuego. Current version has many different kinds of knowledge added. It is also much slower, maybe 5x. If you read my blog it explains this a little more. Martin ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Fuego 1.1 vs current Fuego
Hi Martin, Thanks for the explanation. I'll write Fuego runs 2 (considering early stop) * 2.2 / 3.1 roughly 14200 playouts per move on the empty board using their time limits. Thanks, Aja On Sat, Dec 20, 2014 at 11:27 PM, Martin Mueller mmuel...@ualberta.ca wrote: I ran Fuego (latest svn version) on our machine (Intel(R) Xeon(R) CPU Aja, the 1.1 release of Fuego is from 2011 and much different from current svn Fuego. Current version has many different kinds of knowledge added. It is also much slower, maybe 5x. If you read my blog it explains this a little more. Martin ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
[Computer-go] Interesting paper from Nando de Freitas team, possible replacement for UCT?
Interesting looking paper: On correlation and budget constraints in model-based bandit optimization with application to automatic machine learning, Hoffman, Shahriari, de Freitas, AISTATS 2014 I can't say I've entirely understood yet, but I *think* that: - targets scenario where there are many more arms than we can try, ie trying an arm is quite expensive. - I think this sounds like the situation somewhere around depth 2 to 4 of an MCTS tree, where we only have time to try a couple of different moves? - using Gaussian Processes to model correlations between the arms - means that they dont actually have to try all arms - though I haven't figured out yet where they are getting this correlation information from yet, if they havent tried the arms yet... - in their experiments, their UGap compares favorably with UCBE, which is a variant of UCB (and which UCB is the basis of UCT, right?) From the point of view of UCT, I'm thinking: - might be more principled approach to deciding what to do when there are still few children of an MCTS node - my understanding is that currently we are using 'magic numbers', like 'if there are less than 30 children, then always explore, dont use UCT'? My own interest in this line of thought initially is slightly different angle from replacing UCT. It seems to me that there are lots of hyper-parameters we need to optimize, or choose from, lots of magic numbers, like the 30 magic number for applying UCT, but also things like: - what is the value of adding RAVE? - what is the value of different numbers of playoff? - what if I vary the maximum playoff depth? - etc ... Searching through all these hyper-parameters seems to me to be tedious, and time-consuming, and unprincipled, and I'm looking for a better way, which has taken me through a bunch of papers such as : - Algorithms for hyper-parameter optimization, Bergstra, Bardenet, Bengio, Kegl, NIPS 2007 - Random Search for Hyper-Parameter Optimization, Bergstra, Bengio, JMLR 2012 - Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms, bergstra, yamins, cox, scipy 2013 - (and now) On correlation and budget constraints in model-based bandit optimization with application to automatic machine learning, Hoffman, Shahriari, de Freitas, AISTATS 2014 ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional Neural Networks
Hi Mark, 2014-12-20 19:17 GMT+00:00 Mark Wagner wagner.mar...@gmail.com: Thanks for sharing. I'm intrigued by your strategy for integrating with MCTS. It's clear that latency is a challenge for integration. Do you have any statistics on how many searches new nodes had been through by the time the predictor comes back with an estimation? Did you try any prefetching techniques? Because the CNN will guide much of the search at the frontier of the tree, prefetching should be tractable. Did you do any comparisons between your MCTS with and w/o CNN? That's the direction that many of us will be attempting over the next few months it seems :) I'm glad you like the paper and would consider to attempt. :) Thanks for the interesting suggestions. Regards, Aja - Mark On Sat, Dec 20, 2014 at 10:43 AM, Álvaro Begué alvaro.be...@gmail.com wrote: If you start with a 19x19 grid and you take convolutional filters of size 5x5 (as an example), you'll end up with a board of size 15x15, because a 5x5 box can be placed inside a 19x19 board in 15x15 different locations. We can get 19x19 outputs if we allow the 5x5 box to be centered on any point, but then you need to do multiply by values outside of the original 19x19 board. Zero-padding just means you'll use 0 as the value coming from outside the board. You can either prepare a 23x23 matrix with two rows of zeros along the edges, or you can just keep the 19x19 input and do your math carefully so terms outside the board are ignored. On Sat, Dec 20, 2014 at 12:01 PM, Detlef Schmicker d...@physik.de wrote: Hi, I am still fighting with the NN slang, but why do you zero-padd the output (page 3: 4 Architecture Training)? From all I read up to now, most are zero-padding the input to make the output fit 19x19?! Thanks for the great work Detlef Am Freitag, den 19.12.2014, 23:17 + schrieb Aja Huang: Hi all, We've just submitted our paper to ICLR. We made the draft available at http://www.cs.toronto.edu/~cmaddis/pubs/deepgo.pdf I hope you enjoy our work. Comments and questions are welcome. Regards, Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional NeuralNetworks
Hi Aja, It's more than 5000 playouts but less than 20k. Which version I tried Fuego 1.1(2011, Windows version) on Intel Core i3 540, 2 cores 4 thread. 3.07GHz. I played first 4 moves randomly, and next 4 moves are GamesPlayed28952, Time 6.8, Games/s4249.8 GamesPlayed28750, Time 6.8, Games/s4249.4 GamesPlayed42853, Time 9.7, Games/s4416.4 GamesPlayed32541, Time 7.5, Games/s4357.0 Average is 33000 playout/move. config.txt --- uct_param_search number_threads 2 uct_param_search lock_free 1 uct_param_player reuse_subtree 1 uct_param_player ponder 0 --- C:\Program Files\Fuego\fuego.exe --config config.txt I did not add go_param timelimit 10, because 10sec is default. i7-4702MQ is 4 cores, 8 threads. 2.2 GHz, 3.2 GHz(Turbo Boost) I'm not sure its speed and whether it used 3.2GHz, but I think turbo boost is on when 2 threads. I summed Fuego's cpu time from their first 10 sgfs. Total is 2861 moves, cpu time 21443 sec. 21443 / (2861/2) = 15.0 sec / move It is over 10sec. a bit strange. Regards, Hiroshi Yamashita - Original Message - From: Aja Huang ajahu...@google.com To: computer-go@computer-go.org Cc: y...@bd.mbn.or.jp Sent: Sunday, December 21, 2014 8:16 AM Subject: Re: [Computer-go] Move Evaluation in Go Using Deep Convolutional NeuralNetworks Hi Hiroshi, On Sat, Dec 20, 2014 at 3:31 AM, Hiroshi Yamashita y...@bd.mbn.or.jp wrote: In page 6, approximately 5,000 rollouts per move Christopher Clark's CNN used Fuego with 10 seconds a move, 2 threads on an Intel i7-4702MQ. So maybe it is about 40,000 rollouts per move. I ran Fuego (latest svn version) on our machine (Intel(R) Xeon(R) CPU E5-2687W 0 @ 3.10GHz) for 10 secs with the following config in the empty position (I actually had to manually play 4 moves at the corners to make Fuego out of book, but the speed should be similar anyway): boardsize 19 uct_param_search number_threads 2 uct_param_search lock_free 1 uct_max_memory 10 uct_param_player reuse_subtree 0 uct_param_player ponder 0 uct_param_player resign_threshold 0.1 go_param_rules capture_dead 1 go_rules kgs Count 16648 GamesPlayed16648 Nodes 4989633 Time 8 GameLen435.4 dev=24.7 min=361.0 max=543.0 InTree 6.9 dev=1.9 min=0.0 max=18.0 Aborted0% Games/s2089.3 It's more than 5000 playouts but less than 20k. Which version of Fuego did you run? Did I set anything wrong? I would appreciate if you could help me run Fuego with the strongest and fastest settings. Regards, Aja ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
