Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

The evaluation is always at least as deep as leaves of the tree. Still, you're right that the earlier in the game, the bigger the inherent uncertainty. One thing I don't understand: if the network does a thumbs up or down, instead of answering with a probability, what is the use of MSE? Why not

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Opent to intepretation if this method is brute force. I think it i. Uses huge amounts of CPU power to run simulations and evaluate NN's. Even in chess it was not just about tree search, it needs evaluationfunction ot make sense of the search 2016-02-24 6:52 GMT+02:00 muupan : >

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Congratulations, people at DeepMind! Your paper is very interesting to read. I have a question about the paper. On policy network training it says > On the first pass through the training pipeline, the baseline was set to zero; on the second pass we used the value network vθ(s) as a baseline;

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On Wed, Jan 27, 2016 at 1:46 PM, Aja Huang wrote: > We are very excited to announce that our Go program, AlphaGo, has beaten a > professional player for the first time. AlphaGo beat the European champion > Fan Hui by 5 games to 0. It's interesting to go back nearly a decade

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > One possibility is that 0=loss, 1=win, and the number they are quoting is > sqrt(average((prediction-outcome)^2)). this makes perfectly sense for figure 2. even playouts seem reasonable. But figure 2 is not consistent with the numbers in section

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

Detlef Schmicker: <56b385ce.4080...@physik.de>: >-BEGIN PGP SIGNED MESSAGE- >Hash: SHA1 > >Hi, > >I try to reproduce numbers from section 3: training the value network > >On the test set of kgs games the MSE is 0.37. Is it correct, that the >results are represented as +1 and -1? Looks

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 >> Since all positions of all games in the dataset are used, winrate >> should distributes from 0% to 100%, or -1 to 1, not 1. Then, the >> number 70% could be wrong. MSE is 0.37 just means the average >> error is about 0.6, I think. 0.6 in the

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

I am not sure how exactly they define MSE. If you look at the plot in figure 2b, the MSE at the very beginning of the game (where you can't possibly know anything about the result) is 0.50. That suggests it's something else than your [very sensible] interpretation. Álvaro. On Thu, Feb 4, 2016

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

The positions they used are not from high-quality games. They actually include one last move that is completely random. Álvaro. On Thursday, February 4, 2016, Detlef Schmicker wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hi, > > I try to reproduce numbers from

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I try to reproduce numbers from section 3: training the value network On the test set of kgs games the MSE is 0.37. Is it correct, that the results are represented as +1 and -1? This means, that in a typical board position you get a value of

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Thanks for the response, I do not refer to the finaly used data set: in the referred chapter they state, they have used their kgs dataset in a first try (which is in another part of the paper referred to being a 6d+ data set). Am 04.02.2016 um 18:11

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

I re-read the relevant section and I agree with you. Sorry for adding noise to the conversation. Álvaro. On Thu, Feb 4, 2016 at 12:21 PM, Detlef Schmicker wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Thanks for the response, I do not refer to the finaly

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

That sounds like it'd be the MSE as classification error of the eventual result. I'm currently not able to look at the paper, but couldn't you use a softmax output layer with two nodes and take the probability distribution as winrate? On Thu, Feb 4, 2016 at 8:34 PM, Álvaro Begué

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

I think the error is defined as the difference between the output of the value network and the average output of the simulations done by the policy network (RL) at each position. Hideki Michael Markefka:

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search (value network)

I just want to see how to get 0.5 for the initial position on the board with some definition. One possibility is that 0=loss, 1=win, and the number they are quoting is sqrt(average((prediction-outcome)^2)). On Thu, Feb 4, 2016 at 3:40 PM, Hideki Kato wrote: > I think

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Is the paper still available for download? The direct link appears to be broken. Thanks Oliver On Wed, Feb 3, 2016 at 2:06 AM, Igor Polyakov wrote: > I think it would be an awesome commercial product for strong Go players. > Maybe even if the AI shows the

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

I searched for the file name on the web and found this copy: http://airesearch.com/wp-content/uploads/2016/01/deepmind-mastering-go.pdf Álvaro. On Wed, Feb 3, 2016 at 4:37 AM, Oliver Lewis wrote: > Is the paper still available for download? The direct link appears to be

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On 02.02.2016 17:29, Jim O'Flaherty wrote: AI Software Engineers: Robert, please stop asking our AI for explanations. We don't want to distract it with limited human understanding. And we don't want the Herculean task of coding up that extremely frail and error prone bridge. Currently I do not

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

> > If AlphaGo had lost at least one game, I'd understand how people can have >> an upper bound on its level, but with 5-0 (except for Blitz) it's hard to >> have an upper bound on his level. After all, AlphaGo might just have played >> well enough for crushing Fan Hui, and a weak move while the

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

2016-02-01 12:24 GMT+01:00 Olivier Teytaud : > If AlphaGo had lost at least one game, I'd understand how people can have > an upper bound on its level, but with 5-0 (except for Blitz) it's hard to > have an upper bound on his level. After all, AlphaGo might just have played > well

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

What? You have mixed up things. http://www.europeangodatabase.eu/EGD/Player_Card.php?=17374016 2016-02-02 20:21 GMT+01:00 Olivier Teytaud : >>> If AlphaGo had lost at least one game, I'd understand how people can have >>> an upper bound on its level, but with 5-0

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On 02.02.2016 20:21, Olivier Teytaud wrote: On the other hand, they have super strong people in the team (at the pro level, maybe ? if Aja has pro level...) Ca. 5d amateur in the team is enough, regardless of whether Myongwan Kim thinks that only 9p can understand. Not so. Kim's above 5d

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

I think it would be an awesome commercial product for strong Go players. Maybe even if the AI shows the continuations and the score estimates between different lines, it will give the player enough reasoning to understand why one move is better than the other. On 2016-02-02 8:29, Jim

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On 02.02.2016 19:07, David Fotland wrote: consider some of this as the difference between math and engineering. Math desires rigor. Engineering desires working solutions. When an engineering solution is being described, you shouldn't expect the same level of rigor as in a mathematical proof.

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On 01.02.2016 23:01, Brian Cloutier wrote:> I had to search a lot of papers on MCTS which > mentioned "terminal states" before finding one which defined them. > [...] they defined it as a position where there are no more legal > moves. On 01.02.2016 23:15, Brian Sheppard wrote: You play until

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Hi George, welcome, and thanks for your valuable hint on the Google-whitepaper. Do/did you have/see any cross-relations between your research and computer Go?   Cheers, Ingo.   Gesendet: Dienstag, 02. Februar 2016 um 05:14 Uhr Von: "George Dahl" An: computer-go

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

And to meta this awesome short story... AI Software Engineers: Robert, please stop asking our AI for explanations. We don't want to distract it with limited human understanding. And we don't want the Herculean task of coding up that extremely frail and error prone bridge. On Feb 1, 2016 3:03 PM,

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

> Without clarity, progress is delayed. Every professor at university will > confirm this to you. > IMHO, Petr contributed enough to academic research for not needing a discussion with a professor at university for learning how to do/clarify research :-) --

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On 02.02.2016 11:49, Petr Baudis wrote: you seem to come off as perhaps a little too aggressive in your recent few emails... If I were not aggressively critical about inappropriate ambiguity, it would continue for further decades. Papers containing mathematical contents must clarify when

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Ingo Althofer: : >Hi Hideki, > >first of all congrats to the nice performance of Zen over the weekend! > >> Ingo and all, >> Why you care AlphaGo and DCNN so much? > >I can speak only for myself. DCNNs may be not only

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

~~ Robert: "Hey, AI, you should provide explanations!" AI: "Why?" ~~ Cheers, Rainer Date: Mon, 1 Feb 2016 08:15:12 -0600 From: "Jim O'Flaherty" To: computer-go@computer-go.org

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Hi Hideki, you put it wonderfully into two lines: ** ** ****** *** Much more economical methods should be

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Aja, I read the paper with great interest. [Insert appropriate praises here.] I am trying to understand the part where you use reinforcement learning to improve upon the CNN trained by imitating humans. One thing that is not explained is how to determine that a game is over, particularly when a

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

You play until neither player wishes to make a move. The players are willing to move on any point that is not self-atari, and they are willing to make self-atari plays if capture would result in a Nakade (http://senseis.xmp.net/?Nakade) This correctly plays seki. From: Computer-go

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

> someone cracked Go right before that started. Then I'd have plenty of > time to pick a new research topic." It looks like AlphaGo has > provided. It seems [1] the smart money might be on Lee Sedol: 1. Ke Jie (world champ) – limited strength…but still amazing… Less than 5% chance against Lee

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On Mon, Feb 1, 2016 at 10:19 AM, Darren Cook wrote: > It seems [1] the smart money might be on Lee Sedol: In the DeepMind press conferences ( https://www.youtube.com/watch?v=yR017hmUSC4 - https://www.youtube.com/watch?v=_r3yF4lV0wk ) Demis Hassabis stated, that he was quietly

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

The next type of event could be a new 'Pair Go' Where a human and a program make up a pair, like Mark Zuckerberg and his facebook program against a Google VP and alphaGo. :-) Thomas On Mon, 1 Feb 2016, John Tromp wrote: For those of you who missed it, chess grandmaster Hikaru Nakamura, rated

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Robert, I'm not seeing the ROI in attempting to map human idiosyncratic linguistic systems to/into a Go engine. Which language would be the one to use; English, Chinese, Japanese, etc? As abstraction goes deeper, the nuance of each human language diverges from the others (due to the way the human

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Hi! On Mon, Feb 01, 2016 at 01:38:28PM +, Aja Huang wrote: > On Mon, Feb 1, 2016 at 11:38 AM, Petr Baudis wrote: > > > > That's right, but unless I've overlooked something, I didn't see Fan Hui > > create any complicated fight, there wasn't any semeai or complex > > life

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On Mon, Feb 01, 2016 at 12:24:21PM +0100, Olivier Teytaud wrote: > If AlphaGo had lost at least one game, I'd understand how people can have > an upper bound on its level, but with 5-0 (except for Blitz) it's hard to > have an upper bound on his level. After all, AlphaGo might just have played >

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Olivier Teytaud:

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Hi! On Mon, Feb 01, 2016 at 09:19:56AM +, Darren Cook wrote: > > someone cracked Go right before that started. Then I'd have plenty of > > time to pick a new research topic." It looks like AlphaGo has > > provided. > > It seems [1] the smart money might be on Lee Sedol: > > 1. Ke Jie

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

If AlphaGo had lost at least one game, I'd understand how people can have an upper bound on its level, but with 5-0 (except for Blitz) it's hard to have an upper bound on his level. After all, AlphaGo might just have played well enough for crushing Fan Hui, and a weak move while the position is

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Ok, it's not blitz according to http://senseis.xmp.net/?BlitzGames (limit at 10s/move for Blitz). But really shorter time settings. I've seen (as you all) many posts guessing that AlphaGo will lose, but I find that hard to know. If Fan Hui had won one game, I would say that AlphaGo is not ready

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

For those of you who missed it, chess grandmaster Hikaru Nakamura, rated 2787, recently played a match against the world's top chess program Komodo, rated 3368. Each of the 4 games used a different kind of handicap: Pawn and Move Odds Pawn Odds Exchange Odds 4-Move Odds As you can see, handicaps

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

If anything, the other great DCNN applications predate the application of these methods to Go. Deep neural nets (convnets and other types) have been successfully applied in computer vision, robotics, speech recognition, machine translation, natural language processing, and hosts of other areas.

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On 01.02.2016 15:15, Jim O'Flaherty wrote: I'm not seeing the ROI in attempting to map human idiosyncratic linguistic systems to/into a Go engine. Which language would be the one to use; English, Chinese, Japanese, etc? As abstraction goes deeper, the nuance of each human language diverges from

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Hi Hideki, first of all congrats to the nice performance of Zen over the weekend! > Ingo and all, > Why you care AlphaGo and DCNN so much? I can speak only for myself. DCNNs may be not only applied to achieve better playing strength. One may use them to create playing styles, or bots for go

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On 01.02.2016 14:38, Aja Huang wrote: AlphaGo may do much better in tactical situations than Crazy Stone and Zen. Judging very quickly from the Fan Hui games, AlphaGo's group-local "reading" is very deep and accurate but I'd need to read for myself equally deeply and carefully before I would

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Let me add my congratulations to the chorus. Well done! I'm due for a sabbatical next year. I had been joking, "It sure would be good timing if someone cracked Go right before that started. Then I'd have plenty of time to pick a new research topic." It looks like AlphaGo has provided. On Wed,

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Ingo and all, Why you care AlphaGo and DCNN so much? Surely DeepMind team did a big leap but the big problems, such as detecting double-ko and solving complex positions are left unchanged. Also it's well known that to attack these weakpoint of MCTS bots, the opponents have to be strong

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Explaining why the move is good in human terms is useless goal. Good chess programs cannot do it nor it is meaningful. As the humans and computers have vastly different approach to selecting a move then by the definition have reasons for moves. As an example your second item 'long-term aji', For

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On 31.01.2016 20:28, Peter Drake wrote: pick a new research topic. - explain by the program to human players why MC / DNN play is good in terms of human understanding of the game - incorporate the difficult parts, such as long-term aji - solve the game: prove the correct score, prove a weak

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Ingo, I think you are not alone: When I started computer go 4 years ago I ask a good friend of mine, who did his PhD in Neural Networks back in the 90s, if I have any chance to use them instead of pattern matching and he said, they will probably

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On Thu, Jan 28, 2016 at 3:14 PM, Stefan Kaitschick wrote: > That "value network" is just amazing to me. > It does what computer go failed at for over 20 years, and what MCTS was > designed to sidestep. Thought it worth a mention: Detlef posted about trying to train

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

I always thought the same. But I don't think they tackled the decomposition problem directly. Achieving good(non-terminal) board evaluations must have reduced the problem. If you don't do full playouts, you get much less thrashing between independent problems. It also implies a useful static L

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

On 1/27/16 12:08 PM, Aja Huang wrote: 2016-01-27 18:46 GMT+00:00 Aja Huang >: Hi all, We are very excited to announce that our Go program, AlphaGo, has beaten a professional player for the first time. AlphaGo beat the European

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Hi Simon, do your remember my silly remarks in an email discussion almost a year ago? You had written: >> So, yes, with all the exciting work in DCNN, it is very tempting >> to also do DCNN. But I am not sure if we should do so. And my silly reply had been: > I think that DCNN is somehow in a

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Congratulations! What I find most impressive is the engineering effort, combining so many different parts, which even standalone would be a strong program. I think the design philosophy of using 3 different sources of "go playing" strength is great in it self (and if you read the paper there

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

I think many amateurs would already benefit from a simple blunder check and a short list of viable alternatives and short continuations for every move. If I could leave my PC running over night for a 30s/move analysis at 9d level and then walk through my game with that quality of analysis, I'd be

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

I think such analysis might not bee too usefull. At least chess players think it is not very usefull. Usually for learning you need "wake-up" your brains so computer analysis without reasons probabaly on marginally useful. But very entertaining 2016-01-28 13:27 GMT+02:00 Michael Markefka

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Indeed – Congratulations to Google DeepMind! It’s truly an immense achievement. I’m struggling to think of other examples of reasonably mature and strongly contested AI challenges where a new system has made such a huge improvement over existing systems – and I’m still struggling … Simon Lucas

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Sorry for a typo. I meant > Hello Aja, >   > congratulations to the success of you and the other team memberS! So, not singular, but plural. Ingo. ___ Computer-go mailing list Computer-go@computer-go.org

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

2016-01-27 18:46 GMT+00:00 Aja Huang : > Hi all, > > We are very excited to announce that our Go program, AlphaGo, has beaten a > professional player for the first time. AlphaGo beat the European champion > Fan Hui by 5 games to 0. We hope you enjoy our paper, published in

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Congratulations to Aja! A question to the community. Is anyone going to replicate the experimental results? https://www.quora.com/Is-anyone-replicating-the-experimental-results-of-the-human-level-Go-player-published-by-Google-Deepmind-in-Nature-in-January-2016 ? Jason On Thu, Jan 28, 2016 at

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Wow, excellent results, congratulations Aja & team! I'm surprised to see nothing explicitly on decomposing into subgames (e.g. for semeai). I always thought some kind of adaptive decomposition would be needed to reach pro-strength... I guess you must have looked into this; does this mean that the

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Congratulations to the researchers! On 27.01.2016 21:10, Michael Markefka wrote: I really do hope that this also turns into a good analysis and teaching tool for human player. That would be a fantastic benefit from this advancement in computer Go. The programs successful as computer players

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Congratulations Aja. Do you have a plan to run AlphaGo on KGS? It must be a 9d! Yamato ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Google’s breakthrough is just as impactful as the invention of MCTS. Congratulations to the team. It’s a huge leap for computer go, but more importantly it shows that DNN can be applied to many other difficult problems. I just added an answer. I don’t think anyone will try to exactly

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Congratulations Aja, well done :) On Wed, Jan 27, 2016 at 6:46 PM, Aja Huang wrote: > Hi all, > > We are very excited to announce that our Go program, AlphaGo, has beaten a > professional player for the first time. AlphaGo beat the European champion > Fan Hui by 5 games to

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

I really do hope that this also turns into a good analysis and teaching tool for human player. That would be a fantastic benefit from this advancement in computer Go. On Wed, Jan 27, 2016 at 9:08 PM, Aja Huang wrote: > 2016-01-27 18:46 GMT+00:00 Aja Huang

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Hello Aja,   congratulations to the success of you and the other team member!   To the others: Should we call the game "Goo" in the future, to honour Goo-gles progress? CHeers, Ingo. Gesendet: Mittwoch, 27. Januar 2016 um 19:46 Uhr Von: "Aja Huang" An: 

Re: [Computer-go] Mastering the Game of Go with Deep Neural Networks and Tree Search

Congratulations Aja and David! What an interesting idea to train the value network and surprising power of the cloud! Then, when you will get +400 Elo? :) Hideki Aja Huang: : >Hi all, > >We are very excited to announce that