Re: [Computer-go] Crazy Stone is back

[Hideki Kato]

valky...@phmp.se: <19f31e7e5cdf310b9afa91f577997...@phmp.se>:
>I think you misunderstood what I wrote,
>if perfect play on 9x9 is 6000 Elo, then if the value function is 3000 
>Elo and MC eval is 2000 Elo with 1 second thinking time then  it might 
>be that the combination of a value function and MC eval ends up being 
>2700 Elo. It could also be that it ends up at 3200 Elo.
>
>I personally believe MC eval alone can be very strong so it might be 
>that the capacity of a neural network is not enough to replace MC eval.
>
>When I wrote "converge to perfection" it was not to claim that the Zero 
>approach reaches perfection, just that it get stronger over time.

How to gurantee the improvement does not stop nor 
oscillate?  Actually, the first instance (40 layer version) 
of AlphaGo Zero stopped improvements in three days (at 
least looks so).

>The interesting question is if old school MC evaluation can fill up the 
>knowledge gaps of the value function.

My point is that.  As value networks approximate the value 
function in vary rough manner (ie, smoother parts) due to 
not enough freedom and/or samples and MC rollouts can 
implement (maybe partly) the detail parts of the function, 
mixing those two could yields better approximation (in 
theory :).

Hideki

>For my own Odin project I am not working on the MC evaluation currently, 
>since only when I a have final neural network solution can I see which 
>weaknesses I need to fix.
>
>Best
>Magnus
>
>
>On 2018-03-05 21:12, Hideki Kato wrote:
>> DCNNs are not magic but just non-linear continuous function
>> approximators with finite freedom and we can provide up to
>> 10^8 samples (board positions) in practice.
>> 
>> Why do most people believe VN can approximate (perfect or
>> near perfect) value function?  What do they estimate the
>> complexity of the value function for 19x19 Go?
>> 
>> valky...@phmp.se: :
>>> My guess is that there is some kind of threshold depending on the
>>> relative strength of MC eval and the value function of the NN.
>>> 
>>> If the value function is stronger than MC eval I would guess MCEval
>>> turns into a bad noisy feature with little benefit.
>>> 
>>> Depending on how strong MC eval is this threshold is probably very
>>> different between engines. Also i can imagine that NN value function 
>>> can
>>> have some gaping holes in its knowledge that even simple MC eval can
>>> patch up. Probably true for supervised learning where training data
>>> probably has a lot of holes since bad moves are not in the data.
>>> 
>>> The Zero approach is different because it should converge to 
>>> perfection
>>> in the limit, thus overcome any weaknesses of the value function early
>>> on. At least in theory.
>>> 
>>> 
>>> On 2018-03-05 14:04, Gian-Carlo Pascutto wrote:
 On 5/03/2018 12:28, valky...@phmp.se wrote:
> Remi twittered more details here (see the discussion with gghideki:
> 
> https://twitter.com/Remi_Coulom/status/969936332205318144
 
 Thank you. So Remi gave up on rollouts as well. Interesting 
 "difference
 of opinion" there with Zen.
 
 Last time I tested this in regular Leela, playouts were beneficial, 
 but
 this was before combined value+policy nets and much more training 
 data
 was available. I do not know what the current status would be.
>>> 
>>> ___
>>> Computer-go mailing list
>>> Computer-go@computer-go.org
>>> http://computer-go.org/mailman/listinfo/computer-go
>___
>Computer-go mailing list
>Computer-go@computer-go.org
>http://computer-go.org/mailman/listinfo/computer-go
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[valkyria]

I think you misunderstood what I wrote,
if perfect play on 9x9 is 6000 Elo, then if the value function is 3000 
Elo and MC eval is 2000 Elo with 1 second thinking time then  it might 
be that the combination of a value function and MC eval ends up being 
2700 Elo. It could also be that it ends up at 3200 Elo.


I personally believe MC eval alone can be very strong so it might be 
that the capacity of a neural network is not enough to replace MC eval.


When I wrote "converge to perfection" it was not to claim that the Zero 
approach reaches perfection, just that it get stronger over time.


The interesting question is if old school MC evaluation can fill up the 
knowledge gaps of the value function.


For my own Odin project I am not working on the MC evaluation currently, 
since only when I a have final neural network solution can I see which 
weaknesses I need to fix.


Best
Magnus


On 2018-03-05 21:12, Hideki Kato wrote:

DCNNs are not magic but just non-linear continuous function
approximators with finite freedom and we can provide up to
10^8 samples (board positions) in practice.

Why do most people believe VN can approximate (perfect or
near perfect) value function?  What do they estimate the
complexity of the value function for 19x19 Go?

valky...@phmp.se: :

My guess is that there is some kind of threshold depending on the
relative strength of MC eval and the value function of the NN.

If the value function is stronger than MC eval I would guess MCEval
turns into a bad noisy feature with little benefit.

Depending on how strong MC eval is this threshold is probably very
different between engines. Also i can imagine that NN value function 
can

have some gaping holes in its knowledge that even simple MC eval can
patch up. Probably true for supervised learning where training data
probably has a lot of holes since bad moves are not in the data.

The Zero approach is different because it should converge to 
perfection

in the limit, thus overcome any weaknesses of the value function early
on. At least in theory.


On 2018-03-05 14:04, Gian-Carlo Pascutto wrote:

On 5/03/2018 12:28, valky...@phmp.se wrote:

Remi twittered more details here (see the discussion with gghideki:

https://twitter.com/Remi_Coulom/status/969936332205318144


Thank you. So Remi gave up on rollouts as well. Interesting 
"difference

of opinion" there with Zen.

Last time I tested this in regular Leela, playouts were beneficial, 
but
this was before combined value+policy nets and much more training 
data

was available. I do not know what the current status would be.


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Hideki Kato]

DCNNs are not magic but just non-linear continuous function 
approximators with finite freedom and we can provide up to 
10^8 samples (board positions) in practice.

Why do most people believe VN can approximate (perfect or 
near perfect) value function?  What do they estimate the 
complexity of the value function for 19x19 Go?

valky...@phmp.se: :
>My guess is that there is some kind of threshold depending on the 
>relative strength of MC eval and the value function of the NN.
>
>If the value function is stronger than MC eval I would guess MCEval 
>turns into a bad noisy feature with little benefit.
>
>Depending on how strong MC eval is this threshold is probably very 
>different between engines. Also i can imagine that NN value function can 
>have some gaping holes in its knowledge that even simple MC eval can 
>patch up. Probably true for supervised learning where training data 
>probably has a lot of holes since bad moves are not in the data.
>
>The Zero approach is different because it should converge to perfection 
>in the limit, thus overcome any weaknesses of the value function early 
>on. At least in theory.
>
>
>On 2018-03-05 14:04, Gian-Carlo Pascutto wrote:
>> On 5/03/2018 12:28, valky...@phmp.se wrote:
>>> Remi twittered more details here (see the discussion with gghideki:
>>> 
>>> https://twitter.com/Remi_Coulom/status/969936332205318144
>> 
>> Thank you. So Remi gave up on rollouts as well. Interesting "difference
>> of opinion" there with Zen.
>> 
>> Last time I tested this in regular Leela, playouts were beneficial, but
>> this was before combined value+policy nets and much more training data
>> was available. I do not know what the current status would be.
>
>___
>Computer-go mailing list
>Computer-go@computer-go.org
>http://computer-go.org/mailman/listinfo/computer-go
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Rémi Coulom]

Hi,

I am not convinced it is better. I guess rollouts can bring some additional 
strength, especially in a CPU-only setting. I'll test this later.

For the moment, my main objective is shogi. I will participate in the World 
Computer Shogi Championship in May. So I am developing a game-independent 
AlphaZero framework. I won't be using playouts in shogi, so I am focusing on 
optimizing search without rollouts.

Rémi

- Mail original -
De: "Gian-Carlo Pascutto" <g...@sjeng.org>
À: computer-go@computer-go.org
Envoyé: Lundi 5 Mars 2018 14:04:51
Objet: Re: [Computer-go] Crazy Stone is back

On 5/03/2018 12:28, valky...@phmp.se wrote:
> Remi twittered more details here (see the discussion with gghideki:
> 
> https://twitter.com/Remi_Coulom/status/969936332205318144

Thank you. So Remi gave up on rollouts as well. Interesting "difference
of opinion" there with Zen.

Last time I tested this in regular Leela, playouts were beneficial, but
this was before combined value+policy nets and much more training data
was available. I do not know what the current status would be.

-- 
GCP
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Dave Dyer]

At 06:31 AM 3/5/2018, valky...@phmp.se wrote:
>My guess is that there is some kind of threshold depending on the relative 
>strength of MC eval and the value function of the NN. 

My experiments suggest it's better to train with much longer MCTS
time than will be used in actual play, so the MCTS values that are
the input to training are as good as possible.


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Gian-Carlo Pascutto]

On 5/03/2018 12:28, valky...@phmp.se wrote:
> Remi twittered more details here (see the discussion with gghideki:
> 
> https://twitter.com/Remi_Coulom/status/969936332205318144

Thank you. So Remi gave up on rollouts as well. Interesting "difference
of opinion" there with Zen.

Last time I tested this in regular Leela, playouts were beneficial, but
this was before combined value+policy nets and much more training data
was available. I do not know what the current status would be.

-- 
GCP
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[valkyria]

Remi twittered more details here (see the discussion with gghideki:

https://twitter.com/Remi_Coulom/status/969936332205318144



On 2018-03-05 10:16, Gian-Carlo Pascutto wrote:

On 28-02-18 07:13, Rémi Coulom wrote:

Hi,

I have just connected the newest version of Crazy Stone to CGOS. It
is based on the AlphaZero approach.


In that regard, are you still using Monte Carlo playouts?

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Gian-Carlo Pascutto]

On 28-02-18 07:13, Rémi Coulom wrote:
> Hi,
> 
> I have just connected the newest version of Crazy Stone to CGOS. It
> is based on the AlphaZero approach.

In that regard, are you still using Monte Carlo playouts?

-- 
GCP
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Álvaro Begué]

> I tried chain pooling too, and it was too slow. It made the network about
twice slower in tensorflow (using tf.unsorted_segment_sum or max). I'd
rather have twice more layers.

tf.unsorted_segment_max didn't exist in the first public release of
TensorFlow, so I requested it just for this purpose (
https://github.com/tensorflow/tensorflow/issues/549). Too bad it's too slow
to be useful.

Thanks for sharing some details of what you have learned so far!

Álvaro.




On Thu, Mar 1, 2018 at 5:48 AM, Rémi Coulom <remi.cou...@free.fr> wrote:

> Hi David,
>
> Thanks for sharing your experiments. It is very interesting.
>
> I tried chain pooling too, and it was too slow. It made the network about
> twice slower in tensorflow (using tf.unsorted_segment_sum or max). I'd
> rather have twice more layers.
>
> I never tried dilated convolutions. That sounds interesting.
>
> The value network of AQ has an interesting architecture. It does not go
> directly from 19x19 to scalar, but works like image-recognition networks,
> with 2x2 pooling until it reaches 1x1. I have not tried it yet, but that
> feels like a good idea.
>
> Rémi
>
> - Mail original -
> De: "David Wu" <lightvec...@gmail.com>
> À: computer-go@computer-go.org
> Envoyé: Mercredi 28 Février 2018 20:04:11
> Objet: Re: [Computer-go] Crazy Stone is back
>
>
>
>
> It's not even just liberties and semeai, it's also eyes. Consider for
> example a large dragon that has miai for 2 eyes in distant locations, and
> the opponent then takes one of them - you'd like the policy net to now
> suggest the other eye-making move far away. And you'd also like the value
> net to distinguish the three situations where the whole group has 2 eyes
> even when they are distant versus the ones where it doesn't.
>
>
> I've been doing experiments with somewhat smaller neural nets (roughly 4-7
> residual blocks = 8-14 layers), without sticking to an idealized "zero"
> approach. I've only experimented with policy nets so far, but presumably
> much of this should also transfer to a value net's understanding too.
>
>
>
> 1. One thing I tried was chain pooling, which was neat, but ultimately
> didn't seem promising:
>
> https://github.com/lightvector/GoNN#chain-pooling
> It solves all of these problems when the strings are solidly connected. It
> helps also when the strings are long but not quite solidly connected too,
> the information still propagates faster than without it. But of course, if
> there are lots of little strings forming a group, diagonal connections,
> bamboo joints, etc, then of course it won't help. And also chain pooling is
> computationally costly, at least in Tensorflow, and it might have negative
> effects on the rest of the neural net that I don't understand.
>
>
>
> 2. A new thing I've been trying recently that actually does seem
> moderately promising is dilated convolutions, although I'm still early in
> testing. They also help increase the speed of information propagation, and
> don't require solidly connected strings, and also are reasonably cheap.
>
>
>
> In particular: my residual blocks have 192 channels, so I tried taking
> several of the later residual blocks in the neural net and making 64 of the
> channels of the first convolution in each block use dilated convolutions
> (leaving 128 channels of regular convolutions), with dilation factors of 2
> or 3. Intuitively, the idea is that earlier blocks could learn to compute
> 2x2 or 3x3 connectivity patterns, and then the dilated convolutions in
> later residual blocks will be able to use that to propagate information
> several spaces at a time across connected groups or dragons.
>
>
> So far, indications are that this works. W hen I looked at it in various
> board positions, it helped in a variety of capturing race and
> large-dragon-two-eye-miai situations, correctly suggesting moves that the
> net without dilated convolutions would fail to find due to the move being
> too far away. Also d ilated convolutions seem pretty cheap - it only
> slightly increases the computational cost of the net.
>
>
> So far, I've found that it doesn't significantly improve the overall loss
> function, presumably because now there are 128 channels instead of 192
> channels of ordinary convolutions, so in return for being better at
> long-distance interactions, the neural net has gotten worse at some local
> tactics. But it also hasn't gotten worse the way it would if I simply
> dropped the number of channels from 192 to 128 without adding any new
> channels, so the dilated convolutions are being "used" for real work.
>
> I'd be curious to hear if anyone else has tried dilated convolutions and
> what results

Re: [Computer-go] Crazy Stone is back

[Rémi Coulom]

Hi David,

Thanks for sharing your experiments. It is very interesting.

I tried chain pooling too, and it was too slow. It made the network about twice 
slower in tensorflow (using tf.unsorted_segment_sum or max). I'd rather have 
twice more layers.

I never tried dilated convolutions. That sounds interesting.

The value network of AQ has an interesting architecture. It does not go 
directly from 19x19 to scalar, but works like image-recognition networks, with 
2x2 pooling until it reaches 1x1. I have not tried it yet, but that feels like 
a good idea.

Rémi

- Mail original -
De: "David Wu" <lightvec...@gmail.com>
À: computer-go@computer-go.org
Envoyé: Mercredi 28 Février 2018 20:04:11
Objet: Re: [Computer-go] Crazy Stone is back




It's not even just liberties and semeai, it's also eyes. Consider for example a 
large dragon that has miai for 2 eyes in distant locations, and the opponent 
then takes one of them - you'd like the policy net to now suggest the other 
eye-making move far away. And you'd also like the value net to distinguish the 
three situations where the whole group has 2 eyes even when they are distant 
versus the ones where it doesn't. 


I've been doing experiments with somewhat smaller neural nets (roughly 4-7 
residual blocks = 8-14 layers), without sticking to an idealized "zero" 
approach. I've only experimented with policy nets so far, but presumably much 
of this should also transfer to a value net's understanding too. 



1. One thing I tried was chain pooling, which was neat, but ultimately didn't 
seem promising: 

https://github.com/lightvector/GoNN#chain-pooling 
It solves all of these problems when the strings are solidly connected. It 
helps also when the strings are long but not quite solidly connected too, the 
information still propagates faster than without it. But of course, if there 
are lots of little strings forming a group, diagonal connections, bamboo 
joints, etc, then of course it won't help. And also chain pooling is 
computationally costly, at least in Tensorflow, and it might have negative 
effects on the rest of the neural net that I don't understand. 



2. A new thing I've been trying recently that actually does seem moderately 
promising is dilated convolutions, although I'm still early in testing. They 
also help increase the speed of information propagation, and don't require 
solidly connected strings, and also are reasonably cheap. 



In particular: my residual blocks have 192 channels, so I tried taking several 
of the later residual blocks in the neural net and making 64 of the channels of 
the first convolution in each block use dilated convolutions (leaving 128 
channels of regular convolutions), with dilation factors of 2 or 3. 
Intuitively, the idea is that earlier blocks could learn to compute 2x2 or 3x3 
connectivity patterns, and then the dilated convolutions in later residual 
blocks will be able to use that to propagate information several spaces at a 
time across connected groups or dragons. 


So far, indications are that this works. W hen I looked at it in various board 
positions, it helped in a variety of capturing race and 
large-dragon-two-eye-miai situations, correctly suggesting moves that the net 
without dilated convolutions would fail to find due to the move being too far 
away. Also d ilated convolutions seem pretty cheap - it only slightly increases 
the computational cost of the net. 


So far, I've found that it doesn't significantly improve the overall loss 
function, presumably because now there are 128 channels instead of 192 channels 
of ordinary convolutions, so in return for being better at long-distance 
interactions, the neural net has gotten worse at some local tactics. But it 
also hasn't gotten worse the way it would if I simply dropped the number of 
channels from 192 to 128 without adding any new channels, so the dilated 
convolutions are being "used" for real work. 

I'd be curious to hear if anyone else has tried dilated convolutions and what 
results they got. If there's anything at all to do other than just add more 
layers, I think they're the most promising thing I know of. 




On Wed, Feb 28, 2018 at 12:34 PM, Rémi Coulom < remi.cou...@free.fr > wrote: 


192 and 256 are the numbers of channels. They are fully connected, so the 
number of 3x3 filters is 192^2, and 256^2. 

Having liberty counts and string size as input helps, but it solves only a 
small part of the problem. You can't read a semeai from just the liberty-count 
information. 

I tried to be clever and find ways to propagate information along strings in 
the network. But all the techniques I tried make the network much slower. 
Adding more layers is simple and works. 

Rémi 

- Mail original - 
De: "Darren Cook" < dar...@dcook.org > 
À: computer-go@computer-go.org 
Envoyé: Mercredi 28 Février 2018 16:43:10 
Objet: Re: [Computer-go] Crazy Stone is back 



> We

Re: [Computer-go] Crazy Stone is back

[Rémi Coulom]

Hi Hiroshi,

Yes, Weights_33_400 was trained on 9x9. None of the Weights bot uses playouts. 
I experimented training different network architectures with the same self-play 
data, so that's why newer networks are not necessarily stronger than older ones.

Rémi

- Mail original -
De: "Hiroshi Yamashita" <y...@bd.mbn.or.jp>
À: computer-go@computer-go.org
Envoyé: Mercredi 28 Février 2018 23:24:02
Objet: Re: [Computer-go] Crazy Stone is back

Hi Remi,

Wow, "Weights" is your engine. So my guess was right :-)
In 9x9 CGOS, did you train in 9x9, or just use 19x19 network?
Weights_33_400 is stronger than Weights_40_400.
Maybe it is because Weights_33_400 use CrazyStone's playout, and
  Weights_40_400 does not use?

Thanks,
Hiroshi Yamashita

On 2018/02/28 15:13, Rémi Coulom wrote:
> Hi,
> 
> I have just connected the newest version of Crazy Stone to CGOS. It is based 
> on the AlphaZero approach. The "Weights" engine were in fact previous 
> experimental versions. CrazyStone-18.03 is using time control and pondering 
> instead of a fixed number of evaluations per move. So it should be much 
> stronger than Weights_31_3200.
> 
> Does anybody know who cronus is? It is _extremely_ strong. Its rating is low 
> because it has had only weaker opponents, but it is undefeated so far, except 
> for one loss on time, and some losses against other versions of itself. It 
> has just won two games in a row against Crazy Stone.
> 
> I hope the other strong engines will reconnect, too.
> 
> Rémi
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
> 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Hideki Kato]

A guidline for CGOS users:

Do self-play to pick-up few strongest programs at your site  
(never use CGOS for this purpose) and throw them into CGOS 
to evaluate their ratings among others.  Please note that 
CGOS is the valueable shared resource for all developers.

#We (mainly Hiroshi and me) are managing CGOS in such a 
way that a new program can have solid rating as quick as 
possible (ie, fewer games).  The best (quickest) case could 
be: one new (unrated) program and many anchor-like (ie, 
having stable rating) programs at every 200 to 300 Elo.  In 
this case, every game gives much information to estimate the 
rating of the new program.  More new, unrated programs 
slower the convergence.  Flooding is a very bad idea for 
all.

Best, Hideki

Hiroshi Yamashita: <35aed3ab-235f-adc2-b072-1b317982a...@bd.mbn.or.jp>:
>No Zen on CGOS is pity.
>
>To LZ-0xx-p1600-t1-r1 author,
>I think LZ-073-p1600-t1-r1 has BayesElo already.
>
> From LeelaZero page,
>73 2018-02-05 23:0654bfb7b8
>LZ-54bfb7-t1-p1600, BayesElo is 2903.
>
>Recalculating CGOS rating is not essential.
>And too many same kind bots running makes many selfplay matching.
>Its rating is more unreliable.
>Could you stop them, and run up to two or three bots which has no BayesElo?
>
>Thanks,
>Hiroshi Yamashita
>
>
>On 2018/02/28 17:12, Hideki Kato wrote:
>> Welcome back Remi!
>> 
>> On the 19x19 cgos, recently many LeelaZeros are running.
>> This flood is making CGOS less useful and so I'll reconnect
>> Zen after the flooding ends.  Sorry for inconvinience.
>> 
>> Hideki
>___
>Computer-go mailing list
>Computer-go@computer-go.org
>http://computer-go.org/mailman/listinfo/computer-go
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Hiroshi Yamashita]

Hi Remi,

Wow, "Weights" is your engine. So my guess was right :-)
In 9x9 CGOS, did you train in 9x9, or just use 19x19 network?
Weights_33_400 is stronger than Weights_40_400.
Maybe it is because Weights_33_400 use CrazyStone's playout, and
 Weights_40_400 does not use?

Thanks,
Hiroshi Yamashita

On 2018/02/28 15:13, Rémi Coulom wrote:

Hi,

I have just connected the newest version of Crazy Stone to CGOS. It is based on the 
AlphaZero approach. The "Weights" engine were in fact previous experimental 
versions. CrazyStone-18.03 is using time control and pondering instead of a fixed number 
of evaluations per move. So it should be much stronger than Weights_31_3200.

Does anybody know who cronus is? It is _extremely_ strong. Its rating is low 
because it has had only weaker opponents, but it is undefeated so far, except 
for one loss on time, and some losses against other versions of itself. It has 
just won two games in a row against Crazy Stone.

I hope the other strong engines will reconnect, too.

Rémi
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Andy]

 Rémi, can you share any details about how you are training your network?
Are you doing self-play games? Do you have access to a large number of GPUs?



2018-02-28 13:04 GMT-06:00 David Wu <lightvec...@gmail.com>:

> It's not even just liberties and semeai, it's also eyes. Consider for
> example a large dragon that has miai for 2 eyes in distant locations, and
> the opponent then takes one of them - you'd like the policy net to now
> suggest the other eye-making move far away. And you'd also like the value
> net to distinguish the three situations where the whole group has 2 eyes
> even when they are distant versus the ones where it doesn't.
>
> I've been doing experiments with somewhat smaller neural nets (roughly 4-7
> residual blocks = 8-14 layers), without sticking to an idealized "zero"
> approach. I've only experimented with policy nets so far, but presumably
> much of this should also transfer to a value net's understanding too.
>
> 1. One thing I tried was chain pooling, which was neat, but ultimately
> didn't seem promising:
> https://github.com/lightvector/GoNN#chain-pooling
> It solves all of these problems when the strings are solidly connected. It
> helps also when the strings are long but not quite solidly connected too,
> the information still propagates faster than without it. But of course, if
> there are lots of little strings forming a group, diagonal connections,
> bamboo joints, etc, then of course it won't help. And also chain pooling is
> computationally costly, at least in Tensorflow, and it might have negative
> effects on the rest of the neural net that I don't understand.
>
> 2. A new thing I've been trying recently that actually does seem
> moderately promising is dilated convolutions, although I'm still early in
> testing. They also help increase the speed of information propagation, and
> don't require solidly connected strings, and also are reasonably cheap.
>
> In particular: my residual blocks have 192 channels, so I tried taking
> several of the later residual blocks in the neural net and making 64 of the
> channels of the first convolution in each block use dilated convolutions
> (leaving 128 channels of regular convolutions), with dilation factors of 2
> or 3. Intuitively, the idea is that earlier blocks could learn to compute
> 2x2 or 3x3 connectivity patterns, and then the dilated convolutions in
> later residual blocks will be able to use that to propagate information
> several spaces at a time across connected groups or dragons.
>
> So far, indications are that this works. When I looked at it in various
> board positions, it helped in a variety of capturing race and
> large-dragon-two-eye-miai situations, correctly suggesting moves that the
> net without dilated convolutions would fail to find due to the move being
> too far away. Also dilated convolutions seem pretty cheap - it only
> slightly increases the computational cost of the net.
>
> So far, I've found that it doesn't significantly improve the overall loss
> function, presumably because now there are 128 channels instead of 192
> channels of ordinary convolutions, so in return for being better at
> long-distance interactions, the neural net has gotten worse at some local
> tactics. But it also hasn't gotten worse the way it would if I simply
> dropped the number of channels from 192 to 128 without adding any new
> channels, so the dilated convolutions are being "used" for real work.
>
> I'd be curious to hear if anyone else has tried dilated convolutions and
> what results they got. If there's anything at all to do other than just add
> more layers, I think they're the most promising thing I know of.
>
>
> On Wed, Feb 28, 2018 at 12:34 PM, Rémi Coulom <remi.cou...@free.fr> wrote:
>
>> 192 and 256 are the numbers of channels. They are fully connected, so the
>> number of 3x3 filters is 192^2, and 256^2.
>>
>> Having liberty counts and string size as input helps, but it solves only
>> a small part of the problem. You can't read a semeai from just the
>> liberty-count information.
>>
>> I tried to be clever and find ways to propagate information along strings
>> in the network. But all the techniques I tried make the network much
>> slower. Adding more layers is simple and works.
>>
>> Rémi
>>
>> - Mail original -
>> De: "Darren Cook" <dar...@dcook.org>
>> À: computer-go@computer-go.org
>> Envoyé: Mercredi 28 Février 2018 16:43:10
>> Objet: Re: [Computer-go] Crazy Stone is back
>>
>> > Weights_31_3200 is 20 layers of 192, 3200 board evaluations per move
>> > (no random playout). But it still has difficulties with very long
>> > strings. My ne

Re: [Computer-go] Crazy Stone is back

[David Wu]

It's not even just liberties and semeai, it's also eyes. Consider for
example a large dragon that has miai for 2 eyes in distant locations, and
the opponent then takes one of them - you'd like the policy net to now
suggest the other eye-making move far away. And you'd also like the value
net to distinguish the three situations where the whole group has 2 eyes
even when they are distant versus the ones where it doesn't.

I've been doing experiments with somewhat smaller neural nets (roughly 4-7
residual blocks = 8-14 layers), without sticking to an idealized "zero"
approach. I've only experimented with policy nets so far, but presumably
much of this should also transfer to a value net's understanding too.

1. One thing I tried was chain pooling, which was neat, but ultimately
didn't seem promising:
https://github.com/lightvector/GoNN#chain-pooling
It solves all of these problems when the strings are solidly connected. It
helps also when the strings are long but not quite solidly connected too,
the information still propagates faster than without it. But of course, if
there are lots of little strings forming a group, diagonal connections,
bamboo joints, etc, then of course it won't help. And also chain pooling is
computationally costly, at least in Tensorflow, and it might have negative
effects on the rest of the neural net that I don't understand.

2. A new thing I've been trying recently that actually does seem moderately
promising is dilated convolutions, although I'm still early in testing.
They also help increase the speed of information propagation, and don't
require solidly connected strings, and also are reasonably cheap.

In particular: my residual blocks have 192 channels, so I tried taking
several of the later residual blocks in the neural net and making 64 of the
channels of the first convolution in each block use dilated convolutions
(leaving 128 channels of regular convolutions), with dilation factors of 2
or 3. Intuitively, the idea is that earlier blocks could learn to compute
2x2 or 3x3 connectivity patterns, and then the dilated convolutions in
later residual blocks will be able to use that to propagate information
several spaces at a time across connected groups or dragons.

So far, indications are that this works. When I looked at it in various
board positions, it helped in a variety of capturing race and
large-dragon-two-eye-miai situations, correctly suggesting moves that the
net without dilated convolutions would fail to find due to the move being
too far away. Also dilated convolutions seem pretty cheap - it only
slightly increases the computational cost of the net.

So far, I've found that it doesn't significantly improve the overall loss
function, presumably because now there are 128 channels instead of 192
channels of ordinary convolutions, so in return for being better at
long-distance interactions, the neural net has gotten worse at some local
tactics. But it also hasn't gotten worse the way it would if I simply
dropped the number of channels from 192 to 128 without adding any new
channels, so the dilated convolutions are being "used" for real work.

I'd be curious to hear if anyone else has tried dilated convolutions and
what results they got. If there's anything at all to do other than just add
more layers, I think they're the most promising thing I know of.


On Wed, Feb 28, 2018 at 12:34 PM, Rémi Coulom <remi.cou...@free.fr> wrote:

> 192 and 256 are the numbers of channels. They are fully connected, so the
> number of 3x3 filters is 192^2, and 256^2.
>
> Having liberty counts and string size as input helps, but it solves only a
> small part of the problem. You can't read a semeai from just the
> liberty-count information.
>
> I tried to be clever and find ways to propagate information along strings
> in the network. But all the techniques I tried make the network much
> slower. Adding more layers is simple and works.
>
> Rémi
>
> - Mail original -
> De: "Darren Cook" <dar...@dcook.org>
> À: computer-go@computer-go.org
> Envoyé: Mercredi 28 Février 2018 16:43:10
> Objet: Re: [Computer-go] Crazy Stone is back
>
> > Weights_31_3200 is 20 layers of 192, 3200 board evaluations per move
> > (no random playout). But it still has difficulties with very long
> > strings. My next network will be 40 layers of 256, like Master.
>
> "long strings" here means solidly connected stones?
>
> The 192 vs. 256 is the number of 3x3 convolution filters?
>
> Has anyone been doing experiments with, say, 5x5 filters (and fewer
> layers), and/or putting more raw information in (e.g. liberty counts -
> which makes the long string problem go away, if I've understood
> correctly what that is)?
>
> Darren
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://comp

Re: [Computer-go] Crazy Stone is back

[Rémi Coulom]

192 and 256 are the numbers of channels. They are fully connected, so the 
number of 3x3 filters is 192^2, and 256^2.

Having liberty counts and string size as input helps, but it solves only a 
small part of the problem. You can't read a semeai from just the liberty-count 
information.

I tried to be clever and find ways to propagate information along strings in 
the network. But all the techniques I tried make the network much slower. 
Adding more layers is simple and works.

Rémi

- Mail original -
De: "Darren Cook" <dar...@dcook.org>
À: computer-go@computer-go.org
Envoyé: Mercredi 28 Février 2018 16:43:10
Objet: Re: [Computer-go] Crazy Stone is back

> Weights_31_3200 is 20 layers of 192, 3200 board evaluations per move
> (no random playout). But it still has difficulties with very long
> strings. My next network will be 40 layers of 256, like Master. 

"long strings" here means solidly connected stones?

The 192 vs. 256 is the number of 3x3 convolution filters?

Has anyone been doing experiments with, say, 5x5 filters (and fewer
layers), and/or putting more raw information in (e.g. liberty counts -
which makes the long string problem go away, if I've understood
correctly what that is)?

Darren
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Darren Cook]

> Weights_31_3200 is 20 layers of 192, 3200 board evaluations per move
> (no random playout). But it still has difficulties with very long
> strings. My next network will be 40 layers of 256, like Master. 

"long strings" here means solidly connected stones?

The 192 vs. 256 is the number of 3x3 convolution filters?

Has anyone been doing experiments with, say, 5x5 filters (and fewer
layers), and/or putting more raw information in (e.g. liberty counts -
which makes the long string problem go away, if I've understood
correctly what that is)?

Darren
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Rémi Coulom]

Hi,

Thanks Peter for running Leela. I don't think the many LZ instances cause a big 
problem.

It's a pity Zen did not play cronus. cronus is very impressive. The next run of 
Bayeselo might move cronus to the top. zero40b is very strong too. The new wave 
of AlphaZero clones will become considerably stronger than the current Zen.

Weights_31_3200 is 20 layers of 192, 3200 board evaluations per move (no random 
playout). But it still has difficulties with very long strings. My next network 
will be 40 layers of 256, like Master. I expect it will become much stronger.

Rémi

- Mail original -
De: "Peter Wen" <pe...@whitehalmos.org>
À: computer-go@computer-go.org
Envoyé: Mercredi 28 Février 2018 14:38:01
Objet: Re: [Computer-go] Crazy Stone is back



Hi Hiroshi, 


I've turned off half of the LZ instances and hopefully Hideki will run Zen 
again. The various versions of Zen were the most useful high ranking anchors on 
CGOS. 



There have been many changes to LZ's engine supposed to make it stronger, so 
I'd like to know if the ratings are actually different. Different people ran 
the original ones with the wrong configuration as the names were hashes, making 
their ratings unreliable. 


Thank you Hiroshi for hosting CGOS, it's been invaluable as a source of 
verification. 



Peter 


On Wed, Feb 28, 2018 at 3:50 AM, Hiroshi Yamashita < y...@bd.mbn.or.jp > wrote: 


No Zen on CGOS is pity. 

To LZ-0xx-p1600-t1-r1 author, 
I think LZ-073-p1600-t1-r1 has BayesElo already. 

From LeelaZero page, 
73 2018-02-05 23:06 54bfb7b8 
LZ-54bfb7-t1-p1600, BayesElo is 2903. 

Recalculating CGOS rating is not essential. 
And too many same kind bots running makes many selfplay matching. 
Its rating is more unreliable. 
Could you stop them, and run up to two or three bots which has no BayesElo? 

Thanks, 
Hiroshi Yamashita 


On 2018/02/28 17:12, Hideki Kato wrote: 


Welcome back Remi! 

On the 19x19 cgos, recently many LeelaZeros are running. 
This flood is making CGOS less useful and so I'll reconnect 
Zen after the flooding ends. Sorry for inconvinience. 

Hideki 


___ 
Computer-go mailing list 
Computer-go@computer-go.org 
http://computer-go.org/mailman/listinfo/computer-go 

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Peter Wen]

Disclaimer: I only run the LZ-0xx-p1600-t1-r1 bots.

On Wed, Feb 28, 2018 at 8:38 AM, Peter Wen  wrote:

> Hi Hiroshi,
>
> I've turned off half of the LZ instances and hopefully Hideki will run Zen
> again. The various versions of Zen were the most useful high ranking
> anchors on CGOS.
>
> There have been many changes to LZ's engine supposed to make it stronger,
> so I'd like to know if the ratings are actually different. Different people
> ran the original ones with the wrong configuration as the names were
> hashes, making their ratings unreliable.
>
> Thank you Hiroshi for hosting CGOS, it's been invaluable as a source of
> verification.
>
> Peter
>
> On Wed, Feb 28, 2018 at 3:50 AM, Hiroshi Yamashita 
> wrote:
>
>> No Zen on CGOS is pity.
>>
>> To LZ-0xx-p1600-t1-r1 author,
>> I think LZ-073-p1600-t1-r1 has BayesElo already.
>>
>> From LeelaZero page,
>> 73  2018-02-05 23:0654bfb7b8
>> LZ-54bfb7-t1-p1600, BayesElo is 2903.
>>
>> Recalculating CGOS rating is not essential.
>> And too many same kind bots running makes many selfplay matching.
>> Its rating is more unreliable.
>> Could you stop them, and run up to two or three bots which has no
>> BayesElo?
>>
>> Thanks,
>> Hiroshi Yamashita
>>
>>
>> On 2018/02/28 17:12, Hideki Kato wrote:
>>
>>> Welcome back Remi!
>>>
>>> On the 19x19 cgos, recently many LeelaZeros are running.
>>> This flood is making CGOS less useful and so I'll reconnect
>>> Zen after the flooding ends.  Sorry for inconvinience.
>>>
>>> Hideki
>>>
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Peter Wen]

Hi Hiroshi,

I've turned off half of the LZ instances and hopefully Hideki will run Zen
again. The various versions of Zen were the most useful high ranking
anchors on CGOS.

There have been many changes to LZ's engine supposed to make it stronger,
so I'd like to know if the ratings are actually different. Different people
ran the original ones with the wrong configuration as the names were
hashes, making their ratings unreliable.

Thank you Hiroshi for hosting CGOS, it's been invaluable as a source of
verification.

Peter

On Wed, Feb 28, 2018 at 3:50 AM, Hiroshi Yamashita  wrote:

> No Zen on CGOS is pity.
>
> To LZ-0xx-p1600-t1-r1 author,
> I think LZ-073-p1600-t1-r1 has BayesElo already.
>
> From LeelaZero page,
> 73  2018-02-05 23:0654bfb7b8
> LZ-54bfb7-t1-p1600, BayesElo is 2903.
>
> Recalculating CGOS rating is not essential.
> And too many same kind bots running makes many selfplay matching.
> Its rating is more unreliable.
> Could you stop them, and run up to two or three bots which has no BayesElo?
>
> Thanks,
> Hiroshi Yamashita
>
>
> On 2018/02/28 17:12, Hideki Kato wrote:
>
>> Welcome back Remi!
>>
>> On the 19x19 cgos, recently many LeelaZeros are running.
>> This flood is making CGOS less useful and so I'll reconnect
>> Zen after the flooding ends.  Sorry for inconvinience.
>>
>> Hideki
>>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Hiroshi Yamashita]

No Zen on CGOS is pity.

To LZ-0xx-p1600-t1-r1 author,
I think LZ-073-p1600-t1-r1 has BayesElo already.

From LeelaZero page,
73  2018-02-05 23:0654bfb7b8
LZ-54bfb7-t1-p1600, BayesElo is 2903.

Recalculating CGOS rating is not essential.
And too many same kind bots running makes many selfplay matching.
Its rating is more unreliable.
Could you stop them, and run up to two or three bots which has no BayesElo?

Thanks,
Hiroshi Yamashita


On 2018/02/28 17:12, Hideki Kato wrote:

Welcome back Remi!

On the 19x19 cgos, recently many LeelaZeros are running.
This flood is making CGOS less useful and so I'll reconnect
Zen after the flooding ends.  Sorry for inconvinience.

Hideki

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[Hideki Kato]

Welcome back Remi!

On the 19x19 cgos, recently many LeelaZeros are running.  
This flood is making CGOS less useful and so I'll reconnect 
Zen after the flooding ends.  Sorry for inconvinience.

Hideki

Remi Coulom: 
<1656424330.7641525.1519798424452.javamail.r...@spooler6-g27.priv.proxad.net>:
>Hi,

>

>I have just connected the newest version of Crazy Stone to CGOS. It is 
>based on the AlphaZero approach. The "Weights" engine were in fact previous 
>experimental versions. CrazyStone-18.03 is using time control and pondering 
>instead of a fixed number of evaluations per move. So it should be much 
>stronger than Weights_31_3200.

>

>Does anybody know who cronus is? It is _extremely_ strong. Its rating is 
>low because it has had only weaker opponents, but it is undefeated so far, 
>except for one loss on time, and some losses against other versions of 
>itself. It has just won two games in a row against Crazy Stone.

>

>I hope the other strong engines will reconnect, too.

>

>Rémi

>___

>Computer-go mailing list

>Computer-go@computer-go.org

>http://computer-go.org/mailman/listinfo/computer-go
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Crazy Stone is back

[fotland]

I've been on a forced break from go programming since last August, but I hope 
to get back to it soon.

David

-Original Message-
From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of 
Rémi Coulom
Sent: Tuesday, February 27, 2018 10:14 PM
To: computer-go@computer-go.org
Subject: [Computer-go] Crazy Stone is back

Hi,

I have just connected the newest version of Crazy Stone to CGOS. It is based on 
the AlphaZero approach. The "Weights" engine were in fact previous experimental 
versions. CrazyStone-18.03 is using time control and pondering instead of a 
fixed number of evaluations per move. So it should be much stronger than 
Weights_31_3200.

Does anybody know who cronus is? It is _extremely_ strong. Its rating is low 
because it has had only weaker opponents, but it is undefeated so far, except 
for one loss on time, and some losses against other versions of itself. It has 
just won two games in a row against Crazy Stone.

I hope the other strong engines will reconnect, too.

R�mi
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] Crazy Stone is back

[Rémi Coulom]

Hi,

I have just connected the newest version of Crazy Stone to CGOS. It is based on 
the AlphaZero approach. The "Weights" engine were in fact previous experimental 
versions. CrazyStone-18.03 is using time control and pondering instead of a 
fixed number of evaluations per move. So it should be much stronger than 
Weights_31_3200.

Does anybody know who cronus is? It is _extremely_ strong. Its rating is low 
because it has had only weaker opponents, but it is undefeated so far, except 
for one loss on time, and some losses against other versions of itself. It has 
just won two games in a row against Crazy Stone.

I hope the other strong engines will reconnect, too.

Rémi
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go