[Computer-go] Understanding statistics for benchmarking

[Urban Hafner]

So,

I’m currently running 200 games against GnuGo to see if a change to my
program made a difference. But I now wonder if that’s enough games as I ran
the same benchmark with the same code (but a different compiler version)
and received different results:

85.5% wins (171 games of 200) the first time (+/- 2.5 according to
gogui-twogtp)
79.0% wins (158 games of 200) the second time (+/- 2.9 according to
gogui-twogtp)

Looking at these results would make me believe that the difference is
significant (the intervals don’t overlap) but then the real difference is
only 13 wins …

My statistics knowledge is sketchy at best but assuming that what
gogui-twogtp calculates is the 95% confidence interval (I’m pretty sure I’m
mixing terms here) it could well be that the difference between the two
runs above is just random.

So, this leads me to two questions:

1. How many games do you normally run to test if a change is significant
“enough”?
2. Any good resources on how to calculate these statistics (i.e. if I
wanted to find the error margin for a 99% confidence interval)?

Urban
-- 
Blog: http://bettong.net/
Twitter: https://twitter.com/ujh
Homepage: http://www.urbanhafner.com/
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Understanding statistics for benchmarking

[Rémi Coulom]

The intervals given by gogui are the standard deviation, not the usual 
95% confidence intervals.


For 95% confidence intervals, you have to multiply the standard 
deviation by two.


And you still have the 5% chance of not being inside the interval, so 
you can still get the occasional non-overlapping intervals.


Likelihood of superiority is an interesting statistical tool:
https://chessprogramming.wikispaces.com/LOS+Table

For more advanced tools for deciding when to stop testing, there is SPRT:
http://www.open-chess.org/viewtopic.php?f=5=2477
https://en.wikipedia.org/wiki/Sequential_probability_ratio_test

Rémi

On 11/03/2015 09:38 AM, Urban Hafner wrote:

So,

I’m currently running 200 games against GnuGo to see if a change to my 
program made a difference. But I now wonder if that’s enough games as 
I ran the same benchmark with the same code (but a different compiler 
version) and received different results:


85.5% wins (171 games of 200) the first time (+/- 2.5 according to 
gogui-twogtp)
79.0% wins (158 games of 200) the second time (+/- 2.9 according to 
gogui-twogtp)


Looking at these results would make me believe that the difference is 
significant (the intervals don’t overlap) but then the real difference 
is only 13 wins …


My statistics knowledge is sketchy at best but assuming that what 
gogui-twogtp calculates is the 95% confidence interval (I’m pretty 
sure I’m mixing terms here) it could well be that the difference 
between the two runs above is just random.


So, this leads me to two questions:

1. How many games do you normally run to test if a change is 
significant “enough”?
2. Any good resources on how to calculate these statistics (i.e. if I 
wanted to find the error margin for a 99% confidence interval)?


Urban
--
Blog: http://bettong.net/
Twitter: https://twitter.com/ujh
Homepage: http://www.urbanhafner.com/


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Understanding statistics for benchmarking

[Urban Hafner]

Thank you Remi!

So the 85.5% +/- 2.5 reported by GoGui would be 85.5% +/- 5 for 95% and
85.5% +/- 7.5. Correct?

And thanks for the table. I think that’s good enough for now. I’ve now
figured out how to calculate the std. deviation myself (it is easy) and
with those two tools together I can now see that 200 games is a bit on the
low end. :) I had expected as much but it’s good to know for sure.

Urban

On Tue, Nov 3, 2015 at 9:46 AM, Rémi Coulom  wrote:

> The intervals given by gogui are the standard deviation, not the usual 95%
> confidence intervals.
>
> For 95% confidence intervals, you have to multiply the standard deviation
> by two.
>
> And you still have the 5% chance of not being inside the interval, so you
> can still get the occasional non-overlapping intervals.
>
> Likelihood of superiority is an interesting statistical tool:
> https://chessprogramming.wikispaces.com/LOS+Table
>
> For more advanced tools for deciding when to stop testing, there is SPRT:
> http://www.open-chess.org/viewtopic.php?f=5=2477
> https://en.wikipedia.org/wiki/Sequential_probability_ratio_test
>
> Rémi
>
>
> On 11/03/2015 09:38 AM, Urban Hafner wrote:
>
>> So,
>>
>> I’m currently running 200 games against GnuGo to see if a change to my
>> program made a difference. But I now wonder if that’s enough games as I ran
>> the same benchmark with the same code (but a different compiler version)
>> and received different results:
>>
>> 85.5% wins (171 games of 200) the first time (+/- 2.5 according to
>> gogui-twogtp)
>> 79.0% wins (158 games of 200) the second time (+/- 2.9 according to
>> gogui-twogtp)
>>
>> Looking at these results would make me believe that the difference is
>> significant (the intervals don’t overlap) but then the real difference is
>> only 13 wins …
>>
>> My statistics knowledge is sketchy at best but assuming that what
>> gogui-twogtp calculates is the 95% confidence interval (I’m pretty sure I’m
>> mixing terms here) it could well be that the difference between the two
>> runs above is just random.
>>
>> So, this leads me to two questions:
>>
>> 1. How many games do you normally run to test if a change is significant
>> “enough”?
>> 2. Any good resources on how to calculate these statistics (i.e. if I
>> wanted to find the error margin for a 99% confidence interval)?
>>
>> Urban
>> --
>> Blog: http://bettong.net/
>> Twitter: https://twitter.com/ujh
>> Homepage: http://www.urbanhafner.com/
>>
>>
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go




-- 
Blog: http://bettong.net/
Twitter: https://twitter.com/ujh
Homepage: http://www.urbanhafner.com/
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Understanding statistics for benchmarking

[Urban Hafner]

On Tue, Nov 3, 2015 at 2:22 PM, Petr Baudis  wrote:

> (The situation is a bit dire if you have limited computing resources.
> I admit that sometimes I didn't follow the above myself in less formal
> exploratory experiments, but at least I tried to look only
> "infrequently", e.g. single check every few hours, only at "round"
> numbers of playouts, etc.  I hope it's not a grave sin.)
>

No kidding. ;) I’m already envisioning a system where I deploy the version
of the bot that I want to test on N Amazon EC2 instances to run all the
games in parallel. But that would get expensive pretty quickly and I’ll
stick to my (work) laptop for now and run the benchmarks during the night.

Urban
-- 
Blog: http://bettong.net/
Twitter: https://twitter.com/ujh
Homepage: http://www.urbanhafner.com/
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Understanding statistics for benchmarking

[Peter Drake]

Here's Orego's Java code for this:

It involves a "two-tailed test for difference of proportions".

I usually run 500-1000 games in each condition. (The exact number depends
on the hardware available at the time.)

On Tue, Nov 3, 2015 at 5:50 AM, Urban Hafner 
wrote:

> Yes, I noticed that too. But luckily that's the one thing I didn't even
> consider doing. Running the same number of games feels like the most
> natural thing to do anyway.
>
> Von meinem iPhone gesendet
>
> > Am 03.11.2015 um 14:22 schrieb Petr Baudis :
> >
> >> On Tue, Nov 03, 2015 at 09:46:00AM +0100, Rémi Coulom wrote:
> >> The intervals given by gogui are the standard deviation, not the usual
> 95%
> >> confidence intervals.
> >>
> >> For 95% confidence intervals, you have to multiply the standard
> deviation by
> >> two.
> >>
> >> And you still have the 5% chance of not being inside the interval, so
> you
> >> can still get the occasional non-overlapping intervals.
> >>
> >> Likelihood of superiority is an interesting statistical tool:
> >> https://chessprogramming.wikispaces.com/LOS+Table
> >>
> >> For more advanced tools for deciding when to stop testing, there is
> SPRT:
> >> http://www.open-chess.org/viewtopic.php?f=5=2477
> >> https://en.wikipedia.org/wiki/Sequential_probability_ratio_test
> >
> > An important corollary to this (noted on this list every few years)
> > is that in the most naive scenario where your statistical test is just
> > SD-based overlap after N games, you should fix your N number of games
> > in advance and not rig it by terminating out of schedule.  If you look
> > at the progress of your playtesting often, you could spot a few moments
> > where the intervals do not overlap, enve if in the long run they
> > typically would.
> >
> > (The situation is a bit dire if you have limited computing resources.
> > I admit that sometimes I didn't follow the above myself in less formal
> > exploratory experiments, but at least I tried to look only
> > "infrequently", e.g. single check every few hours, only at "round"
> > numbers of playouts, etc.  I hope it's not a grave sin.)
> >
> > --
> >Petr Baudis
> >If you have good ideas, good data and fast computers,
> >you can do almost anything. -- Geoffrey Hinton
> > ___
> > Computer-go mailing list
> > Computer-go@computer-go.org
> > http://computer-go.org/mailman/listinfo/computer-go
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>



-- 
Peter Drake
https://sites.google.com/a/lclark.edu/drake/


Significance.java
Description: Binary data
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Understanding statistics for benchmarking

[Petr Baudis]

On Tue, Nov 03, 2015 at 09:46:00AM +0100, Rémi Coulom wrote:
> The intervals given by gogui are the standard deviation, not the usual 95%
> confidence intervals.
> 
> For 95% confidence intervals, you have to multiply the standard deviation by
> two.
> 
> And you still have the 5% chance of not being inside the interval, so you
> can still get the occasional non-overlapping intervals.
> 
> Likelihood of superiority is an interesting statistical tool:
> https://chessprogramming.wikispaces.com/LOS+Table
> 
> For more advanced tools for deciding when to stop testing, there is SPRT:
> http://www.open-chess.org/viewtopic.php?f=5=2477
> https://en.wikipedia.org/wiki/Sequential_probability_ratio_test

An important corollary to this (noted on this list every few years)
is that in the most naive scenario where your statistical test is just
SD-based overlap after N games, you should fix your N number of games
in advance and not rig it by terminating out of schedule.  If you look
at the progress of your playtesting often, you could spot a few moments
where the intervals do not overlap, enve if in the long run they
typically would.

(The situation is a bit dire if you have limited computing resources.
I admit that sometimes I didn't follow the above myself in less formal
exploratory experiments, but at least I tried to look only
"infrequently", e.g. single check every few hours, only at "round"
numbers of playouts, etc.  I hope it's not a grave sin.)

-- 
Petr Baudis
If you have good ideas, good data and fast computers,
you can do almost anything. -- Geoffrey Hinton
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Understanding statistics for benchmarking

[Kahn Jonas]

On Tue, 3 Nov 2015, Urban Hafner wrote:


Thank you Remi!
So the 85.5% +/- 2.5 reported by GoGui would be 85.5% +/- 5 for 95% and 85.5% 
+/- 7.5.
Correct?


Correct.
But you do not need that intervals do not overlap for significativity.
You may divide by $\sqrt{2}$ those intervals before testing if they
overlap (in the limit, of course, but the whole discussion till here has been).
The value $\sqrt{2}$ is when you have played the same number of games,
as you have in your example.
More precisely, you are computing a confidence interval on the
difference of expectations. You would need a few corrections to be
perfectly rigorous, but that should be enouhgh for your needs.

Jonas



And thanks for the table. I think that’s good enough for now. I’ve now figured 
out how
to calculate the std. deviation myself (it is easy) and with those two tools 
together
I can now see that 200 games is a bit on the low end. :) I had expected as much 
but
it’s good to know for sure.

Urban

On Tue, Nov 3, 2015 at 9:46 AM, Rémi Coulom  wrote:
  The intervals given by gogui are the standard deviation, not the usual 95%
  confidence intervals.

  For 95% confidence intervals, you have to multiply the standard deviation
  by two.

  And you still have the 5% chance of not being inside the interval, so you
  can still get the occasional non-overlapping intervals.

  Likelihood of superiority is an interesting statistical tool:
  https://chessprogramming.wikispaces.com/LOS+Table

  For more advanced tools for deciding when to stop testing, there is SPRT:
  http://www.open-chess.org/viewtopic.php?f=5=2477
  https://en.wikipedia.org/wiki/Sequential_probability_ratio_test

  Rémi

  On 11/03/2015 09:38 AM, Urban Hafner wrote:
  So,

  I’m currently running 200 games against GnuGo to see if a change to
  my program made a difference. But I now wonder if that’s enough
  games as I ran the same benchmark with the same code (but a
  different compiler version) and received different results:

  85.5% wins (171 games of 200) the first time (+/- 2.5 according to
  gogui-twogtp)
  79.0% wins (158 games of 200) the second time (+/- 2.9 according to
  gogui-twogtp)

  Looking at these results would make me believe that the difference
  is significant (the intervals don’t overlap) but then the real
  difference is only 13 wins …

  My statistics knowledge is sketchy at best but assuming that what
  gogui-twogtp calculates is the 95% confidence interval (I’m pretty
  sure I’m mixing terms here) it could well be that the difference
  between the two runs above is just random.

  So, this leads me to two questions:

  1. How many games do you normally run to test if a change is
  significant “enough”?
  2. Any good resources on how to calculate these statistics (i.e. if
  I wanted to find the error margin for a 99% confidence interval)?

  Urban
  --
  Blog: http://bettong.net/
  Twitter: https://twitter.com/ujh
  Homepage: http://www.urbanhafner.com/


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go




--
Blog: http://bettong.net/
Twitter: https://twitter.com/ujh
Homepage: http://www.urbanhafner.com/




___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Number of 3x3 patterns

[Detlef Schmicker]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Thanks, but I need them reduced by reflection and rotation symmetries
(and leave the center empty so 3^8 + 3^5 + 3^3 and than reduce)



Am 03.11.2015 um 19:32 schrieb Gonçalo Mendes Ferreira:
> If you are considering only black stone, white, empty and border, 
> ignoring symmetry, wouldn't it be
> 
> 3^9 + 3^6 + 3^4
> 
> 3^9 for patterns away from the border, 3^6 for near the sides and
> 3^4 near the corners, assuming you are also interested in the
> center value.
> 
> This makes 20493, then you need to take out illegal patterns
> (surrounded middle stone). So I'd hint it's close to 2.
> 
> On 03/11/2015 18:17, Detlef Schmicker wrote: I could not find the
> number of 3x3 patterns in Go, if used all symmetrie s.
> 
> Can anybody give me a hint, were to find. Harvesting 4 games I
> get 1093:)
> 
> Thanks, Detlef
>> ___ Computer-go
>> mailing list Computer-go@computer-go.org 
>> http://computer-go.org/mailman/listinfo/computer-go
> 
> ___ Computer-go mailing
> list Computer-go@computer-go.org 
> http://computer-go.org/mailman/listinfo/computer-go
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJWOQRAAAoJEInWdHg+Znf4SCcP/1oQI5ZI4nIMym8c/1gg1OWh
IDqhO3Ie1Rd7nxgDxaXZDaYACZr9UYRsueyLCLNGignLw6FVpXZvfC9YRn+NKenw
fruwUt/lRzfZGhRyxS5R55pq1X8ux1F/+aa63sRo4/SZGFCVCKeX/hsv3RyM6nuw
iuO9BhFSr0upmarcAJiyt4qSg9JWVdrQI7CaWE6C5XdJI3v5zXxV6yG7mcrSl8oy
66fVbpLo++TjWRUZoChn+HSLuq1rTfp6fWPbMsV3Wwfk7Y3kGkBzMMjvjUpVRktv
5Nu3tcFEf2Hts1iudQ/lPTdA9UYPhkSKwD3l4Z8khR6HpxO8kKClbU6g/uXBwLP+
k/ORYx4imeXMhSfgbzBxZAljRqL86cpUT9A0F0+Llqq4lClCq7I+CjZDXamAH8P8
AqpDQhizni0wEy0KLizkwvJ8mZnqg310553wKaZs+BfP2kkAw+iUtrqvkQ6dsk2t
8fgwMT5Yuw/BfvXLHT/Bp2/GSw5Q97yMNXBBdMqvbIpWkann5bwqVmJljz1VV+TU
G8CA9xeIj8HjVETBUIMURZDxTs+RkxT3fIdlEhH0MvreKBEw0Xp/eSUffqINVSk4
pIcUdjTnutQgL/AtxePcnQuGksvB0KPK5zvUscNgXw9US3JUu1DXcJ6mn+y1PRKK
9xaazKCRBfJjAkhnBlzR
=j9hC
-END PGP SIGNATURE-
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] AMAF/RAVE + heavy playouts - is it save?

[Tobias Pfeiffer]

Hi everyone,

I haven't yet caught up on most recent go papers. If what I ask is
answered in one of these, please point there.

It seems everyone is using quite heavy playouts these days (nxn
patterns, atari escapes, opening libraris, lots of stuff that I don't
know yet, ...) - my question is how does that mix with AMAF/RAVE? I
remember from the early papers, that they said it'd be dangerous to do
it with non random playouts and that they shouldn't have too much logic.

Which, well, makes sense (to me) because the argument is that we play
random moves so they are order independent. With patterns that doesn't
hold true anymore.

What's the experience out there? Does it just still work? Does it not
matter because you just "warm up" the tree? Or do you need to be careful
with what heuristics you apply not too break RAVE/AMAF?

Thank you!
Tobi

-- 
www.pragtob.info




signature.asc
Description: OpenPGP digital signature
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Number of 3x3 patterns

[Álvaro Begué]

I get 1107 (954 in the middle + 135 on the edge + 18 on a corner).

Álvaro.



On Tue, Nov 3, 2015 at 2:00 PM, Detlef Schmicker  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Thanks, but I need them reduced by reflection and rotation symmetries
> (and leave the center empty so 3^8 + 3^5 + 3^3 and than reduce)
>
>
>
> Am 03.11.2015 um 19:32 schrieb Gonçalo Mendes Ferreira:
> > If you are considering only black stone, white, empty and border,
> > ignoring symmetry, wouldn't it be
> >
> > 3^9 + 3^6 + 3^4
> >
> > 3^9 for patterns away from the border, 3^6 for near the sides and
> > 3^4 near the corners, assuming you are also interested in the
> > center value.
> >
> > This makes 20493, then you need to take out illegal patterns
> > (surrounded middle stone). So I'd hint it's close to 2.
> >
> > On 03/11/2015 18:17, Detlef Schmicker wrote: I could not find the
> > number of 3x3 patterns in Go, if used all symmetrie s.
> >
> > Can anybody give me a hint, were to find. Harvesting 4 games I
> > get 1093:)
> >
> > Thanks, Detlef
> >> ___ Computer-go
> >> mailing list Computer-go@computer-go.org
> >> http://computer-go.org/mailman/listinfo/computer-go
> >
> > ___ Computer-go mailing
> > list Computer-go@computer-go.org
> > http://computer-go.org/mailman/listinfo/computer-go
> >
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iQIcBAEBAgAGBQJWOQRAAAoJEInWdHg+Znf4SCcP/1oQI5ZI4nIMym8c/1gg1OWh
> IDqhO3Ie1Rd7nxgDxaXZDaYACZr9UYRsueyLCLNGignLw6FVpXZvfC9YRn+NKenw
> fruwUt/lRzfZGhRyxS5R55pq1X8ux1F/+aa63sRo4/SZGFCVCKeX/hsv3RyM6nuw
> iuO9BhFSr0upmarcAJiyt4qSg9JWVdrQI7CaWE6C5XdJI3v5zXxV6yG7mcrSl8oy
> 66fVbpLo++TjWRUZoChn+HSLuq1rTfp6fWPbMsV3Wwfk7Y3kGkBzMMjvjUpVRktv
> 5Nu3tcFEf2Hts1iudQ/lPTdA9UYPhkSKwD3l4Z8khR6HpxO8kKClbU6g/uXBwLP+
> k/ORYx4imeXMhSfgbzBxZAljRqL86cpUT9A0F0+Llqq4lClCq7I+CjZDXamAH8P8
> AqpDQhizni0wEy0KLizkwvJ8mZnqg310553wKaZs+BfP2kkAw+iUtrqvkQ6dsk2t
> 8fgwMT5Yuw/BfvXLHT/Bp2/GSw5Q97yMNXBBdMqvbIpWkann5bwqVmJljz1VV+TU
> G8CA9xeIj8HjVETBUIMURZDxTs+RkxT3fIdlEhH0MvreKBEw0Xp/eSUffqINVSk4
> pIcUdjTnutQgL/AtxePcnQuGksvB0KPK5zvUscNgXw9US3JUu1DXcJ6mn+y1PRKK
> 9xaazKCRBfJjAkhnBlzR
> =j9hC
> -END PGP SIGNATURE-
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Facebook Go AI

[Nick Wedd]

I think this Facebook AI may be the program playing on KGS as darkforest
and darkfores1.

Nick

On 3 November 2015 at 14:28, Petr Baudis  wrote:

>   Hi!
>
>   Facebook is working on a Go AI too, now:
>
> https://www.facebook.com/Engineering/videos/10153621562717200/
> https://code.facebook.com/posts/1478523512478471
>
> http://www.wired.com/2015/11/facebook-is-aiming-its-ai-at-go-the-game-no-computer-can-crack/
>
> The way it's presented triggers my hype alerts, but nevertheless:
> does anyone know any details about this?  Most interestingly, how
> strong is it?
>
> --
> Petr Baudis
> If you have good ideas, good data and fast computers,
> you can do almost anything. -- Geoffrey Hinton
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go




-- 
Nick Wedd  mapr...@gmail.com
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Number of 3x3 patterns

[Detlef Schmicker]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Am 03.11.2015 um 20:24 schrieb Jim O'Flaherty:
> I don't see how "leave the center empty" works as a valid case,
> assuming this it just any valid 3x3 window on the board. Given bots
> playing each other, there can be 9x9 clumps of a stone of the same
> color. I can see it being argued there is no computational value in
> this specific pattern instance. But, then what are the conditions
> of the exceptions to the generalization? And how do you effectively
> iterate through the other +20,000 variations (not reduced by
> location or color symmetry)?
> 
> So, I'm curious, is there some other assumption about the 3x3
> window other than it be a view into any valid 3x3 space on a Go
> board?

Sorry, I did not explain the details, the assumption is:
I play in the middle, so it must be empty. I thought legal moves might
not really reduce the number of 3x3 patterns, as there can be no
suicide known from 3x3 patterns, as a capture is always possible.

Therefore I wonder, what 14 patterns did not appear in my 4 games
harvested:)


> 
> On Tue, Nov 3, 2015 at 1:04 PM, Álvaro Begué
>  wrote:
> 
>> I get 1107 (954 in the middle + 135 on the edge + 18 on a
>> corner).
>> 
>> Álvaro.
>> 
>> 
>> 
>> On Tue, Nov 3, 2015 at 2:00 PM, Detlef Schmicker 
>> wrote:
>> 
> Thanks, but I need them reduced by reflection and rotation
> symmetries (and leave the center empty so 3^8 + 3^5 + 3^3 and than
> reduce)
> 
> 
> 
> Am 03.11.2015 um 19:32 schrieb Gonçalo Mendes Ferreira:
> If you are considering only black stone, white, empty and
> border, ignoring symmetry, wouldn't it be
> 
> 3^9 + 3^6 + 3^4
> 
> 3^9 for patterns away from the border, 3^6 for near the
> sides and 3^4 near the corners, assuming you are also
> interested in the center value.
> 
> This makes 20493, then you need to take out illegal
> patterns (surrounded middle stone). So I'd hint it's close
> to 2.
> 
> On 03/11/2015 18:17, Detlef Schmicker wrote: I could not
> find the number of 3x3 patterns in Go, if used all
> symmetrie s.
> 
> Can anybody give me a hint, were to find. Harvesting 4
> games I get 1093:)
> 
> Thanks, Detlef
>> ___
>> Computer-go mailing list Computer-go@computer-go.org 
>> http://computer-go.org/mailman/listinfo/computer-go
> 
> ___ Computer-go
> mailing list Computer-go@computer-go.org 
> http://computer-go.org/mailman/listinfo/computer-go
> 
>>> ___ Computer-go
>>> mailing list Computer-go@computer-go.org 
>>> http://computer-go.org/mailman/listinfo/computer-go
>>> 
>> 
>> 
>> ___ Computer-go
>> mailing list Computer-go@computer-go.org 
>> http://computer-go.org/mailman/listinfo/computer-go
>> 
> 
> 
> 
> ___ Computer-go mailing
> list Computer-go@computer-go.org 
> http://computer-go.org/mailman/listinfo/computer-go
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJWOQv1AAoJEInWdHg+Znf4GTYQAIATv45HU7fR1S4bfiygapDI
IOOnTtHTdjNoqHWGD07Y3MUy8rP24AcWHtEmlH+uwt42HBFXhCW9Hr2ul/Yreofl
e/lxcoawYYWs1tPuHEKV8TPQUVM3aHvPREoQgBMbkMlDpKQA1Jj3Q0Kv8T9cUVOW
S2URrTyOFrLiEbl4znYJwiH7hVI7q0HKom/XGFYWkfwhvJjDdKDrPbTUyl4IWo2Q
v/HdIXC/6WrPSnkeFnkc595w0qTUiXWj+B/0JYMnKvBml3aEsG8W6uT79SdDJ1MN
OJ4iW9L08p68Ovxt6Wp+eXopiPZSQ90PxPtI3cfmWrPWhs3/P95mLPg+u0CEt+PH
iuMaCM/XR68rWqQhMjRVbJkM+udo0f5iIGwN3xSDQiqfD1OO4Ks60Bdbj2qmKu/B
npEMGGeCqQmiyPftCYSdeMTHPScH+CvcL1nZaC4kdW7+aDfrC7JvU3L5nfKhVxMK
RfuXdNeX6mVAI2uL+MvFFea1B38qvdBS4y1XCQ8QObQxuxNJJupzQ8fixYGdOotj
UzuuXI4pyCzEcWWG+dr58pA35MbEpUWVsw/UMSA96RjevaqAUQ7nyFvNxcBahzE/
PGHRvcdp/c9AtlxKLDCqMd4+XMVWkSzj75jFhJuyRM4hkcszYdzVngWj6D4h9Npk
bzUgyQTYpesLVPNtVfjx
=Vqlf
-END PGP SIGNATURE-
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] Number of 3x3 patterns

[Detlef Schmicker]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I could not find the number of 3x3 patterns in Go, if used all symmetrie
s.

Can anybody give me a hint, were to find. Harvesting 4 games I get
1093:)

Thanks, Detlef
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJWOPpEAAoJEInWdHg+Znf45nEP/j4Xy8lJTesaydsiEeuAUZGZ
vfmmOEOYoX6kH2wZ6BA9GqD2NpDlZrm9+DtobTMg3EyrIkIc2kXZVkSm/rS24Jo5
p18Rub2mzeYEw4Qj3/mlr0L4NsO88wNz+vuJdDswr+UqgpEFA8Mqzf8dbC22YegO
HFGOMKPF5BHzl8clZ899N1SUSCRbuZAM/YKmtmJvi56/+MoJCO7gYOWdmUC2KYGu
J72asmmzkGgOZhYPykDF1+nGeed4KiIdtG5b9J0b0/7oV0lu4D36mjhpwbXXA8f3
xCtDO9jm6OG4kIRamCibEmYwVrJFnriltSP5RumyMGYuv7Bv7Ret3+Fgdg7IQFGp
xpH3c4cTE+GJ9d7Afr+FCMJcbEPcYSz+xPDSWhLWfTqYfQj7fh/qepTw2SLs+eD4
bjMWa3dISGnwKhzJykwXUZTqWnL/h3MvXe95Fi8bvYNEMSXCWlG4ajoSDQwwdPT2
Fktv8RJwVmtOYl5EDZhCbXi7zxvTsATyYtAxNbLcGbSyerJVrKh7cFp07GaaKyy/
tbo9tw9AEmb3xZ+qBQNI5lzQqtbWYn3sbu4eLSjIs/njtp2A7cVVW/O1SUrW+wAQ
ePkehsPjQ7yB++op8VZmN/J3f5M2UmO6TgSPDKQNUSeyCeZHzjZDJjH7qoLDr1Mw
fXMMo7Pg5lc6GOOBL885
=LKxP
-END PGP SIGNATURE-
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Number of 3x3 patterns

[Jim O'Flaherty]

Ah. That makes sense. It's a pattern centered on a possible next move. Very
cool. Tysvm for explaining.

On Tue, Nov 3, 2015 at 1:33 PM, Detlef Schmicker  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
>
>
> Am 03.11.2015 um 20:24 schrieb Jim O'Flaherty:
> > I don't see how "leave the center empty" works as a valid case,
> > assuming this it just any valid 3x3 window on the board. Given bots
> > playing each other, there can be 9x9 clumps of a stone of the same
> > color. I can see it being argued there is no computational value in
> > this specific pattern instance. But, then what are the conditions
> > of the exceptions to the generalization? And how do you effectively
> > iterate through the other +20,000 variations (not reduced by
> > location or color symmetry)?
> >
> > So, I'm curious, is there some other assumption about the 3x3
> > window other than it be a view into any valid 3x3 space on a Go
> > board?
>
> Sorry, I did not explain the details, the assumption is:
> I play in the middle, so it must be empty. I thought legal moves might
> not really reduce the number of 3x3 patterns, as there can be no
> suicide known from 3x3 patterns, as a capture is always possible.
>
> Therefore I wonder, what 14 patterns did not appear in my 4 games
> harvested:)
>
>
> >
> > On Tue, Nov 3, 2015 at 1:04 PM, Álvaro Begué
> >  wrote:
> >
> >> I get 1107 (954 in the middle + 135 on the edge + 18 on a
> >> corner).
> >>
> >> Álvaro.
> >>
> >>
> >>
> >> On Tue, Nov 3, 2015 at 2:00 PM, Detlef Schmicker 
> >> wrote:
> >>
> > Thanks, but I need them reduced by reflection and rotation
> > symmetries (and leave the center empty so 3^8 + 3^5 + 3^3 and than
> > reduce)
> >
> >
> >
> > Am 03.11.2015 um 19:32 schrieb Gonçalo Mendes Ferreira:
> > If you are considering only black stone, white, empty and
> > border, ignoring symmetry, wouldn't it be
> >
> > 3^9 + 3^6 + 3^4
> >
> > 3^9 for patterns away from the border, 3^6 for near the
> > sides and 3^4 near the corners, assuming you are also
> > interested in the center value.
> >
> > This makes 20493, then you need to take out illegal
> > patterns (surrounded middle stone). So I'd hint it's close
> > to 2.
> >
> > On 03/11/2015 18:17, Detlef Schmicker wrote: I could not
> > find the number of 3x3 patterns in Go, if used all
> > symmetrie s.
> >
> > Can anybody give me a hint, were to find. Harvesting 4
> > games I get 1093:)
> >
> > Thanks, Detlef
> >> ___
> >> Computer-go mailing list Computer-go@computer-go.org
> >> http://computer-go.org/mailman/listinfo/computer-go
> >
> > ___ Computer-go
> > mailing list Computer-go@computer-go.org
> > http://computer-go.org/mailman/listinfo/computer-go
> >
> >>> ___ Computer-go
> >>> mailing list Computer-go@computer-go.org
> >>> http://computer-go.org/mailman/listinfo/computer-go
> >>>
> >>
> >>
> >> ___ Computer-go
> >> mailing list Computer-go@computer-go.org
> >> http://computer-go.org/mailman/listinfo/computer-go
> >>
> >
> >
> >
> > ___ Computer-go mailing
> > list Computer-go@computer-go.org
> > http://computer-go.org/mailman/listinfo/computer-go
> >
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iQIcBAEBAgAGBQJWOQv1AAoJEInWdHg+Znf4GTYQAIATv45HU7fR1S4bfiygapDI
> IOOnTtHTdjNoqHWGD07Y3MUy8rP24AcWHtEmlH+uwt42HBFXhCW9Hr2ul/Yreofl
> e/lxcoawYYWs1tPuHEKV8TPQUVM3aHvPREoQgBMbkMlDpKQA1Jj3Q0Kv8T9cUVOW
> S2URrTyOFrLiEbl4znYJwiH7hVI7q0HKom/XGFYWkfwhvJjDdKDrPbTUyl4IWo2Q
> v/HdIXC/6WrPSnkeFnkc595w0qTUiXWj+B/0JYMnKvBml3aEsG8W6uT79SdDJ1MN
> OJ4iW9L08p68Ovxt6Wp+eXopiPZSQ90PxPtI3cfmWrPWhs3/P95mLPg+u0CEt+PH
> iuMaCM/XR68rWqQhMjRVbJkM+udo0f5iIGwN3xSDQiqfD1OO4Ks60Bdbj2qmKu/B
> npEMGGeCqQmiyPftCYSdeMTHPScH+CvcL1nZaC4kdW7+aDfrC7JvU3L5nfKhVxMK
> RfuXdNeX6mVAI2uL+MvFFea1B38qvdBS4y1XCQ8QObQxuxNJJupzQ8fixYGdOotj
> UzuuXI4pyCzEcWWG+dr58pA35MbEpUWVsw/UMSA96RjevaqAUQ7nyFvNxcBahzE/
> PGHRvcdp/c9AtlxKLDCqMd4+XMVWkSzj75jFhJuyRM4hkcszYdzVngWj6D4h9Npk
> bzUgyQTYpesLVPNtVfjx
> =Vqlf
> -END PGP SIGNATURE-
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Number of 3x3 patterns

[Gonçalo Mendes Ferreira]

If you are considering only black stone, white, empty and border, 
ignoring symmetry, wouldn't it be


3^9 + 3^6 + 3^4

3^9 for patterns away from the border, 3^6 for near the sides and 3^4 
near the corners, assuming you are also interested in the center value.


This makes 20493, then you need to take out illegal patterns (surrounded 
middle stone). So I'd hint it's close to 2.


On 03/11/2015 18:17, Detlef Schmicker wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I could not find the number of 3x3 patterns in Go, if used all symmetrie
s.

Can anybody give me a hint, were to find. Harvesting 4 games I get
1093:)

Thanks, Detlef
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJWOPpEAAoJEInWdHg+Znf45nEP/j4Xy8lJTesaydsiEeuAUZGZ
vfmmOEOYoX6kH2wZ6BA9GqD2NpDlZrm9+DtobTMg3EyrIkIc2kXZVkSm/rS24Jo5
p18Rub2mzeYEw4Qj3/mlr0L4NsO88wNz+vuJdDswr+UqgpEFA8Mqzf8dbC22YegO
HFGOMKPF5BHzl8clZ899N1SUSCRbuZAM/YKmtmJvi56/+MoJCO7gYOWdmUC2KYGu
J72asmmzkGgOZhYPykDF1+nGeed4KiIdtG5b9J0b0/7oV0lu4D36mjhpwbXXA8f3
xCtDO9jm6OG4kIRamCibEmYwVrJFnriltSP5RumyMGYuv7Bv7Ret3+Fgdg7IQFGp
xpH3c4cTE+GJ9d7Afr+FCMJcbEPcYSz+xPDSWhLWfTqYfQj7fh/qepTw2SLs+eD4
bjMWa3dISGnwKhzJykwXUZTqWnL/h3MvXe95Fi8bvYNEMSXCWlG4ajoSDQwwdPT2
Fktv8RJwVmtOYl5EDZhCbXi7zxvTsATyYtAxNbLcGbSyerJVrKh7cFp07GaaKyy/
tbo9tw9AEmb3xZ+qBQNI5lzQqtbWYn3sbu4eLSjIs/njtp2A7cVVW/O1SUrW+wAQ
ePkehsPjQ7yB++op8VZmN/J3f5M2UmO6TgSPDKQNUSeyCeZHzjZDJjH7qoLDr1Mw
fXMMo7Pg5lc6GOOBL885
=LKxP
-END PGP SIGNATURE-
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Facebook Go AI

[David Doshay]

This looks like GnuGo at level 1. Note things like filling at Q15, which GnuGo 
would not do on level 10 or higher.

Cheers,
David G Doshay

ddos...@mac.com





> On 3, Nov 2015, at 8:31 AM, Marc Landgraf  wrote:
> 
> then again, Gnugo donked that game pretty badly. 
> Showing one game, where Gnugo just throws away the entire top before move 50 
> is not really telling about the overall strength, imho. Gnugo repeats the 
> failure by suiciding the top right as well. 
> What is shown after is hard to evaluate, considering the score difference. 
> The fact that the FB Bot prefers to play random moves in the center instead 
> of removing the possible Ko in the lower left later is weird, but may be due 
> to it's gigantic lead at this point.
> 
> Another interesting thing to note is, that the values shown on the right do 
> not always correspond to the played moves. E.g. at move S17 (killing the top 
> right) actually S16 had been give a higher score then the played S17
> 
> 2015-11-03 17:22 GMT+01:00 Aja Huang  >:
> Yes I checked the game. The agent looks pretty strong. It crushed GnuGo 
> easily.
> 
> Aja
> 
> On Tue, Nov 3, 2015 at 4:20 PM, Rémi Coulom  > wrote:
> Can a strong player look at the video and give impressions about the game?
> 
> On 11/03/2015 03:28 PM, Petr Baudis wrote:
>Hi!
> 
>Facebook is working on a Go AI too, now:
> 
> https://www.facebook.com/Engineering/videos/10153621562717200/ 
> 
> https://code.facebook.com/posts/1478523512478471 
> 
> 
> http://www.wired.com/2015/11/facebook-is-aiming-its-ai-at-go-the-game-no-computer-can-crack/
>  
> 
> 
> The way it's presented triggers my hype alerts, but nevertheless:
> does anyone know any details about this?  Most interestingly, how
> strong is it?
> 
> 
> ___
> Computer-go mailing list
> Computer-go@computer-go.org 
> http://computer-go.org/mailman/listinfo/computer-go 
> 
> 
> ___
> Computer-go mailing list
> Computer-go@computer-go.org 
> http://computer-go.org/mailman/listinfo/computer-go 
> 
> 
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Number of 3x3 patterns

[Jim O'Flaherty]

I don't see how "leave the center empty" works as a valid case, assuming
this it just any valid 3x3 window on the board. Given bots playing each
other, there can be 9x9 clumps of a stone of the same color. I can see it
being argued there is no computational value in this specific pattern
instance. But, then what are the conditions of the exceptions to the
generalization? And how do you effectively iterate through the other
+20,000 variations (not reduced by location or color symmetry)?

So, I'm curious, is there some other assumption about the 3x3 window other
than it be a view into any valid 3x3 space on a Go board?

On Tue, Nov 3, 2015 at 1:04 PM, Álvaro Begué  wrote:

> I get 1107 (954 in the middle + 135 on the edge + 18 on a corner).
>
> Álvaro.
>
>
>
> On Tue, Nov 3, 2015 at 2:00 PM, Detlef Schmicker  wrote:
>
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>
>> Thanks, but I need them reduced by reflection and rotation symmetries
>> (and leave the center empty so 3^8 + 3^5 + 3^3 and than reduce)
>>
>>
>>
>> Am 03.11.2015 um 19:32 schrieb Gonçalo Mendes Ferreira:
>> > If you are considering only black stone, white, empty and border,
>> > ignoring symmetry, wouldn't it be
>> >
>> > 3^9 + 3^6 + 3^4
>> >
>> > 3^9 for patterns away from the border, 3^6 for near the sides and
>> > 3^4 near the corners, assuming you are also interested in the
>> > center value.
>> >
>> > This makes 20493, then you need to take out illegal patterns
>> > (surrounded middle stone). So I'd hint it's close to 2.
>> >
>> > On 03/11/2015 18:17, Detlef Schmicker wrote: I could not find the
>> > number of 3x3 patterns in Go, if used all symmetrie s.
>> >
>> > Can anybody give me a hint, were to find. Harvesting 4 games I
>> > get 1093:)
>> >
>> > Thanks, Detlef
>> >> ___ Computer-go
>> >> mailing list Computer-go@computer-go.org
>> >> http://computer-go.org/mailman/listinfo/computer-go
>> >
>> > ___ Computer-go mailing
>> > list Computer-go@computer-go.org
>> > http://computer-go.org/mailman/listinfo/computer-go
>> >
>> -BEGIN PGP SIGNATURE-
>> Version: GnuPG v2.0.22 (GNU/Linux)
>>
>> iQIcBAEBAgAGBQJWOQRAAAoJEInWdHg+Znf4SCcP/1oQI5ZI4nIMym8c/1gg1OWh
>> IDqhO3Ie1Rd7nxgDxaXZDaYACZr9UYRsueyLCLNGignLw6FVpXZvfC9YRn+NKenw
>> fruwUt/lRzfZGhRyxS5R55pq1X8ux1F/+aa63sRo4/SZGFCVCKeX/hsv3RyM6nuw
>> iuO9BhFSr0upmarcAJiyt4qSg9JWVdrQI7CaWE6C5XdJI3v5zXxV6yG7mcrSl8oy
>> 66fVbpLo++TjWRUZoChn+HSLuq1rTfp6fWPbMsV3Wwfk7Y3kGkBzMMjvjUpVRktv
>> 5Nu3tcFEf2Hts1iudQ/lPTdA9UYPhkSKwD3l4Z8khR6HpxO8kKClbU6g/uXBwLP+
>> k/ORYx4imeXMhSfgbzBxZAljRqL86cpUT9A0F0+Llqq4lClCq7I+CjZDXamAH8P8
>> AqpDQhizni0wEy0KLizkwvJ8mZnqg310553wKaZs+BfP2kkAw+iUtrqvkQ6dsk2t
>> 8fgwMT5Yuw/BfvXLHT/Bp2/GSw5Q97yMNXBBdMqvbIpWkann5bwqVmJljz1VV+TU
>> G8CA9xeIj8HjVETBUIMURZDxTs+RkxT3fIdlEhH0MvreKBEw0Xp/eSUffqINVSk4
>> pIcUdjTnutQgL/AtxePcnQuGksvB0KPK5zvUscNgXw9US3JUu1DXcJ6mn+y1PRKK
>> 9xaazKCRBfJjAkhnBlzR
>> =j9hC
>> -END PGP SIGNATURE-
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] AMAF/RAVE + heavy playouts - is it save?

[David Fotland]

Many Faces of Go doesn’t use Remi’s playout policy and I don’t think Zen does 
either.  I don’t think Remi’s and Mogo’s are similar either, since they were in 
some ways competing developments.  The bias issue is very real, so as you add 
knowledge to the playouts you have to be careful to add (for example) both 
attack and defense moves in a situation.

 

David

 

From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of 
Tobias Pfeiffer
Sent: Tuesday, November 03, 2015 12:39 PM
To: r...@ffles.com; computer-go@computer-go.org
Subject: Re: [Computer-go] AMAF/RAVE + heavy playouts - is it save?

 

This helps very much, thank you for taking the time to answer!

You might be looking for for "Combining Online and Offline Knowledge in UCT" 
[1] by Gelly and Silver. Silver Tesauroreference it in "Monte-carlo Simulation 
Balancing" [2] with "Unfortunately, a stronger simulation policy can actually 
lead to a weaker Monte-Carlo search (Gelly & Silver, 2007), a paradox that we 
explore further in this paper."

I'll make it a priority to read both papers in detail thank you! If you meant 
another paper, someone else knows one I'm happy to see more references.

Thanks!
Tobi


[1] http://www.machinelearning.org/proceedings/icml2007/papers/387.pdf
[2] http://www.machinelearning.org/archive/icml2009/papers/500.pdf



On 03.11.2015 21:03, robertfinkng...@o2.co.uk wrote:

You have to be careful what heuristics you apply. This was a surprising result: 
using a playout policy which in itself is a stronger go player can actually 
make MCTS/AMAF weaker. The reason is that MCTS depends entirely on accurate 
estimations of the value of each position in the tree. Any playout policy which 
introduces a bias therefore weakens MCTS. It may increase precision (lower 
standard deviation) but gives a less accurate assessment of the value (an 
incorrect mean). Most playouts at the moment (at least published ones) are 
based on Remi's Mogo playout policy, which increases precision without 
sacrificing accuracy.

There's a really nice diagram in one of David Silver's papers illustrating the 
effect that bias can have on playouts. As soon as you see it you understand the 
problem. Unfortunately I don't have it to hand and have unfortunately run out 
of time looking for it, otherwise I'd reference it. Hopefully somebody else can 
give the reference. I suspect David probably co-authored the paper in which 
case apologies to the other author for not crediting them here!

I hope this helps

Regards

Raffles

On 03-Nov-15 19:38, Tobias Pfeiffer wrote:

Hi everyone,
 
I haven't yet caught up on most recent go papers. If what I ask is
answered in one of these, please point there.
 
It seems everyone is using quite heavy playouts these days (nxn
patterns, atari escapes, opening libraris, lots of stuff that I don't
know yet, ...) - my question is how does that mix with AMAF/RAVE? I
remember from the early papers, that they said it'd be dangerous to do
it with non random playouts and that they shouldn't have too much logic.
 
Which, well, makes sense (to me) because the argument is that we play
random moves so they are order independent. With patterns that doesn't
hold true anymore.
 
What's the experience out there? Does it just still work? Does it not
matter because you just "warm up" the tree? Or do you need to be careful
with what heuristics you apply not too break RAVE/AMAF?
 
Thank you!
Tobi
 






___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go






-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2016.0.7163 / Virus Database: 4457/10906 - Release Date: 10/28/15







___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go





-- 
www.pragtob.info
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Number of 3x3 patterns

[David Fotland]

Many Faces of Go has 2052 3x3 patterns.  All have a empty point in the center.  
One value is used for all the illegal patterns, so there are 2051 valid 
patterns.  I use Aja’s idea of including in the pattern the Atari status of 
zero to four adjacent groups.  That’s why it’s more than Álvaro’s 1107.

 

There is no reason to iterate over all patterns.  Just iterate over the ones 
that are identical through rotation or symmetry.  One easy way to find the 
canonical pattern is to calculate hashes for all rotations and reflections and 
choose the smallest one as the pattern ID.   I use a table to map the pattern 
IDs to a set of consecutive index, 0-1251.

 

 

 

David

 

From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Jim 
O'Flaherty
Sent: Tuesday, November 03, 2015 11:35 AM
To: computer-go@computer-go.org
Subject: Re: [Computer-go] Number of 3x3 patterns

 

Ah. That makes sense. It's a pattern centered on a possible next move. Very 
cool. Tysvm for explaining.

 

On Tue, Nov 3, 2015 at 1:33 PM, Detlef Schmicker  wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Am 03.11.2015 um 20:24 schrieb Jim O'Flaherty:
> I don't see how "leave the center empty" works as a valid case,
> assuming this it just any valid 3x3 window on the board. Given bots
> playing each other, there can be 9x9 clumps of a stone of the same
> color. I can see it being argued there is no computational value in
> this specific pattern instance. But, then what are the conditions
> of the exceptions to the generalization? And how do you effectively
> iterate through the other +20,000 variations (not reduced by
> location or color symmetry)?
>
> So, I'm curious, is there some other assumption about the 3x3
> window other than it be a view into any valid 3x3 space on a Go
> board?

Sorry, I did not explain the details, the assumption is:
I play in the middle, so it must be empty. I thought legal moves might
not really reduce the number of 3x3 patterns, as there can be no
suicide known from 3x3 patterns, as a capture is always possible.

Therefore I wonder, what 14 patterns did not appear in my 4 games
harvested:)


>
> On Tue, Nov 3, 2015 at 1:04 PM, Álvaro Begué
>  wrote:
>
>> I get 1107 (954 in the middle + 135 on the edge + 18 on a
>> corner).
>>
>> Álvaro.
>>
>>
>>
>> On Tue, Nov 3, 2015 at 2:00 PM, Detlef Schmicker 
>> wrote:
>>

> Thanks, but I need them reduced by reflection and rotation
> symmetries (and leave the center empty so 3^8 + 3^5 + 3^3 and than
> reduce)
>
>
>
> Am 03.11.2015 um 19:32 schrieb Gonçalo Mendes Ferreira:
> If you are considering only black stone, white, empty and
> border, ignoring symmetry, wouldn't it be
>
> 3^9 + 3^6 + 3^4
>
> 3^9 for patterns away from the border, 3^6 for near the
> sides and 3^4 near the corners, assuming you are also
> interested in the center value.
>
> This makes 20493, then you need to take out illegal
> patterns (surrounded middle stone). So I'd hint it's close
> to 2.
>
> On 03/11/2015 18:17, Detlef Schmicker wrote: I could not
> find the number of 3x3 patterns in Go, if used all
> symmetrie s.
>
> Can anybody give me a hint, were to find. Harvesting 4
> games I get 1093:)
>
> Thanks, Detlef
>> ___
>> Computer-go mailing list Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>
> ___ Computer-go
> mailing list Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
>>> ___ Computer-go
>>> mailing list Computer-go@computer-go.org
>>> http://computer-go.org/mailman/listinfo/computer-go
>>>
>>
>>
>> ___ Computer-go
>> mailing list Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
>
>
> ___ Computer-go mailing
> list Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJWOQv1AAoJEInWdHg+Znf4GTYQAIATv45HU7fR1S4bfiygapDI
IOOnTtHTdjNoqHWGD07Y3MUy8rP24AcWHtEmlH+uwt42HBFXhCW9Hr2ul/Yreofl
e/lxcoawYYWs1tPuHEKV8TPQUVM3aHvPREoQgBMbkMlDpKQA1Jj3Q0Kv8T9cUVOW
S2URrTyOFrLiEbl4znYJwiH7hVI7q0HKom/XGFYWkfwhvJjDdKDrPbTUyl4IWo2Q
v/HdIXC/6WrPSnkeFnkc595w0qTUiXWj+B/0JYMnKvBml3aEsG8W6uT79SdDJ1MN
OJ4iW9L08p68Ovxt6Wp+eXopiPZSQ90PxPtI3cfmWrPWhs3/P95mLPg+u0CEt+PH
iuMaCM/XR68rWqQhMjRVbJkM+udo0f5iIGwN3xSDQiqfD1OO4Ks60Bdbj2qmKu/B
npEMGGeCqQmiyPftCYSdeMTHPScH+CvcL1nZaC4kdW7+aDfrC7JvU3L5nfKhVxMK
RfuXdNeX6mVAI2uL+MvFFea1B38qvdBS4y1XCQ8QObQxuxNJJupzQ8fixYGdOotj
UzuuXI4pyCzEcWWG+dr58pA35MbEpUWVsw/UMSA96RjevaqAUQ7nyFvNxcBahzE/

Re: [Computer-go] AMAF/RAVE + heavy playouts - is it save?

[robertfinkng...@o2.co.uk]

  
  
You have to be careful what heuristics you apply. This was a
  surprising result: using a playout policy which in itself is a
  stronger go player can actually make MCTS/AMAF weaker. The reason
  is that MCTS depends entirely on accurate estimations of the value
  of each position in the tree. Any playout policy which introduces
  a bias therefore weakens MCTS. It may increase precision (lower
  standard deviation) but gives a less accurate assessment of the
  value (an incorrect mean). Most playouts at the moment (at least
  published ones) are based on Remi's Mogo playout policy, which
  increases precision without sacrificing accuracy.
  
  There's a really nice diagram in one of David Silver's papers
  illustrating the effect that bias can have on playouts. As soon as
  you see it you understand the problem. Unfortunately I don't have
  it to hand and have unfortunately run out of time looking for it,
  otherwise I'd reference it. Hopefully somebody else can give the
  reference. I suspect David probably co-authored the paper in which
  case apologies to the other author for not crediting them here!
  
  I hope this helps
  
  Regards
  
  Raffles

On 03-Nov-15 19:38, Tobias Pfeiffer
  wrote:


  Hi everyone,

I haven't yet caught up on most recent go papers. If what I ask is
answered in one of these, please point there.

It seems everyone is using quite heavy playouts these days (nxn
patterns, atari escapes, opening libraris, lots of stuff that I don't
know yet, ...) - my question is how does that mix with AMAF/RAVE? I
remember from the early papers, that they said it'd be dangerous to do
it with non random playouts and that they shouldn't have too much logic.

Which, well, makes sense (to me) because the argument is that we play
random moves so they are order independent. With patterns that doesn't
hold true anymore.

What's the experience out there? Does it just still work? Does it not
matter because you just "warm up" the tree? Or do you need to be careful
with what heuristics you apply not too break RAVE/AMAF?

Thank you!
Tobi


  
  
  
  ___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
  
  
  
  -
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2016.0.7163 / Virus Database: 4457/10906 - Release Date: 10/28/15



  

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] AMAF/RAVE + heavy playouts - is it save?

[Tobias Pfeiffer]

This helps very much, thank you for taking the time to answer!

You might be looking for for "Combining Online and Offline Knowledge in
UCT" [1] by Gelly and Silver. Silver Tesauroreference it in "Monte-carlo
Simulation Balancing" [2] with "Unfortunately, a stronger simulation
policy can actually lead to a weaker Monte-Carlo search (Gelly & Silver,
2007), a paradox that we explore further in this paper."

I'll make it a priority to read both papers in detail thank you! If you
meant another paper, someone else knows one I'm happy to see more
references.

Thanks!
Tobi


[1] http://www.machinelearning.org/proceedings/icml2007/papers/387.pdf
[2] http://www.machinelearning.org/archive/icml2009/papers/500.pdf


On 03.11.2015 21:03, robertfinkng...@o2.co.uk wrote:
> You have to be careful what heuristics you apply. This was a
> surprising result: using a playout policy which in itself is a
> stronger go player can actually make MCTS/AMAF weaker. The reason is
> that MCTS depends entirely on accurate estimations of the value of
> each position in the tree. Any playout policy which introduces a bias
> therefore weakens MCTS. It may increase precision (lower standard
> deviation) but gives a less accurate assessment of the value (an
> incorrect mean). Most playouts at the moment (at least published ones)
> are based on Remi's Mogo playout policy, which increases precision
> without sacrificing accuracy.
>
> There's a really nice diagram in one of David Silver's papers
> illustrating the effect that bias can have on playouts. As soon as you
> see it you understand the problem. Unfortunately I don't have it to
> hand and have unfortunately run out of time looking for it, otherwise
> I'd reference it. Hopefully somebody else can give the reference. I
> suspect David probably co-authored the paper in which case apologies
> to the other author for not crediting them here!
>
> I hope this helps
>
> Regards
>
> Raffles
>
> On 03-Nov-15 19:38, Tobias Pfeiffer wrote:
>> Hi everyone,
>>
>> I haven't yet caught up on most recent go papers. If what I ask is
>> answered in one of these, please point there.
>>
>> It seems everyone is using quite heavy playouts these days (nxn
>> patterns, atari escapes, opening libraris, lots of stuff that I don't
>> know yet, ...) - my question is how does that mix with AMAF/RAVE? I
>> remember from the early papers, that they said it'd be dangerous to do
>> it with non random playouts and that they shouldn't have too much logic.
>>
>> Which, well, makes sense (to me) because the argument is that we play
>> random moves so they are order independent. With patterns that doesn't
>> hold true anymore.
>>
>> What's the experience out there? Does it just still work? Does it not
>> matter because you just "warm up" the tree? Or do you need to be careful
>> with what heuristics you apply not too break RAVE/AMAF?
>>
>> Thank you!
>> Tobi
>>
>>
>>
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>>
>> -
>> No virus found in this message.
>> Checked by AVG - www.avg.com
>> Version: 2016.0.7163 / Virus Database: 4457/10906 - Release Date: 10/28/15
>
>
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go

-- 
www.pragtob.info

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Number of 3x3 patterns

[Hiroshi Yamashita]

Hi Detlef,

I have same number 1093 for 3x3 from 4 pro games as symmetory
is same pattern. Each point has 4 types, empty, black, white and
out of board.

3x3 pattern is small. So recently I combined it with distance of
previous move. Each 3x3 pattern has 17 kinds of distance of previous
move. And by using Latent Factor Ranking, I got 40.2% accuracy from
38.8% by MM.

Latent Factor Ranking(LFR) is almost same as Factorization Machines.
And original paper's explanation is nice. I use LFR_K=5, and AdaGrad
to change learning rate. It is effective (Thanks to Kato-san).

My LFR test code.
http://yss-aya.com/lfr_20151104.tar.gz
Factorization Machines
http://www.ismll.uni-hildesheim.de/pub/pdfs/Rendle2010_Factorization_Machines.pdf
Move Prediction in Go, Modelling Feature Interactions Using Latent Factors
http://www.ismll.uni-hildesheim.de/pub/pdfs/wistuba_et_al_KI_2013.pdf

Regards,
Hiroshi Yamashita

- Original Message - 
From: "Detlef Schmicker" 

To: 
Sent: Wednesday, November 04, 2015 3:17 AM
Subject: [Computer-go] Number of 3x3 patterns


I could not find the number of 3x3 patterns in Go, if used all symmetries.

Can anybody give me a hint, were to find. Harvesting 4 games I get
1093:)

Thanks, Detlef


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] some issues with the beijing tournament go client

[David Fotland]

I've been helping them test their client and there are some issues.  They
are working on fixing them.  If you are having problems while testing,
please email me directly at fotl...@smart-games.com for details, and I can
save you some time or provide worarounds.  I don't have email addresses for
most of the participants or I would contact them directly.

David


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Facebook Go AI

[Rémi Coulom]

Can a strong player look at the video and give impressions about the game?

On 11/03/2015 03:28 PM, Petr Baudis wrote:

   Hi!

   Facebook is working on a Go AI too, now:

https://www.facebook.com/Engineering/videos/10153621562717200/
https://code.facebook.com/posts/1478523512478471

http://www.wired.com/2015/11/facebook-is-aiming-its-ai-at-go-the-game-no-computer-can-crack/

The way it's presented triggers my hype alerts, but nevertheless:
does anyone know any details about this?  Most interestingly, how
strong is it?



___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Facebook Go AI

[Marc Landgraf]

then again, Gnugo donked that game pretty badly.
Showing one game, where Gnugo just throws away the entire top before move
50 is not really telling about the overall strength, imho. Gnugo repeats
the failure by suiciding the top right as well.
What is shown after is hard to evaluate, considering the score difference.
The fact that the FB Bot prefers to play random moves in the center instead
of removing the possible Ko in the lower left later is weird, but may be
due to it's gigantic lead at this point.

Another interesting thing to note is, that the values shown on the right do
not always correspond to the played moves. E.g. at move S17 (killing the
top right) actually S16 had been give a higher score then the played S17

2015-11-03 17:22 GMT+01:00 Aja Huang :

> Yes I checked the game. The agent looks pretty strong. It crushed GnuGo
> easily.
>
> Aja
>
> On Tue, Nov 3, 2015 at 4:20 PM, Rémi Coulom  wrote:
>
>> Can a strong player look at the video and give impressions about the game?
>>
>> On 11/03/2015 03:28 PM, Petr Baudis wrote:
>>
>>>Hi!
>>>
>>>Facebook is working on a Go AI too, now:
>>>
>>> https://www.facebook.com/Engineering/videos/10153621562717200/
>>> https://code.facebook.com/posts/1478523512478471
>>>
>>> http://www.wired.com/2015/11/facebook-is-aiming-its-ai-at-go-the-game-no-computer-can-crack/
>>>
>>> The way it's presented triggers my hype alerts, but nevertheless:
>>> does anyone know any details about this?  Most interestingly, how
>>> strong is it?
>>>
>>>
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Understanding statistics for benchmarking

[Urban Hafner]

Yes, I noticed that too. But luckily that's the one thing I didn't even 
consider doing. Running the same number of games feels like the most natural 
thing to do anyway. 

Von meinem iPhone gesendet

> Am 03.11.2015 um 14:22 schrieb Petr Baudis :
> 
>> On Tue, Nov 03, 2015 at 09:46:00AM +0100, Rémi Coulom wrote:
>> The intervals given by gogui are the standard deviation, not the usual 95%
>> confidence intervals.
>> 
>> For 95% confidence intervals, you have to multiply the standard deviation by
>> two.
>> 
>> And you still have the 5% chance of not being inside the interval, so you
>> can still get the occasional non-overlapping intervals.
>> 
>> Likelihood of superiority is an interesting statistical tool:
>> https://chessprogramming.wikispaces.com/LOS+Table
>> 
>> For more advanced tools for deciding when to stop testing, there is SPRT:
>> http://www.open-chess.org/viewtopic.php?f=5=2477
>> https://en.wikipedia.org/wiki/Sequential_probability_ratio_test
> 
> An important corollary to this (noted on this list every few years)
> is that in the most naive scenario where your statistical test is just
> SD-based overlap after N games, you should fix your N number of games
> in advance and not rig it by terminating out of schedule.  If you look
> at the progress of your playtesting often, you could spot a few moments
> where the intervals do not overlap, enve if in the long run they
> typically would.
> 
> (The situation is a bit dire if you have limited computing resources.
> I admit that sometimes I didn't follow the above myself in less formal
> exploratory experiments, but at least I tried to look only
> "infrequently", e.g. single check every few hours, only at "round"
> numbers of playouts, etc.  I hope it's not a grave sin.)
> 
> -- 
>Petr Baudis
>If you have good ideas, good data and fast computers,
>you can do almost anything. -- Geoffrey Hinton
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] Facebook Go AI

[Petr Baudis]

  Hi!

  Facebook is working on a Go AI too, now:

https://www.facebook.com/Engineering/videos/10153621562717200/
https://code.facebook.com/posts/1478523512478471

http://www.wired.com/2015/11/facebook-is-aiming-its-ai-at-go-the-game-no-computer-can-crack/

The way it's presented triggers my hype alerts, but nevertheless:
does anyone know any details about this?  Most interestingly, how
strong is it?

-- 
Petr Baudis
If you have good ideas, good data and fast computers,
you can do almost anything. -- Geoffrey Hinton
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go