Re: [Computer-go] Congratulations to Zen!

[Erik van der Werf]

Oh that's silly! IIRC if your bot is not ranked than users can do all kind
of cheating in the scoring phase (e.g., mark all your living stones dead).

On Tue, May 10, 2016 at 12:07 AM, Gian-Carlo Pascutto  wrote:

> On 10/05/2016 0:01, Erik van der Werf wrote:
> > Well then why not make that a criterion for entering the tournament? For
> > any half-decent bot it shouldn't be hard to get a rating.
>
> FWIW I requested ranked status for LeelaBot 3 weeks ago and this was not
> granted.
>
> Technically I'm not sure if this is needed as the rating can just be
> calculated from the unranked games. I've been tempted to write a script
> to do exactly this.
>
> --
> GCP
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Gian-Carlo Pascutto]

On 10/05/2016 0:01, Erik van der Werf wrote:
> Well then why not make that a criterion for entering the tournament? For
> any half-decent bot it shouldn't be hard to get a rating.

FWIW I requested ranked status for LeelaBot 3 weeks ago and this was not
granted.

Technically I'm not sure if this is needed as the rating can just be
calculated from the unranked games. I've been tempted to write a script
to do exactly this.

-- 
GCP
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Erik van der Werf]

Well then why not make that a criterion for entering the tournament? For
any half-decent bot it shouldn't be hard to get a rating.

Any idea what happens for unrated bots? Do they end up somewhere at the
bottom, or are they rejected?

Erik

On Mon, May 9, 2016 at 11:40 PM, Nick Wedd  wrote:

> A problem with McMahon is that the bots would all need KGS ratings.  I
> can't assign ratings myself, the scheduler uses the ratings assigned by
> KGS.  In the five tournaments held so far this year, there are sixteen bots
> that have competed at least once: eight have been rated, eight not.
>
> Nick
>
> On 9 May 2016 at 22:16, Erik van der Werf 
> wrote:
>
>> Why not McMahon? (possibly with reduced handicap).  It works fine in
>> human Go tournaments.
>>
>> IMO KGS Swiss is pretty boring for most of the time, and the scheduler
>> often seems to have a lot of undesired influence on the final ranking. Also
>> at this point I'm really not that interested any more to see some top
>> engine win yet another bot tournament without serious competition; I'd be
>> more interested to see how many stones they could give to the rest.
>> Wouldn't it be fun to see how many stones AlphaGo could give to CS?
>>
>> E.
>>
>>
>> On Mon, May 9, 2016 at 10:29 PM, "Ingo Althöfer" <3-hirn-ver...@gmx.de>
>> wrote:
>>
>>> Hi Gian-Carlo,
>>>
>>> I have thought carefully about your question on
>>> determinning handicaps properly.
>>> It seems you are very right with your doubts
>>>
>>> > The first obvious question is then: how will you determine the
>>> handicaps?
>>>
>>> A naive approach would be to take the KGS ranks of the bots.
>>> But even for those who really have this may be a problem. Namely,
>>> the program may use other/stronger hardware in the tournament,
>>> or may have made a jump in performance without playing openly
>>> on KGS.
>>>
>>> > As to the "large gaps in strength": the actual rating of Zen is
>>> > 1 stone above abakus, which is 1 stone above HiraBot. That seems
>>> > to conflict with your classification.
>>>
>>> Yes, but only according to KGS ranks. My impression yesterday was
>>> that Zen has made another jump in performance and is now more
>>> an 8-dan than a 7-dan. But this is indeed only a personal opinion
>>> and can not be taken for "serious" handicapping.
>>>
>>> Concerning abakus and Hirabot, it is indeed my opinion that they
>>> are at most 1 stone apart of each other.
>>>
>>> In total: my handicap idea seems not to be practicable.
>>>
>>> Ingo.
>>> ___
>>> Computer-go mailing list
>>> Computer-go@computer-go.org
>>> http://computer-go.org/mailman/listinfo/computer-go
>>>
>>
>>
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
>
>
> --
> Nick Wedd  mapr...@gmail.com
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Gian-Carlo Pascutto]

On 9/05/2016 23:16, Erik van der Werf wrote:
> Why not McMahon? (possibly with reduced handicap).  It works fine in
> human Go tournaments.

http://senseis.xmp.net/?McMahon

How does this work? That page doesn't mention handicaps. Indeed, the
idea seems to be to eliminate large strength disparities in the early
rounds of Swiss.

> IMO KGS Swiss is pretty boring for most of the time, and the scheduler
> often seems to have a lot of undesired influence on the final ranking.
> Also at this point I'm really not that interested any more to see some
> top engine win yet another bot tournament without serious competition;
> I'd be more interested to see how many stones they could give to the
> rest. Wouldn't it be fun to see how many stones AlphaGo could give to CS?

I'm not sure if this is a comment on the tournament format or wishful
thinking about the number and level of the tournament participants :-)

-- 
GCP
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Nick Wedd]

A problem with McMahon is that the bots would all need KGS ratings.  I
can't assign ratings myself, the scheduler uses the ratings assigned by
KGS.  In the five tournaments held so far this year, there are sixteen bots
that have competed at least once: eight have been rated, eight not.

Nick

On 9 May 2016 at 22:16, Erik van der Werf  wrote:

> Why not McMahon? (possibly with reduced handicap).  It works fine in human
> Go tournaments.
>
> IMO KGS Swiss is pretty boring for most of the time, and the scheduler
> often seems to have a lot of undesired influence on the final ranking. Also
> at this point I'm really not that interested any more to see some top
> engine win yet another bot tournament without serious competition; I'd be
> more interested to see how many stones they could give to the rest.
> Wouldn't it be fun to see how many stones AlphaGo could give to CS?
>
> E.
>
>
> On Mon, May 9, 2016 at 10:29 PM, "Ingo Althöfer" <3-hirn-ver...@gmx.de>
> wrote:
>
>> Hi Gian-Carlo,
>>
>> I have thought carefully about your question on
>> determinning handicaps properly.
>> It seems you are very right with your doubts
>>
>> > The first obvious question is then: how will you determine the
>> handicaps?
>>
>> A naive approach would be to take the KGS ranks of the bots.
>> But even for those who really have this may be a problem. Namely,
>> the program may use other/stronger hardware in the tournament,
>> or may have made a jump in performance without playing openly
>> on KGS.
>>
>> > As to the "large gaps in strength": the actual rating of Zen is
>> > 1 stone above abakus, which is 1 stone above HiraBot. That seems
>> > to conflict with your classification.
>>
>> Yes, but only according to KGS ranks. My impression yesterday was
>> that Zen has made another jump in performance and is now more
>> an 8-dan than a 7-dan. But this is indeed only a personal opinion
>> and can not be taken for "serious" handicapping.
>>
>> Concerning abakus and Hirabot, it is indeed my opinion that they
>> are at most 1 stone apart of each other.
>>
>> In total: my handicap idea seems not to be practicable.
>>
>> Ingo.
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>



-- 
Nick Wedd  mapr...@gmail.com
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Petr Baudis]

  Another matter is that, in case of MCTS programs, encouraging them
to play well in handicap games was a troublesome point.  This cuts both
ways (may discourage participation, or encourage implementation of
better handling of handicap games).

  A possible strategy would be:

  - Require ranked programs (with at least 30 rated games played in last
30 days using the tournament version)

  - Use McMahon scheduling

  - Use handicaps reduced by 2, topped at 4 stones.  (More liberally,
topped at 6 stones.)

  - Apply these rules only every other month to test it out; this should
be out of phase with the other variations?

  This might be a lot more fun to watch. :)  Handicap reduction is
important to still give the stronger programs a good edge.

On Mon, May 09, 2016 at 11:16:58PM +0200, Erik van der Werf wrote:
> Why not McMahon? (possibly with reduced handicap).  It works fine in human
> Go tournaments.
> 
> IMO KGS Swiss is pretty boring for most of the time, and the scheduler
> often seems to have a lot of undesired influence on the final ranking. Also
> at this point I'm really not that interested any more to see some top
> engine win yet another bot tournament without serious competition; I'd be
> more interested to see how many stones they could give to the rest.
> Wouldn't it be fun to see how many stones AlphaGo could give to CS?
> 
> E.
> 
> 
> On Mon, May 9, 2016 at 10:29 PM, "Ingo Althöfer" <3-hirn-ver...@gmx.de>
> wrote:
> 
> > Hi Gian-Carlo,
> >
> > I have thought carefully about your question on
> > determinning handicaps properly.
> > It seems you are very right with your doubts
> >
> > > The first obvious question is then: how will you determine the handicaps?
> >
> > A naive approach would be to take the KGS ranks of the bots.
> > But even for those who really have this may be a problem. Namely,
> > the program may use other/stronger hardware in the tournament,
> > or may have made a jump in performance without playing openly
> > on KGS.
> >
> > > As to the "large gaps in strength": the actual rating of Zen is
> > > 1 stone above abakus, which is 1 stone above HiraBot. That seems
> > > to conflict with your classification.
> >
> > Yes, but only according to KGS ranks. My impression yesterday was
> > that Zen has made another jump in performance and is now more
> > an 8-dan than a 7-dan. But this is indeed only a personal opinion
> > and can not be taken for "serious" handicapping.
> >
> > Concerning abakus and Hirabot, it is indeed my opinion that they
> > are at most 1 stone apart of each other.
> >
> > In total: my handicap idea seems not to be practicable.
> >
> > Ingo.
> > ___
> > Computer-go mailing list
> > Computer-go@computer-go.org
> > http://computer-go.org/mailman/listinfo/computer-go
> >

> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go


-- 
Petr Baudis
If you have good ideas, good data and fast computers,
you can do almost anything. -- Geoffrey Hinton
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Erik van der Werf]

Why not McMahon? (possibly with reduced handicap).  It works fine in human
Go tournaments.

IMO KGS Swiss is pretty boring for most of the time, and the scheduler
often seems to have a lot of undesired influence on the final ranking. Also
at this point I'm really not that interested any more to see some top
engine win yet another bot tournament without serious competition; I'd be
more interested to see how many stones they could give to the rest.
Wouldn't it be fun to see how many stones AlphaGo could give to CS?

E.


On Mon, May 9, 2016 at 10:29 PM, "Ingo Althöfer" <3-hirn-ver...@gmx.de>
wrote:

> Hi Gian-Carlo,
>
> I have thought carefully about your question on
> determinning handicaps properly.
> It seems you are very right with your doubts
>
> > The first obvious question is then: how will you determine the handicaps?
>
> A naive approach would be to take the KGS ranks of the bots.
> But even for those who really have this may be a problem. Namely,
> the program may use other/stronger hardware in the tournament,
> or may have made a jump in performance without playing openly
> on KGS.
>
> > As to the "large gaps in strength": the actual rating of Zen is
> > 1 stone above abakus, which is 1 stone above HiraBot. That seems
> > to conflict with your classification.
>
> Yes, but only according to KGS ranks. My impression yesterday was
> that Zen has made another jump in performance and is now more
> an 8-dan than a 7-dan. But this is indeed only a personal opinion
> and can not be taken for "serious" handicapping.
>
> Concerning abakus and Hirabot, it is indeed my opinion that they
> are at most 1 stone apart of each other.
>
> In total: my handicap idea seems not to be practicable.
>
> Ingo.
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Ingo Althöfer]

Hi Gian-Carlo,

I have thought carefully about your question on
determinning handicaps properly.
It seems you are very right with your doubts

> The first obvious question is then: how will you determine the handicaps?

A naive approach would be to take the KGS ranks of the bots.
But even for those who really have this may be a problem. Namely,
the program may use other/stronger hardware in the tournament,
or may have made a jump in performance without playing openly 
on KGS. 
 
> As to the "large gaps in strength": the actual rating of Zen is 
> 1 stone above abakus, which is 1 stone above HiraBot. That seems 
> to conflict with your classification.

Yes, but only according to KGS ranks. My impression yesterday was
that Zen has made another jump in performance and is now more
an 8-dan than a 7-dan. But this is indeed only a personal opinion
and can not be taken for "serious" handicapping.

Concerning abakus and Hirabot, it is indeed my opinion that they
are at most 1 stone apart of each other.

In total: my handicap idea seems not to be practicable.

Ingo.
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Jason House]

I'm not sure what options are available for tournament setup, but assuming
we can enter skill levels manually...

I think it would be hard to pick perfect ratings, but I bet it wouldn't be
too difficult to generate a guess at ELO/kyu levels based on past
performance. Inputting something like ELO/2 (or similar fractional ELO)
might be a good way to start out without giving too much handicap. Future
tournaments could then be based on how well that works out.
On May 9, 2016 1:18 PM, "Gian-Carlo Pascutto"  wrote:

On 09-05-16 16:04, "Ingo Althöfer" wrote:
> Another point for discussion:
> Although there were only six participants they split in
> at least 4 classes, seperated by large gaps in strength:
> Zen >> abakus, HiraBot >> LeelaBot >> Imrsel, matilda
> Perhaps it makes really sense to think about a tournament
> with handicaps.

The first obvious question is then: how will you determine the handicaps?

As to the "large gaps in strength": the actual rating of Zen is 1 stone
above abakus, which is 1 stone above HiraBot. That seems to conflict
with your classification. I am not sure it's possible to draw these
conclusions by observing <= 3 games from every matchup.

--
GCP
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Jason House]

On May 9, 2016 10:38 AM, "Urban Hafner"  wrote:
>
>Also, you give me too much credit. I’m not the primary author of HouseBot,
that is Jason House. I was merely a co-author/contributor.
>
> Urban

I didn't even notice that in the report! I'm not too worried about credit
for my weak bot :)
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Gian-Carlo Pascutto]

On 09-05-16 16:04, "Ingo Althöfer" wrote:
> Another point for discussion:
> Although there were only six participants they split in
> at least 4 classes, seperated by large gaps in strength:
> Zen >> abakus, HiraBot >> LeelaBot >> Imrsel, matilda
> Perhaps it makes really sense to think about a tournament
> with handicaps.  

The first obvious question is then: how will you determine the handicaps?

As to the "large gaps in strength": the actual rating of Zen is 1 stone
above abakus, which is 1 stone above HiraBot. That seems to conflict
with your classification. I am not sure it's possible to draw these
conclusions by observing <= 3 games from every matchup.

-- 
GCP
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Urban Hafner]

Thanks for organising this Nick!

Even though Iomrascálaí came second to last it was very useful as I could
figure out how to make it play on KGS. :) And like Ingo already said, it
played against abakus in the second round. Also, you give me too much
credit. I’m not the primary author of HouseBot, that is Jason House. I was
merely a co-author/contributor.

Urban

On Mon, May 9, 2016 at 4:04 PM, "Ingo Althöfer" <3-hirn-ver...@gmx.de>
wrote:

> Hi Nick,
>
> thanks for organizing the tournament and also for the
> report. I found one small inconsistency:
> You write that Imrsel had connection problems in
> round 2 against matilda. But in round 2, imrsel had
> been paired with abakus, according to the table.
>
> 
> Another point for discussion:
> Although there were only six participants they split in
> at least 4 classes, seperated by large gaps in strength:
> Zen >> abakus, HiraBot >> LeelaBot >> Imrsel, matilda
> Perhaps it makes really sense to think about a tournament
> with handicaps.
>
> Ingo.
>
> PS. Nice to see Leela back after such a long break.
>
>
>
>
> Gesendet: Montag, 09. Mai 2016 um 15:33 Uhr
> Von: "Nick Wedd" 
> An: computer-go@computer-go.org
> Betreff: [Computer-go] Congratulations to Zen!
>
> Congratulations to Xen19X, winner of yesterday's KGS bot tournament, with
> 12 wins from 12 games!
>
> My report is at http://www.weddslist.com/kgs/past/122/index.html
> As always, I will welcome your comments and corrections.
>
> Nick--
> Nick Wedd  
> mapr...@gmail.com[mapr...@gmail.com]___
> Computer-go mailing list Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go[http://computer-go.org/mailman/listinfo/computer-go]
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go




-- 
Blog: http://bettong.net/
Twitter: https://twitter.com/ujh
Homepage: http://www.urbanhafner.com/
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Congratulations to Zen!

[Ingo Althöfer]

Hi Nick,

thanks for organizing the tournament and also for the 
report. I found one small inconsistency:
You write that Imrsel had connection problems in
round 2 against matilda. But in round 2, imrsel had
been paired with abakus, according to the table.
 

Another point for discussion:
Although there were only six participants they split in
at least 4 classes, seperated by large gaps in strength:
Zen >> abakus, HiraBot >> LeelaBot >> Imrsel, matilda
Perhaps it makes really sense to think about a tournament
with handicaps.  
 
Ingo.

PS. Nice to see Leela back after such a long break.




Gesendet: Montag, 09. Mai 2016 um 15:33 Uhr
Von: "Nick Wedd" 
An: computer-go@computer-go.org
Betreff: [Computer-go] Congratulations to Zen!

Congratulations to Xen19X, winner of yesterday's KGS bot tournament, with 12 
wins from 12 games!
 
My report is at http://www.weddslist.com/kgs/past/122/index.html
As always, I will welcome your comments and corrections.
 
Nick--
Nick Wedd      
mapr...@gmail.com[mapr...@gmail.com]___
 Computer-go mailing list Computer-go@computer-go.org 
http://computer-go.org/mailman/listinfo/computer-go[http://computer-go.org/mailman/listinfo/computer-go]
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] Congratulations to Zen!

[Nick Wedd]

Congratulations to Xen19X, winner of yesterday's KGS bot tournament, with
12 wins from 12 games!

My report is at http://www.weddslist.com/kgs/past/122/index.html
As always, I will welcome your comments and corrections.

Nick
-- 
Nick Wedd  mapr...@gmail.com
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go