Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Gonçalo Mendes Ferreira]

On 10/10/2015 18:30, David Doshay wrote:

I agree completely that there is no way to enforce computational limits over 
the internet.

I am against ‘identical hardware’ tournaments because people have worked to get 
their programs working on the hardware they have, and some people will be on 
the other side of any hardware decision, Mac v.s. PC being the most obvious.

There is no "Mac hardware".


I am left wondering what the point is for such a tournament. Is it to show who 
is the most efficient programmer? Is it to show how these programs might run on 
somebody’s home computer? These things are not important for research code that 
is not intended for resale.
I'm also against identical hardware restrictions, but divisions can be 
very flexible. Not everyone cares for research and you wouldn't be using 
open tournaments for research results either way.


Gonçalo F.
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Jim O'Flaherty]

I second Peter's response.
On Oct 10, 2015 10:33 AM, "Peter Drake"  wrote:

> I'm also for no limits, if only because there's no way to enforce them.
>
> If there is to be a limited division, I'd like to see all programs run on
> identical hardware.
>
> On Fri, Oct 9, 2015 at 6:07 AM, Hiroshi Yamashita 
> wrote:
>
>> Hi Nick,
>>
>> I'd like no limit. Restriction will lose a chance of massive
>> computer's programming. But one thread limit tournament
>> once a year may be interesting.
>>
>> I like (2), and (3) is nice, but I'm already happy with your reports!
>>
>> Regards,
>> Hiroshi Yamashita
>>
>>
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
>
>
> --
> Peter Drake
> https://sites.google.com/a/lclark.edu/drake/
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Nick Wedd]

Thanks to everyone for their interest and responses.  The second and third
questions are easy:  I shall keep the zeroes in the "annual" table, and I
shall update the crosstable after each round whenever this is convenient
for me.  I really don't feel qualified to contribute to question 1, the
"limited hardware" issue.  I like the idea of making the Slow tournaments
limited to one core, one processor, one thread, or whatever, so as to
reduce power consumed and contribute less to global warming, but I don't
know if there is a practicable way of doing this.

I hope you will continue to discuss question 1.

Nick

On 10 October 2015 at 17:06, Jim O'Flaherty 
wrote:

> I second Peter's response.
> On Oct 10, 2015 10:33 AM, "Peter Drake"  wrote:
>
>> I'm also for no limits, if only because there's no way to enforce them.
>>
>> If there is to be a limited division, I'd like to see all programs run on
>> identical hardware.
>>
>> On Fri, Oct 9, 2015 at 6:07 AM, Hiroshi Yamashita 
>> wrote:
>>
>>> Hi Nick,
>>>
>>> I'd like no limit. Restriction will lose a chance of massive
>>> computer's programming. But one thread limit tournament
>>> once a year may be interesting.
>>>
>>> I like (2), and (3) is nice, but I'm already happy with your reports!
>>>
>>> Regards,
>>> Hiroshi Yamashita
>>>
>>>
>>> ___
>>> Computer-go mailing list
>>> Computer-go@computer-go.org
>>> http://computer-go.org/mailman/listinfo/computer-go
>>>
>>
>>
>>
>> --
>> Peter Drake
>> https://sites.google.com/a/lclark.edu/drake/
>>
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>



-- 
Nick Wedd  mapr...@gmail.com
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[David Fotland]

There is an easy way to enforce computational limits.  Ask everyone to run on 
an identical AWS instance.  Nevertheless, I’m against identical hardware 
tournaments except as a special rare exception.

 

From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of 
David Doshay
Sent: Saturday, October 10, 2015 10:31 AM
To: computer-go@computer-go.org
Subject: *SPAM* Re: [Computer-go] KGS bot tournaments - what are your 
opinions?

 

I agree completely that there is no way to enforce computational limits over 
the internet.

 

I am against ‘identical hardware’ tournaments because people have worked to get 
their programs working on the hardware they have, and some people will be on 
the other side of any hardware decision, Mac v.s. PC being the most obvious.

 

I am left wondering what the point is for such a tournament. Is it to show who 
is the most efficient programmer? Is it to show how these programs might run on 
somebody’s home computer? These things are not important for research code that 
is not intended for resale.


Cheers,
David G Doshay


ddos...@mac.com

 





 

On 10, Oct 2015, at 8:33 AM, Peter Drake <dr...@lclark.edu> wrote:

 

I'm also for no limits, if only because there's no way to enforce them.

 

If there is to be a limited division, I'd like to see all programs run on 
identical hardware.

 

On Fri, Oct 9, 2015 at 6:07 AM, Hiroshi Yamashita <y...@bd.mbn.or.jp> wrote:

Hi Nick,

I'd like no limit. Restriction will lose a chance of massive
computer's programming. But one thread limit tournament
once a year may be interesting.

I like (2), and (3) is nice, but I'm already happy with your reports!

Regards,
Hiroshi Yamashita



___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go





 

-- 

Peter Drake
https://sites.google.com/a/lclark.edu/drake/

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

 

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Marc Landgraf]

I still like the idea of "1 Desktop/Notebook" for the lowspec category.
And what is the point? Comparability. How are you comparing your "research
results" if it is not clear, if the advantage comes from an hardware
advantage or from your newly developed algorithms? If tried to improve the
aerodynamic of a Ferrari, you are also not proving that by racing against
some Skodas.
Of course, improving parallelization etc is an important part of Computer
Go Development, but sometimes it is nice to eliminate this factor to see
how the rest of the coders work is doing.


2015-10-10 19:30 GMT+02:00 David Doshay :

> I agree completely that there is no way to enforce computational limits
> over the internet.
>
> I am against ‘identical hardware’ tournaments because people have worked
> to get their programs working on the hardware they have, and some people
> will be on the other side of any hardware decision, Mac v.s. PC being the
> most obvious.
>
> I am left wondering what the point is for such a tournament. Is it to show
> who is the most efficient programmer? Is it to show how these programs
> might run on somebody’s home computer? These things are not important for
> research code that is not intended for resale.
>
> Cheers,
> David G Doshay
>
> ddos...@mac.com
>
>
>
>
>
> On 10, Oct 2015, at 8:33 AM, Peter Drake  wrote:
>
> I'm also for no limits, if only because there's no way to enforce them.
>
> If there is to be a limited division, I'd like to see all programs run on
> identical hardware.
>
> On Fri, Oct 9, 2015 at 6:07 AM, Hiroshi Yamashita 
> wrote:
>
>> Hi Nick,
>>
>> I'd like no limit. Restriction will lose a chance of massive
>> computer's programming. But one thread limit tournament
>> once a year may be interesting.
>>
>> I like (2), and (3) is nice, but I'm already happy with your reports!
>>
>> Regards,
>> Hiroshi Yamashita
>>
>>
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
>
>
>
> --
> Peter Drake
> https://sites.google.com/a/lclark.edu/drake/
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
>
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Rémi Coulom]

Hi Nick,

If you are to limit hardware in one tournament, I would prefer that it 
is not the slow tournament. The slow tournament is interesting because 
it pushes programs to their limits.


Rémi

On 10/10/2015 07:28 PM, Nick Wedd wrote:
Thanks to everyone for their interest and responses.  The second and 
third questions are easy:  I shall keep the zeroes in the "annual" 
table, and I shall update the crosstable after each round whenever 
this is convenient for me.  I really don't feel qualified to 
contribute to question 1, the "limited hardware" issue.  I like the 
idea of making the Slow tournaments limited to one core, one 
processor, one thread, or whatever, so as to reduce power consumed and 
contribute less to global warming, but I don't know if there is a 
practicable way of doing this.


I hope you will continue to discuss question 1.

Nick

On 10 October 2015 at 17:06, Jim O'Flaherty 
> wrote:


I second Peter's response.

On Oct 10, 2015 10:33 AM, "Peter Drake" > wrote:

I'm also for no limits, if only because there's no way to
enforce them.

If there is to be a limited division, I'd like to see all
programs run on identical hardware.

On Fri, Oct 9, 2015 at 6:07 AM, Hiroshi Yamashita
> wrote:

Hi Nick,

I'd like no limit. Restriction will lose a chance of massive
computer's programming. But one thread limit tournament
once a year may be interesting.

I like (2), and (3) is nice, but I'm already happy with
your reports!

Regards,
Hiroshi Yamashita


___
Computer-go mailing list
Computer-go@computer-go.org

http://computer-go.org/mailman/listinfo/computer-go




-- 
Peter Drake

https://sites.google.com/a/lclark.edu/drake/

___
Computer-go mailing list
Computer-go@computer-go.org 
http://computer-go.org/mailman/listinfo/computer-go


___
Computer-go mailing list
Computer-go@computer-go.org 
http://computer-go.org/mailman/listinfo/computer-go




--
Nick Wedd mapr...@gmail.com 


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Ingo Althöfer]

2 Cents from a non-programmer;

>> 1.  Limit on processor power?

From my computer chess background I can only recommend not to do
such a thing. The only consequence might be slower progress in computer
go in general. This would conflict with my longterm hopes:

* I want to see a bot win against a pro player not later than 2025

* and a win of a bot+amateur-team against a pro not later than 2020.

The programs on strong hardware were (in computer chess) always in
the forefront of progress.

Ingo.
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hiroshi Yamashita]

Hi Nick,

I'd like no limit. Restriction will lose a chance of massive
computer's programming. But one thread limit tournament
once a year may be interesting.

I like (2), and (3) is nice, but I'm already happy with your reports!

Regards,
Hiroshi Yamashita

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Stefan Kaitschick]

On Wed, Oct 7, 2015 at 7:21 PM, Dave Dyer  wrote:

>
> How about handicapping the hardware based on time.  Programs running
> on more powerful hardware would get less time.
>
>
 I think that's a good idea. Programs could even aquire a time ranking,
depending on their success in previous tournaments.
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Tobias Graf]

Hi,
just my 2 cents:

1. "Reducing computing power." Just let me quote the standings of the 
last 9x9 tournament.

1) 18 Cores
2) 80 Cores
3) 12 Cores
4) 288 Cores
5) 8 Cores

Moreover, using the 18 cores of place number one is affordable to 
everyone as Remi outlined.


Still, i would compete in a few desktop-hardware-tournaments. It should 
just have reasonable limits, there is no point to make a tournament on 
raspberry pi's ;-) Maybe one of the slow-tournaments can be changed into 
a desktop-hardware tournament.


2) stefan kaitschick proposed to "force a minimum time consumption on 
the first moves of 9*9 games. It's annoying as a spectator to have the 
first 8 moves or so just spit out on the board, forcing you to go back 
to see what happened there."
I don't like forcing too many things. But it is a very good idea, so i 
just added a one second book-delay to abakus. Maybe other authors find 
this useful, too.


Best,
Tobias

On 10/07/2015 12:27 PM, Nick Wedd wrote:
I am thinking of making some small changes to the way I run bot 
tournaments on KGS.  If you have ever taken part in a KGS bot 
tournament, I would like to hear your opinions on three things.



1.  Limit on processor power?

This is the main point on which I want your opinions.  The other two 
are trivial.


Several people have suggested to me that these events would be fairer 
if there were a limit on the computing power of the entrants. I would 
like to do this, but I don't know how. I have little understanding of 
the terminology, I don't know how /e.g./ multiple cores in one 
computer compare with multiple computers on one network, and I don't 
know how to count a graphics card. /If/ someone can find a way to 
specify an upper limit to permitted power which is clear and easy to 
understand, and /if/ most entrants would favor imposing such a limit, 
I will discuss what it should be, and apply it.  I am not able to 
check what entrants are really running on, but I will trust people.



2. Zeroes in the "Annual Championship" table.

The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in 
a cell where a program competed but did not score, and a blank where 
it did not compete (at least it should do, I sometimes get it wrong). 
I would prefer to omit these zeroes, they seem a bit rude. Also there 
is no clear distinction between competing and not competing - how 
should I treat a program which crashes and disappears after two 
rounds, or one (like AyaMC last Sunday) which plays in every round but 
is broken and has no chance of winning?  I realise that the zeroes 
some convey information that may be of interest. Should I continue to 
use them, or just leave those cells blank?



3. Live crosstable

When I write up my reports, I include a crosstable, like the one near 
the top of http://www.weddslist.com/kgs/past/116/index.html .  This is 
easy for me, I run a script which reads the data from the KGS page 
(http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) 
and builds the crosstable in html, which I copy into the tournament 
report. It only works for Swiss (and maybe Round Robin) tournaments. 
It works while the tournament is still running, though only between 
rounds.I could build a current crosstable each round during a 
tournament if there is any demand for it.


--
Nick Wedd mapr...@gmail.com 


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Erik van der Werf]

On Thu, Oct 8, 2015 at 1:10 PM, Tobias Graf  wrote:
> 1. "Reducing computing power." Just let me quote the standings of the last
> 9x9 tournament.
> 1) 18 Cores
> 2) 80 Cores
> 3) 12 Cores
> 4) 288 Cores
> 5) 8 Cores

Counting 'cores' is a bad idea; 'core' is mostly just a marketing term.
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

Tobias and all,

Tobias Graf: <56164f25.8010...@gmx.de>:
>Hi,
>just my 2 cents:
>
>1. "Reducing computing power." Just let me quote the standings of the 
>last 9x9 tournament.
>1) 18 Cores
>2) 80 Cores
>3) 12 Cores
>4) 288 Cores
>5) 8 Cores
>
>Moreover, using the 18 cores of place number one is affordable to 
>everyone as Remi outlined.

Looking at the before last 9x9 tournament 
(http://www.weddslist.com/kgs/past/113/index.html):
1) 20 cores x 3 GHz (Zen)
2) 18 cores x 2.9 GHz (CrazyStone)
3) 12 cores x 2.8 GHz (DolBaram)
4) 6 cores x 3.3 GHz (Aya)
5) 80 cores x 2.6GHz (Abakus).
#Zen, DolBaram and Aya were (practically) absent at the last 9x9.
##Above clock frequency numbers are rated ones and actual values can 
vary due to Intel Turbo Boost Tech.  My Xeon box usually runs at 3.3 to 
3.5 GHz.

Even 6-cores can beat 80-cores.  This is not surprising on small boards.  
Please evaluate the boost of your network parallelization on 9x9.  It's 
(maybe) very small (or sometimes negative; depends on the method, number 
of node computers, etc).  Cluster parallelization is, however, very 
important to push-up last one or two ranks on larger boards.

>Still, i would compete in a few desktop-hardware-tournaments. It should 
>just have reasonable limits, there is no point to make a tournament on 
>raspberry pi's ;-) Maybe one of the slow-tournaments can be changed into 
>a desktop-hardware tournament.
>
>2) stefan kaitschick proposed to "force a minimum time consumption on 
>the first moves of 9*9 games. It's annoying as a spectator to have the 
>first 8 moves or so just spit out on the board, forcing you to go back 
>to see what happened there."
>I don't like forcing too many things. But it is a very good idea, so i 
>just added a one second book-delay to abakus. Maybe other authors find 
>this useful, too.

Yes, it also helps human players prevent making a bad move on the spur 
of the moment.

Hideki

>On 10/07/2015 12:27 PM, Nick Wedd wrote:
>> I am thinking of making some small changes to the way I run bot 
>> tournaments on KGS.  If you have ever taken part in a KGS bot 
>> tournament, I would like to hear your opinions on three things.
>>
>>
>> 1.  Limit on processor power?
>>
>> This is the main point on which I want your opinions.  The other two 
>> are trivial.
>>
>> Several people have suggested to me that these events would be fairer 
>> if there were a limit on the computing power of the entrants. I would 
>> like to do this, but I don't know how. I have little understanding of 
>> the terminology, I don't know how /e.g./ multiple cores in one 
>> computer compare with multiple computers on one network, and I don't 
>> know how to count a graphics card. /If/ someone can find a way to 
>> specify an upper limit to permitted power which is clear and easy to 
>> understand, and /if/ most entrants would favor imposing such a limit, 
>> I will discuss what it should be, and apply it.  I am not able to 
>> check what entrants are really running on, but I will trust people.
>>
>>
>> 2. Zeroes in the "Annual Championship" table.
>>
>> The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in 
>> a cell where a program competed but did not score, and a blank where 
>> it did not compete (at least it should do, I sometimes get it wrong). 
>> I would prefer to omit these zeroes, they seem a bit rude. Also there 
>> is no clear distinction between competing and not competing - how 
>> should I treat a program which crashes and disappears after two 
>> rounds, or one (like AyaMC last Sunday) which plays in every round but 
>> is broken and has no chance of winning?  I realise that the zeroes 
>> some convey information that may be of interest. Should I continue to 
>> use them, or just leave those cells blank?
>>
>>
>> 3. Live crosstable
>>
>> When I write up my reports, I include a crosstable, like the one near 
>> the top of http://www.weddslist.com/kgs/past/116/index.html .  This is 
>> easy for me, I run a script which reads the data from the KGS page 
>> (http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) 
>> and builds the crosstable in html, which I copy into the tournament 
>> report. It only works for Swiss (and maybe Round Robin) tournaments. 
>> It works while the tournament is still running, though only between 
>> rounds.I could build a current crosstable each round during a 
>> tournament if there is any demand for it.
>>
>> -- 
>> Nick Wedd mapr...@gmail.com 
>>
>>
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
> inline file
>___
>Computer-go mailing list
>Computer-go@computer-go.org
>http://computer-go.org/mailman/listinfo/computer-go
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[John Tromp]

On Wed, Oct 7, 2015 at 7:44 PM, Petr Baudis  wrote:
> On Wed, Oct 07, 2015 at 02:29:27PM +0200, Erik van der Werf wrote:
>> A measure that I find reasonable is a limit on number of threads x
>> clock frequency.

>   I'm not sure this would work well.  The #playouts difference between
> an old Bulldozer and new Haswell with the same nominal frequency and
> #cores might be many tens of percents.  Another question is how to find
> the scaling factors for other architectures?

Instead of frequency, one could use the single threaded Fhourstones score.

https://en.wikipedia.org/wiki/Fhourstones
http://tromp.github.io/c4/fhour.html

regards,
-John
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Detlef Schmicker]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

If I compare hardware specs in the KGS tournaments I usually use

http://spec.org/cpu2006/results/rint2006.html

(Multithread Integer operations are the ones most important for
computer go programs I think)

Detlef

Am 08.10.2015 um 05:48 schrieb Hideki Kato:
> Petr Baudis: <20151007234420.gb9...@machine.or.cz>:
> 
>> (I might propose relaxing the requirement even further, from one 
>> desktop cpu to just one cpu - as in physical package.  Many
>> cloud providers might give you a Xeon instance that's about as
>> good as a regular i7.  You mainly want to exclude dedicated
>> multi-CPU servers and clusters.)
> 
> For Amazon Clusters, Amazon provides vCPU spec (for example, EC2: 
> https://aws.amazon.com/ec2/details/ and 
> http://aws.amazon.com/ec2/previous-generation/). Only such
> instances that vCPU spec number is 1 belong PC class.
> 
> The principle is simple; such computers with one physical CPU
> installed or equivalents can belong PC class.  The exceptions are
> Intel Xeon E7 series processors, AMD Opteron, and IBM Power, as
> they have so many physical cores in a socket and cannot be said
> "personal".  In other words, processors up-to regular Intel Core i7
> processor can belong PC class as Petr suggested.
> 
> Then, how about Core i7 5960X Extreme Edition?  5960X has eight
> cores and actually a Xeon processor.  Is this regular i7?>Petr I
> think, as there are many PC with 5960X are sold widely as a
> high-end desktop for gamers/enthusiasts, 5960X can belong PC class.
> Comments are welcome. #This definition could prefer Intel...
> 
> Hideki Hideki
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJWFgTVAAoJEInWdHg+Znf4L8sQAIAiDAfhvT/Dn+08VMof1nCI
603p+IlHgP45P63coAsPed1TqIEC+fiskBvqObgC0A1InDIaAtOrlS3+BbG/kHKH
WCO5JqeWxNvbYQP/h3AsJQQMeqrfBglO5ZFBGkcglDYDzDwdP5zt/P8pmO85NSXs
6NaefFQRbqFlSk3MqdSWV4OVh9iDd4J2kFFk10iQnSVu/uULju2xXY6S8wCVAGED
yhpeZ346eUBQXNtgCBOve5vsFCDpggq/k0VVb/vF7VXjL9IvsIyP3+X6koNeaPcR
+jeRoS5ORXlByHpwX31DNxSqEZK8RkpWOmQtv63/GYc3wL1+Qy9TUPrEHQLEYSdt
FaLGYjsWcY+aZphTdTG9kfZt8QPB/ArBHP4QOdS8b3XP+gsNrEnZ+cSvAbdFtxrG
NyhjT4YzZ7nZpD41L9nXiXhDdnC/UBYX2ynjxtU9B65NjddTz7qAssVvsiaql73m
o8Wpo/GgW/pIljc2LtdbCemCjFmHU+5yHrYzsyWIzFkClF6RIOuMTQ3794PaArC9
qLAYLTzNke/a4blpzhzByF7DdD06FVNFi/NZluBgD42DGGW+bKzHN34lJ2/qTfXy
zy7R+uymV5MCMgcLovVYi7Pmi3HKs8m2+oiR96KOZ0jtZDwx3bfgR3KXk6jsmYj/
VHbg112fKssMPqV5kP3b
=48Gw
-END PGP SIGNATURE-
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Erik van der Werf]

Hi Nick,

Some kind of limit on processing power would be interesting. To me it
seems clear that a program like Zen benefits a lot by using more
processing power than it's close competitors.

A measure that I find reasonable is a limit on number of threads x
clock frequency. E.g., a program running on 64 intel cores, with 2
threads per core, at 3 Ghz would be counted as using 64x2x3 = 384 GHz,
and a program running on 24 amd cores, with 1 thread per core, at 2.6
Ghz would be counted as 62.4 GHz. As long as the top just uses x86
type processors this should work reasonably well. For GPU's or ARM
processors there probably needs to be a scaling factor.

Another option could be to limit the power supply, e.g., you may not
use more than say 400 Watt.

BR,
Erik


PS As someone that participates only occasionally I like the zeros,
but perhaps there is a friendlier way to indicate
participation/absence.

PS2 Crazy idea?: "Machine A plays black, machine B plays white. First
player proposes the komi, the second player chooses the
color"


On Wed, Oct 7, 2015 at 12:52 PM, Detlef Schmicker  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hi Nick,
>
> 3. would be great, I am very often going through round results to see
> where CS or abakus has lost its games:)
>
> 1. I do not see a way to do this but running on same hardware (e.g.
> Amazon EC2 with graphic card). Even this is unfair, as programs might
> be optimized to other configurations (cluster)
>
> Detlef
>
> Am 07.10.2015 um 12:27 schrieb Nick Wedd:
>> I am thinking of making some small changes to the way I run bot
>> tournaments on KGS.  If you have ever taken part in a KGS bot
>> tournament, I would like to hear your opinions on three things.
>>
>>
>> 1.  Limit on processor power?
>>
>> This is the main point on which I want your opinions.  The other
>> two are trivial.
>>
>> Several people have suggested to me that these events would be
>> fairer if there were a limit on the computing power of the
>> entrants. I would like to do this, but I don't know how. I have
>> little understanding of the terminology, I don't know how *e.g.*
>> multiple cores in one computer compare with multiple computers on
>> one network, and I don't know how to count a graphics card.  *If*
>> someone can find a way to specify an upper limit to permitted power
>> which is clear and easy to understand, and *if* most entrants would
>> favor imposing such a limit, I will discuss what it should be, and
>> apply it.  I am not able to check what entrants are really running
>> on, but I will trust people.
>>
>>
>> 2. Zeroes in the "Annual Championship" table.
>>
>> The table at http://www.weddslist.com/kgs/annual/index.html has a 0
>> in a cell where a program competed but did not score, and a blank
>> where it did not compete (at least it should do, I sometimes get it
>> wrong). I would prefer to omit these zeroes, they seem a bit rude.
>> Also there is no clear distinction between competing and not
>> competing - how should I treat a program which crashes and
>> disappears after two rounds, or one (like AyaMC last Sunday) which
>> plays in every round but is broken and has no chance of winning?  I
>> realise that the zeroes some convey information that may be of
>> interest.  Should I continue to use them, or just leave those cells
>> blank?
>>
>>
>> 3. Live crosstable
>>
>> When I write up my reports, I include a crosstable, like the one
>> near the top of http://www.weddslist.com/kgs/past/116/index.html .
>> This is easy for me, I run a script which reads the data from the
>> KGS page ( http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in
>> this case) and builds the crosstable in html, which I copy into the
>> tournament report. It only works for Swiss (and maybe Round Robin)
>> tournaments. It works while the tournament is still running, though
>> only between rounds.I could build a current crosstable each round
>> during a tournament if there is any demand for it.
>>
>>
>>
>> ___ Computer-go mailing
>> list Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iQIcBAEBAgAGBQJWFPljAAoJEInWdHg+Znf4pMwQAI/4c2HhXA1DgAQEcOpulRAg
> SSrDGCLyAypTQb3p1YDx45bvIIE9r0qBL+Ze+GRO/gD9peYmbayQWO7pQiL+2A/i
> 2NSkr5gj6SR2JKW924Ba7NnBBIOil+bVX4Jku8PZJkz7MWwsP7LxFKhX2hA81Iis
> EFdLAJ74atr7LSbERuhDYpwn/FcS9ag2h9k1zcwXD6R/OSuB46+OyR6dWw5NCrAL
> FhjHo22rXig741ZTHtAxx7VXRwMdn6RV2oqkMbajFa7CLHFTcLRLMv3ix2TFyyd9
> JXwYKzJLxCeUNBWQ8WI5wvEqf7BPRjZcg0jujfR29zpg0AEooglrnnyIwRee9DPy
> BGAxR8FmW/5kC1tygvM2c+shxvdhGrIB+1f8UoKIMp/IdhSLGuHc3Dq077+jCjG/
> QYA766C9tg+mqEPRp3nzqTP4G6cdTGlPfPLxMGGz6r1ltdlUwcqAy+Q/x6vztRJz
> HH7ThXZgNpruhKqoDkltxW6udGpdeUiRW5u2JDCHVWPI6S+AjhdbtpI4EG+7Awwq
> GUqk2LFlh3PItxl3UkYBpfevdHnUKXXG5UDxODEIQx43QgyYqLRjMsMXfaJGCbce
> TBsJ9CvvyadAN2JhOzeIVRVZoWbzyk3t/+Pkyg4erhyi517jhmaXJApBQiKl1lOL
> EqL1TfDx1ZJm+Ow0y18D
> 

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Stefan Kaitschick]

>
> 1. I do not see a way to do this but running on same hardware (e.g.
> Amazon EC2 with graphic card). Even this is unfair, as programs might
> be optimized to other configurations (cluster)
>
>
 First, there is the question is fairness is even desirable.
But also, as you say, it is really impossible to make an event "fair", even
with identical hardware, because this will give the reverse advantage to
programs that put a premium on resource conservation in their design. And
with the new CNN developments, it would be a shame to put resource limits
to the top of the agenda.

As a Kibitz, I would have an added trivial wish: force a minimum time
consumption on the first moves of 9*9 games. It's annoying as a spectator
to have the first 8 moves or so just spit out on the board, forcing you to
go back to see what happened there. It would also slightly lessen the
advantage of bots with books.

Stefan
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Erik van der Werf]

On Wed, Oct 7, 2015 at 5:02 PM, Hideki Kato  wrote:
> Erik,
>
> Erik van der Werf: 
> 

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

I'd like to explain my idea more.

The monthly tournaments (and current annual championship (FL class)) are 
not necessary to change.   Just creating one more championship class for 
desktop computers (PC class).  

An entrant has to state so if he/she wants to belong PC class (i.e., all 
entrants automatically belong FL class).  One program can entry both 
classes.

Hideki

Goncalo Mendes Ferreira: <561535a9.7010...@sapo.pt>:
>I think this is a good compromise. Monthly tournaments free for everyone 

>and maybe an yearly one segregated by hardware. Having segregated 

>monthly tournaments would be a bit taxing on the organization and people 

>who would submit their programs for all hardware divisions. Segregation 

>based on consumer grade desktops vs others is simple enough.

>

>Since there is no way to confirm the hardware and KGS tournament results 

>cannot be used to measure program strength in a reliable way, I'd prefer 

>the current theme of casual competitive tournaments.

>

>Gonçalo

>

>On 07/10/2015 15:56, Hideki Kato wrote:

>> Nick & all,

>>

>> Another direction for the hardware.  How about introducing two classes

>> for the Annual Championship?  I.e., no-limit (formula libre) class

>> and personal computer one.  My proposal for the later is very simple;

>> one desktop (i.e., non-server) cpu and one video card.

>>

>> Hideki

>

>___

>Computer-go mailing list

>Computer-go@computer-go.org

>http://computer-go.org/mailman/listinfo/computer-go
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

Nick & all,

1. Although introducing some limitation of cpu power is an intersting 
idea (actually my GPW Cup does), I think it's too early for KGS bot 
tournaments.  

How to utilize computer clusters' power for planning tasks is a common 
and important reseach theme now.  As communication over network is less 
effective than in-box one, playing-strength per (total) cpu-power get 
smaller when using computer clusters.  Zen's root parallelization 
improves up-to 1 or 2 ranks but TDS based algorithms (used for Gommora 
and MP-Fuego) are expected to give more improvements.  Preventing such 
effort must be a bad idea, I strongly believe.  So, at least, a simple 
sum of cpu power of all node computers is not acceptable.  (Some 
discouting could be?)

Cpu power of each SMP or NUMA box can be computed by 
number-of-(physical-)cores times clock-frequency (although Erik used 
logical-cores).  Using number-of-threads instead might be better.  For 
more fairness, some factors can be defined for processor arichitectures 
or manufacturers, because some participants have to use non-Intel 
processors due to their environments.  

For GPUs, I have no concrete idea now.  Simple flops is not enough and 
more discussion is necessary, I believe.

Another idea, you (or we?) can define some benchmark program(s).

2. I don't understand this at all.  It's just a record of fact.  
Intentional omitting of information must be a bad idea.

3. Watching the crosstable in real-time should be a fan for all 
observers.

Hideki

Nick Wedd: :
>I am thinking of making some small changes to the way I run bot tournaments
>on KGS.  If you have ever taken part in a KGS bot tournament, I would like
>to hear your opinions on three things.
>
>
>1.  Limit on processor power?
>
>This is the main point on which I want your opinions.  The other two are
>trivial.
>
>Several people have suggested to me that these events would be fairer if
>there were a limit on the computing power of the entrants. I would like to
>do this, but I don't know how. I have little understanding of the
>terminology, I don't know how *e.g.* multiple cores in one computer compare
>with multiple computers on one network, and I don't know how to count a
>graphics card.  *If* someone can find a way to specify an upper limit to
>permitted power which is clear and easy to understand, and *if* most
>entrants would favor imposing such a limit, I will discuss what it should
>be, and apply it.  I am not able to check what entrants are really running
>on, but I will trust people.
>
>
>2. Zeroes in the "Annual Championship" table.
>
>The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a
>cell where a program competed but did not score, and a blank where it did
>not compete (at least it should do, I sometimes get it wrong). I would
>prefer to omit these zeroes, they seem a bit rude. Also there is no clear
>distinction between competing and not competing - how should I treat a
>program which crashes and disappears after two rounds, or one (like AyaMC
>last Sunday) which plays in every round but is broken and has no chance of
>winning?  I realise that the zeroes some convey information that may be of
>interest.  Should I continue to use them, or just leave those cells blank?
>
>
>3. Live crosstable
>
>When I write up my reports, I include a crosstable, like the one near the
>top of http://www.weddslist.com/kgs/past/116/index.html .  This is easy for
>me, I run a script which reads the data from the KGS page (
>http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and
>builds the crosstable in html, which I copy into the tournament report. It
>only works for Swiss (and maybe Round Robin) tournaments. It works while
>the tournament is still running, though only between rounds.I could build a
>current crosstable each round during a tournament if there is any demand
>for it.
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

Nick & all,

Another direction for the hardware.  How about introducing two classes 
for the Annual Championship?  I.e., no-limit (formula libre) class 
and personal computer one.  My proposal for the later is very simple; 
one desktop (i.e., non-server) cpu and one video card.

Hideki

Nick Wedd: :
>I am thinking of making some small changes to the way I run bot tournaments
>on KGS.  If you have ever taken part in a KGS bot tournament, I would like
>to hear your opinions on three things.
>
>
>1.  Limit on processor power?
>
>This is the main point on which I want your opinions.  The other two are
>trivial.
>
>Several people have suggested to me that these events would be fairer if
>there were a limit on the computing power of the entrants. I would like to
>do this, but I don't know how. I have little understanding of the
>terminology, I don't know how *e.g.* multiple cores in one computer compare
>with multiple computers on one network, and I don't know how to count a
>graphics card.  *If* someone can find a way to specify an upper limit to
>permitted power which is clear and easy to understand, and *if* most
>entrants would favor imposing such a limit, I will discuss what it should
>be, and apply it.  I am not able to check what entrants are really running
>on, but I will trust people.
>
>
>2. Zeroes in the "Annual Championship" table.
>
>The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a
>cell where a program competed but did not score, and a blank where it did
>not compete (at least it should do, I sometimes get it wrong). I would
>prefer to omit these zeroes, they seem a bit rude. Also there is no clear
>distinction between competing and not competing - how should I treat a
>program which crashes and disappears after two rounds, or one (like AyaMC
>last Sunday) which plays in every round but is broken and has no chance of
>winning?  I realise that the zeroes some convey information that may be of
>interest.  Should I continue to use them, or just leave those cells blank?
>
>
>3. Live crosstable
>
>When I write up my reports, I include a crosstable, like the one near the
>top of http://www.weddslist.com/kgs/past/116/index.html .  This is easy for
>me, I run a script which reads the data from the KGS page (
>http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and
>builds the crosstable in html, which I copy into the tournament report. It
>only works for Swiss (and maybe Round Robin) tournaments. It works while
>the tournament is still running, though only between rounds.I could build a
>current crosstable each round during a tournament if there is any demand
>for it.
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

Erik,

Erik van der Werf: 

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Gonçalo Mendes Ferreira]

I think this is a good compromise. Monthly tournaments free for everyone 
and maybe an yearly one segregated by hardware. Having segregated 
monthly tournaments would be a bit taxing on the organization and people 
who would submit their programs for all hardware divisions. Segregation 
based on consumer grade desktops vs others is simple enough.


Since there is no way to confirm the hardware and KGS tournament results 
cannot be used to measure program strength in a reliable way, I'd prefer 
the current theme of casual competitive tournaments.


Gonçalo

On 07/10/2015 15:56, Hideki Kato wrote:

Nick & all,

Another direction for the hardware.  How about introducing two classes
for the Annual Championship?  I.e., no-limit (formula libre) class
and personal computer one.  My proposal for the later is very simple;
one desktop (i.e., non-server) cpu and one video card.

Hideki


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

Erik van der Werf: 
:
>On Wed, Oct 7, 2015 at 5:02 PM, Hideki Kato  wrote:
>> Erik,
>>
>> Erik van der Werf: 
>> 

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Erik van der Werf]

Although I agree on the research argument (setting no limits
encourages work on massive parallel distributed architectures), I do
find it a bit funny to see this argument coming from team Zen. As far
as I know team Zen does not publish their research findings (or did I
miss some papers?).

Erik


On Wed, Oct 7, 2015 at 4:20 PM, Hideki Kato  wrote:
> Nick & all,
>
> 1. Although introducing some limitation of cpu power is an intersting
> idea (actually my GPW Cup does), I think it's too early for KGS bot
> tournaments.
>
> How to utilize computer clusters' power for planning tasks is a common
> and important reseach theme now.  As communication over network is less
> effective than in-box one, playing-strength per (total) cpu-power get
> smaller when using computer clusters.  Zen's root parallelization
> improves up-to 1 or 2 ranks but TDS based algorithms (used for Gommora
> and MP-Fuego) are expected to give more improvements.  Preventing such
> effort must be a bad idea, I strongly believe.  So, at least, a simple
> sum of cpu power of all node computers is not acceptable.  (Some
> discouting could be?)
>
> Cpu power of each SMP or NUMA box can be computed by
> number-of-(physical-)cores times clock-frequency (although Erik used
> logical-cores).  Using number-of-threads instead might be better.  For
> more fairness, some factors can be defined for processor arichitectures
> or manufacturers, because some participants have to use non-Intel
> processors due to their environments.
>
> For GPUs, I have no concrete idea now.  Simple flops is not enough and
> more discussion is necessary, I believe.
>
> Another idea, you (or we?) can define some benchmark program(s).
>
> 2. I don't understand this at all.  It's just a record of fact.
> Intentional omitting of information must be a bad idea.
>
> 3. Watching the crosstable in real-time should be a fan for all
> observers.
>
> Hideki
>
> Nick Wedd: 
> :
>>I am thinking of making some small changes to the way I run bot tournaments
>>on KGS.  If you have ever taken part in a KGS bot tournament, I would like
>>to hear your opinions on three things.
>>
>>
>>1.  Limit on processor power?
>>
>>This is the main point on which I want your opinions.  The other two are
>>trivial.
>>
>>Several people have suggested to me that these events would be fairer if
>>there were a limit on the computing power of the entrants. I would like to
>>do this, but I don't know how. I have little understanding of the
>>terminology, I don't know how *e.g.* multiple cores in one computer compare
>>with multiple computers on one network, and I don't know how to count a
>>graphics card.  *If* someone can find a way to specify an upper limit to
>>permitted power which is clear and easy to understand, and *if* most
>>entrants would favor imposing such a limit, I will discuss what it should
>>be, and apply it.  I am not able to check what entrants are really running
>>on, but I will trust people.
>>
>>
>>2. Zeroes in the "Annual Championship" table.
>>
>>The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a
>>cell where a program competed but did not score, and a blank where it did
>>not compete (at least it should do, I sometimes get it wrong). I would
>>prefer to omit these zeroes, they seem a bit rude. Also there is no clear
>>distinction between competing and not competing - how should I treat a
>>program which crashes and disappears after two rounds, or one (like AyaMC
>>last Sunday) which plays in every round but is broken and has no chance of
>>winning?  I realise that the zeroes some convey information that may be of
>>interest.  Should I continue to use them, or just leave those cells blank?
>>
>>
>>3. Live crosstable
>>
>>When I write up my reports, I include a crosstable, like the one near the
>>top of http://www.weddslist.com/kgs/past/116/index.html .  This is easy for
>>me, I run a script which reads the data from the KGS page (
>>http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and
>>builds the crosstable in html, which I copy into the tournament report. It
>>only works for Swiss (and maybe Round Robin) tournaments. It works while
>>the tournament is still running, though only between rounds.I could build a
>>current crosstable each round during a tournament if there is any demand
>>for it.
> --
> Hideki Kato 
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Gonçalo Mendes Ferreira]

I think cluster renting is a little more complex than Rémi makes it 
seem, because behind the few hours of tournament play will be many more 
hours of testing. There are also other reasons why programs may only 
target personal computers, for instance if they're commercial for 
personal use.


If we're thinking of categories, we could adopt an honor system like:
Category A0 - If your computing power is similar to that of a Raspberry Pi
and so on, instead of actually specifying how "computing power" can be 
compared across GPUs/clusters. I'd still prefer very little categories 
(2 or 3) though.



So finally, my actual proposal, is that programs with advanced hardware
configurations handicap themselves.  For example, Zen could unilaterally
announce a lower time limit that it intends to adhere to.


As long as this isn't enforced, I think it's great if a program 
handicaps itself and still wins, it adds excitement to the tournaments.


Gonçalo
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Rémi Coulom]

Hi Nick,

I don’t care much about having a limit on processing power. I’d be happy either 
way.

Cloud computing platforms like Amazon EC2 allows to rent powerful servers at a 
low price. The machine I used for the tournament cost me 0.3$/hour or so. So 
the argument that only rich or academic people can get powerful hardware is not 
good. A cluster of 8 such machines would still be quite cheap. And making an 
efficient distributed search algorithm is an interesting and challenging 
technical problem. So I feel it is interesting to allow big clusters.

Thanks for organizing the KGS tournaments, by the way.

Rémi


On 7 oct. 2015, at 12:27, Nick Wedd  wrote:

> I am thinking of making some small changes to the way I run bot tournaments 
> on KGS.  If you have ever taken part in a KGS bot tournament, I would like to 
> hear your opinions on three things.
> 
> 
> 1.  Limit on processor power?
> 
> This is the main point on which I want your opinions.  The other two are 
> trivial.
> 
> Several people have suggested to me that these events would be fairer if 
> there were a limit on the computing power of the entrants. I would like to do 
> this, but I don't know how. I have little understanding of the terminology, I 
> don't know how e.g. multiple cores in one computer compare with multiple 
> computers on one network, and I don't know how to count a graphics card.  If 
> someone can find a way to specify an upper limit to permitted power which is 
> clear and easy to understand, and if most entrants would favor imposing such 
> a limit, I will discuss what it should be, and apply it.  I am not able to 
> check what entrants are really running on, but I will trust people.
> 
> 
> 2. Zeroes in the "Annual Championship" table.
> 
> The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a cell 
> where a program competed but did not score, and a blank where it did not 
> compete (at least it should do, I sometimes get it wrong). I would prefer to 
> omit these zeroes, they seem a bit rude. Also there is no clear distinction 
> between competing and not competing - how should I treat a program which 
> crashes and disappears after two rounds, or one (like AyaMC last Sunday) 
> which plays in every round but is broken and has no chance of winning?  I 
> realise that the zeroes some convey information that may be of interest.  
> Should I continue to use them, or just leave those cells blank?
> 
> 
> 3. Live crosstable
> 
> When I write up my reports, I include a crosstable, like the one near the top 
> of http://www.weddslist.com/kgs/past/116/index.html .  This is easy for me, I 
> run a script which reads the data from the KGS page 
> (http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and 
> builds the crosstable in html, which I copy into the tournament report. It 
> only works for Swiss (and maybe Round Robin) tournaments. It works while the 
> tournament is still running, though only between rounds.I could build a 
> current crosstable each round during a tournament if there is any demand for 
> it.
> 
> -- 
> Nick Wedd  mapr...@gmail.com
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Rémi Coulom]

Hi Nick, 


I don’t care much about having a limit on processing power. I’d be happy either 
way. 


Cloud computing platforms like Amazon EC2 allows to rent powerful servers at a 
low price. The machine I used for the tournament cost me 0.3$/hour or so. So 
the argument that only rich or academic people can get powerful hardware is not 
good. A cluster of 8 such machines would still be quite cheap. And making an 
efficient distributed search algorithm is an interesting and challenging 
technical problem. So I feel it is interesting to allow big clusters. 

I can beat them on a single machine anyway ;-) 



Thanks for organizing the KGS tournaments, by the way. 


Rémi 





On 7 oct. 2015, at 12:27, Nick Wedd < mapr...@gmail.com > wrote: 



I am thinking of making some small changes to the way I run bot tournaments on 
KGS. If you have ever taken part in a KGS bot tournament, I would like to hear 
your opinions on three things. 




1. Limit on processor power? 


This is the main point on which I want your opinions. The other two are 
trivial. 


Several people have suggested to me that these events would be fairer if there 
were a limit on the computing power of the entrants. I would like to do this, 
but I don't know how. I have little understanding of the terminology, I don't 
know how e.g. multiple cores in one computer compare with multiple computers on 
one network, and I don't know how to count a graphics card. If someone can find 
a way to specify an upper limit to permitted power which is clear and easy to 
understand, and if most entrants would favor imposing such a limit, I will 
discuss what it should be, and apply it. I am not able to check what entrants 
are really running on, but I will trust people. 




2. Zeroes in the "Annual Championship" table. 


The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a cell 
where a program competed but did not score, and a blank where it did not 
compete (at least it should do, I sometimes get it wrong). I would prefer to 
omit these zeroes, they seem a bit rude. Also there is no clear distinction 
between competing and not competing - how should I treat a program which 
crashes and disappears after two rounds, or one (like AyaMC last Sunday) which 
plays in every round but is broken and has no chance of winning? I realise that 
the zeroes some convey information that may be of interest. Should I continue 
to use them, or just leave those cells blank? 




3. Live crosstable 


When I write up my reports, I include a crosstable, like the one near the top 
of http://www.weddslist.com/kgs/past/116/index.html . This is easy for me, I 
run a script which reads the data from the KGS page ( 
http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and builds 
the crosstable in html, which I copy into the tournament report. It only works 
for Swiss (and maybe Round Robin) tournaments. It works while the tournament is 
still running, though only between rounds.I could build a current crosstable 
each round during a tournament if there is any demand for it. 


-- 

Nick Wedd mapr...@gmail.com ___ 
Computer-go mailing list 
Computer-go@computer-go.org 
http://computer-go.org/mailman/listinfo/computer-go 


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Dave Dyer]

How about handicapping the hardware based on time.  Programs running
on more powerful hardware would get less time.  

On the other hand, improving the software includes making use of more
powerful hardware.  Handicapping (or banning) powerful hardware would
discourage that.

On third hand, development teams get the most information out of competitions
if the race is close, so it's in their interest to not crush competitors.

So finally, my actual proposal, is that programs with advanced hardware
configurations handicap themselves.  For example, Zen could unilaterally
announce a lower time limit that it intends to adhere to.

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Sebastian Scheib]

I know, this is a lot of work, but what about "caegories" ?

2015-10-07 14:06 GMT-03:00 Rémi Coulom :

> Hi Nick,
>
> I don’t care much about having a limit on processing power. I’d be happy
> either way.
>
> Cloud computing platforms like Amazon EC2 allows to rent powerful servers
> at a low price. The machine I used for the tournament cost me 0.3$/hour or
> so. So the argument that only rich or academic people can get powerful
> hardware is not good. A cluster of 8 such machines would still be quite
> cheap. And making an efficient distributed search algorithm is an
> interesting and challenging technical problem. So I feel it is interesting
> to allow big clusters.
>
> I can beat them on a single machine anyway ;-)
>
> Thanks for organizing the KGS tournaments, by the way.
>
> Rémi
>
>
> On 7 oct. 2015, at 12:27, Nick Wedd  wrote:
>
> I am thinking of making some small changes to the way I run bot
> tournaments on KGS.  If you have ever taken part in a KGS bot tournament, I
> would like to hear your opinions on three things.
>
>
> 1.  Limit on processor power?
>
> This is the main point on which I want your opinions.  The other two are
> trivial.
>
> Several people have suggested to me that these events would be fairer if
> there were a limit on the computing power of the entrants. I would like to
> do this, but I don't know how. I have little understanding of the
> terminology, I don't know how *e.g.* multiple cores in one computer
> compare with multiple computers on one network, and I don't know how to
> count a graphics card.  *If* someone can find a way to specify an upper
> limit to permitted power which is clear and easy to understand, and *if* most
> entrants would favor imposing such a limit, I will discuss what it should
> be, and apply it.  I am not able to check what entrants are really running
> on, but I will trust people.
>
>
> 2. Zeroes in the "Annual Championship" table.
>
> The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a
> cell where a program competed but did not score, and a blank where it did
> not compete (at least it should do, I sometimes get it wrong). I would
> prefer to omit these zeroes, they seem a bit rude. Also there is no clear
> distinction between competing and not competing - how should I treat a
> program which crashes and disappears after two rounds, or one (like AyaMC
> last Sunday) which plays in every round but is broken and has no chance of
> winning?  I realise that the zeroes some convey information that may be of
> interest.  Should I continue to use them, or just leave those cells blank?
>
>
> 3. Live crosstable
>
> When I write up my reports, I include a crosstable, like the one near the
> top of http://www.weddslist.com/kgs/past/116/index.html .  This is easy
> for me, I run a script which reads the data from the KGS page (
> http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and
> builds the crosstable in html, which I copy into the tournament report. It
> only works for Swiss (and maybe Round Robin) tournaments. It works while
> the tournament is still running, though only between rounds.I could build a
> current crosstable each round during a tournament if there is any demand
> for it.
>
> --
> Nick Wedd  mapr...@gmail.com
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
>
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>



-- 
Dracux
*http://www.dracux.com *
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go