Re: [Computer-go] KGS bot tournaments - what are your opinions?

[John Tromp]

On Wed, Oct 7, 2015 at 7:44 PM, Petr Baudis  wrote:
> On Wed, Oct 07, 2015 at 02:29:27PM +0200, Erik van der Werf wrote:
>> A measure that I find reasonable is a limit on number of threads x
>> clock frequency.

>   I'm not sure this would work well.  The #playouts difference between
> an old Bulldozer and new Haswell with the same nominal frequency and
> #cores might be many tens of percents.  Another question is how to find
> the scaling factors for other architectures?

Instead of frequency, one could use the single threaded Fhourstones score.

https://en.wikipedia.org/wiki/Fhourstones
http://tromp.github.io/c4/fhour.html

regards,
-John
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Detlef Schmicker]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

If I compare hardware specs in the KGS tournaments I usually use

http://spec.org/cpu2006/results/rint2006.html

(Multithread Integer operations are the ones most important for
computer go programs I think)

Detlef

Am 08.10.2015 um 05:48 schrieb Hideki Kato:
> Petr Baudis: <20151007234420.gb9...@machine.or.cz>:
> 
>> (I might propose relaxing the requirement even further, from one 
>> desktop cpu to just one cpu - as in physical package.  Many
>> cloud providers might give you a Xeon instance that's about as
>> good as a regular i7.  You mainly want to exclude dedicated
>> multi-CPU servers and clusters.)
> 
> For Amazon Clusters, Amazon provides vCPU spec (for example, EC2: 
> https://aws.amazon.com/ec2/details/ and 
> http://aws.amazon.com/ec2/previous-generation/). Only such
> instances that vCPU spec number is 1 belong PC class.
> 
> The principle is simple; such computers with one physical CPU
> installed or equivalents can belong PC class.  The exceptions are
> Intel Xeon E7 series processors, AMD Opteron, and IBM Power, as
> they have so many physical cores in a socket and cannot be said
> "personal".  In other words, processors up-to regular Intel Core i7
> processor can belong PC class as Petr suggested.
> 
> Then, how about Core i7 5960X Extreme Edition?  5960X has eight
> cores and actually a Xeon processor.  Is this regular i7?>Petr I
> think, as there are many PC with 5960X are sold widely as a
> high-end desktop for gamers/enthusiasts, 5960X can belong PC class.
> Comments are welcome. #This definition could prefer Intel...
> 
> Hideki Hideki
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJWFgTVAAoJEInWdHg+Znf4L8sQAIAiDAfhvT/Dn+08VMof1nCI
603p+IlHgP45P63coAsPed1TqIEC+fiskBvqObgC0A1InDIaAtOrlS3+BbG/kHKH
WCO5JqeWxNvbYQP/h3AsJQQMeqrfBglO5ZFBGkcglDYDzDwdP5zt/P8pmO85NSXs
6NaefFQRbqFlSk3MqdSWV4OVh9iDd4J2kFFk10iQnSVu/uULju2xXY6S8wCVAGED
yhpeZ346eUBQXNtgCBOve5vsFCDpggq/k0VVb/vF7VXjL9IvsIyP3+X6koNeaPcR
+jeRoS5ORXlByHpwX31DNxSqEZK8RkpWOmQtv63/GYc3wL1+Qy9TUPrEHQLEYSdt
FaLGYjsWcY+aZphTdTG9kfZt8QPB/ArBHP4QOdS8b3XP+gsNrEnZ+cSvAbdFtxrG
NyhjT4YzZ7nZpD41L9nXiXhDdnC/UBYX2ynjxtU9B65NjddTz7qAssVvsiaql73m
o8Wpo/GgW/pIljc2LtdbCemCjFmHU+5yHrYzsyWIzFkClF6RIOuMTQ3794PaArC9
qLAYLTzNke/a4blpzhzByF7DdD06FVNFi/NZluBgD42DGGW+bKzHN34lJ2/qTfXy
zy7R+uymV5MCMgcLovVYi7Pmi3HKs8m2+oiR96KOZ0jtZDwx3bfgR3KXk6jsmYj/
VHbg112fKssMPqV5kP3b
=48Gw
-END PGP SIGNATURE-
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] Fast pick from a probability list

[Detlef Schmicker]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I have a probability table of all possible moves. What is the fastest
way to pick with probability, possibly with reducing the quality of
probability?!

I could not find any discussion on this on computer-go, but probably I
missed it :(

Thansk a lot

Detlef
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJWFPijAAoJEInWdHg+Znf4LZwQAKGuQBtxb6/v+zaczghStCcU
5/a6oKaMNM2CJgCPMzgR2JKKb0gJVS0EldsOCmtQIgS2ZiBn4yP7y6uMGdtvEdMp
ytod1uV/KnW0GaTbaQ2RItIifCX/ft8JMrcchgW9wFVFitZpaPierQZuSOAoqQL1
IWXyZqmr92IoarZWI1MABtGOfiP7OFT91X17tUvhCmJ+dxrx7bczxnhkLfczEK4l
shkuourEX3VYiS/55wE/RwNmyuUnUYXy08SK3Ii4Tp9e5p0lS+i7eCJI/+wzqnUZ
HfjNDF+rPa/EpBo3a9VrTsxN3O8gYkzbiQLmWAZBKYrhwlFikOZ196VFALyZMfXE
KWFrK1niRaubP+00iQb1sXlntGbaAHm6e6+cGOVgykNMjQwseCMrD7xnYpQtaa8q
OHHz72ybGUe9USOKGTtGp+WxNS5j++zcQG7BKoRvRMJHEXomX6z+YeI2a1q19fk4
c/aeSvHGZuVU/xbDp1gIJADz1OtZFv8zzL38wGWoEPBBVxCpGezEk4O/m3Aleksx
rHmyDtcHuWQb0cuFMdwrqW37x+ier6SvAM6XNCTz2bqN0S6/3udTZmhHoFDCxzZ0
h2fZBtlJXEsR8y7MZ1nY6YewWNErbLK1rnkIe7lW7nTDjelwelknAtDTaJVpSQfp
3gUSnNUC5ZZ9F2eJxpCY
=UuDw
-END PGP SIGNATURE-
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Fast pick from a probability list

[Darren Cook]

> I have a probability table of all possible moves. What is the
> fastest way to pick with probability, possibly with reducing the
> quality of probability?!
> 
> I could not find any discussion on this on computer-go, but probably
> I missed it :(

I may have misunderstood the question, but there was some discussion on
how to pick from moves which have different weights. ... the one I found
is from May 2009, and it seems the online archives only go back to 2010!
So I've pasted together the thread, below.

Darren


Question by Isaac:

> I'm about to work on heavy playouts, and I'm not sure how to choose a
> move during the playout. I intend to have weights for various
> features. I thought about 3 versions:
> 
> 1. In a position, calculate all the weights and the total weight.
> Then, play one move i with the probability weight_i/total_weight.
> 
> 2. Select a move randomly. Calculate the weight of it, then squash
> that weight in the [0,1] range. Play that move with that
> "probability".
> 
> 3. Same as 2., but play that move if the "probability" is higher than
> a certain treshold.
> 
> Which one do you think works best? I'm looking forward to other
> ideas, too. :)

Álvaro replies:


You have the most control with option 1. You can implement this fast
by keeping the sum of the weights for each row and for the total
board. You then "roll" a number between 0 and total_weight, and
advance through the rows subtracting the probability of each row until
you would cross 0, then go along the row subtracting the probability
of each point, until you would cross zero. Pick the point where the
process ends.

I initially implemented a similar scheme using a binary tree, and I
think it was Rémi who told me about this method, which is simpler and
faster in practice.

You may have problems with floating-point precission doing this. The
easy solution is using integers for weights, but perhaps there are
ways to make the code robust while keeping the more natural
floating-point values.


Bill Spight also replied:

Keeping cumulative weights, as Alvaro suggested, is one way to go. You
can improve #1 by choosing a possible play randomly, and then making the
play with the probability weight/maximum_weight.

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] KGS bot tournaments - what are your opinions?

[Nick Wedd]

I am thinking of making some small changes to the way I run bot tournaments
on KGS.  If you have ever taken part in a KGS bot tournament, I would like
to hear your opinions on three things.


1.  Limit on processor power?

This is the main point on which I want your opinions.  The other two are
trivial.

Several people have suggested to me that these events would be fairer if
there were a limit on the computing power of the entrants. I would like to
do this, but I don't know how. I have little understanding of the
terminology, I don't know how *e.g.* multiple cores in one computer compare
with multiple computers on one network, and I don't know how to count a
graphics card.  *If* someone can find a way to specify an upper limit to
permitted power which is clear and easy to understand, and *if* most
entrants would favor imposing such a limit, I will discuss what it should
be, and apply it.  I am not able to check what entrants are really running
on, but I will trust people.


2. Zeroes in the "Annual Championship" table.

The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a
cell where a program competed but did not score, and a blank where it did
not compete (at least it should do, I sometimes get it wrong). I would
prefer to omit these zeroes, they seem a bit rude. Also there is no clear
distinction between competing and not competing - how should I treat a
program which crashes and disappears after two rounds, or one (like AyaMC
last Sunday) which plays in every round but is broken and has no chance of
winning?  I realise that the zeroes some convey information that may be of
interest.  Should I continue to use them, or just leave those cells blank?


3. Live crosstable

When I write up my reports, I include a crosstable, like the one near the
top of http://www.weddslist.com/kgs/past/116/index.html .  This is easy for
me, I run a script which reads the data from the KGS page (
http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and
builds the crosstable in html, which I copy into the tournament report. It
only works for Swiss (and maybe Round Robin) tournaments. It works while
the tournament is still running, though only between rounds.I could build a
current crosstable each round during a tournament if there is any demand
for it.

-- 
Nick Wedd  mapr...@gmail.com
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Fast pick from a probability list

[Olivier Teytaud]

In case the question is more on the computational part, you might
use a binary tree, so that you do the selection in time O(log(number of
moves))
instead of O(number of moves). The update is also in logarithmic time for
some probability update rules (to be discussed, depends on how you modify
your probabilities - well, if you modify them :-) ).
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Limit on processor power?

[djhbrown .]

John McCarthy advocated a level playing-field like one-design sailboat
races for AI competitions.  having competed in some of those (sailboat
races, that is), i can attest that even in one-design formulas, the field
is not entirely level, as flukey winds and the occasional freighter going
through the field can cause upsets of all kinds.

if you want a truly level playing field, you would have to insist on same
programming language as well as same flops and bytes.  And same internet
connection speed and and and

as a human player, i do not find it to be unfair that more often than not
my opponents have greater Go processor power than me, whether because they
are basically smarter or have had more/better training/teachers or whatever.

one of the nice things about Go, which it shares with golf but with no
other competitive recreations that i can think of offhand, is that
differences in playing strengths can be compensated for by handicapping so
that both sides can enjoy pitting themselves against the golf course rather
than the other player per se.  i used to enjoy playing tennis with a fellow
much stronger than me, by introducing a form of handicapping; for every
match won, the winner would have to start with one point fewer.  He would
start a game at -40, thus needing to win 8 points to get to deuce to my 4.

my recommendation would be to not try to make rules, which would only open
a Pandora's box of argumentation - and competitively-minded people are the
most argumentative of all ! -  but instead to introduce handicaps based on
current kgs ratings.

the issue of drugs in athletics was once discussed by the panellists of
"Mock The Week";  Frankie Boyle suggested that there could be two kinds of
Olympics, one without any rules at all (like the "America's Cup" races have
settled on) so we could all enjoy seeing the spectacle of someone with
giraffe-leg extensions run the 100 metres in 0.3 seconds instead of 9 and a
bit (i've made up the scene as i can't remember exactly the hilarious
scenario Frankie used to illustrate his great idea).

i think it rather boils down to whether you see computer Go tournaments as
"mine's bigger than yours" facedowns of the programs, or of the programmers.
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Erik van der Werf]

Hi Nick,

Some kind of limit on processing power would be interesting. To me it
seems clear that a program like Zen benefits a lot by using more
processing power than it's close competitors.

A measure that I find reasonable is a limit on number of threads x
clock frequency. E.g., a program running on 64 intel cores, with 2
threads per core, at 3 Ghz would be counted as using 64x2x3 = 384 GHz,
and a program running on 24 amd cores, with 1 thread per core, at 2.6
Ghz would be counted as 62.4 GHz. As long as the top just uses x86
type processors this should work reasonably well. For GPU's or ARM
processors there probably needs to be a scaling factor.

Another option could be to limit the power supply, e.g., you may not
use more than say 400 Watt.

BR,
Erik


PS As someone that participates only occasionally I like the zeros,
but perhaps there is a friendlier way to indicate
participation/absence.

PS2 Crazy idea?: "Machine A plays black, machine B plays white. First
player proposes the komi, the second player chooses the
color"


On Wed, Oct 7, 2015 at 12:52 PM, Detlef Schmicker  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hi Nick,
>
> 3. would be great, I am very often going through round results to see
> where CS or abakus has lost its games:)
>
> 1. I do not see a way to do this but running on same hardware (e.g.
> Amazon EC2 with graphic card). Even this is unfair, as programs might
> be optimized to other configurations (cluster)
>
> Detlef
>
> Am 07.10.2015 um 12:27 schrieb Nick Wedd:
>> I am thinking of making some small changes to the way I run bot
>> tournaments on KGS.  If you have ever taken part in a KGS bot
>> tournament, I would like to hear your opinions on three things.
>>
>>
>> 1.  Limit on processor power?
>>
>> This is the main point on which I want your opinions.  The other
>> two are trivial.
>>
>> Several people have suggested to me that these events would be
>> fairer if there were a limit on the computing power of the
>> entrants. I would like to do this, but I don't know how. I have
>> little understanding of the terminology, I don't know how *e.g.*
>> multiple cores in one computer compare with multiple computers on
>> one network, and I don't know how to count a graphics card.  *If*
>> someone can find a way to specify an upper limit to permitted power
>> which is clear and easy to understand, and *if* most entrants would
>> favor imposing such a limit, I will discuss what it should be, and
>> apply it.  I am not able to check what entrants are really running
>> on, but I will trust people.
>>
>>
>> 2. Zeroes in the "Annual Championship" table.
>>
>> The table at http://www.weddslist.com/kgs/annual/index.html has a 0
>> in a cell where a program competed but did not score, and a blank
>> where it did not compete (at least it should do, I sometimes get it
>> wrong). I would prefer to omit these zeroes, they seem a bit rude.
>> Also there is no clear distinction between competing and not
>> competing - how should I treat a program which crashes and
>> disappears after two rounds, or one (like AyaMC last Sunday) which
>> plays in every round but is broken and has no chance of winning?  I
>> realise that the zeroes some convey information that may be of
>> interest.  Should I continue to use them, or just leave those cells
>> blank?
>>
>>
>> 3. Live crosstable
>>
>> When I write up my reports, I include a crosstable, like the one
>> near the top of http://www.weddslist.com/kgs/past/116/index.html .
>> This is easy for me, I run a script which reads the data from the
>> KGS page ( http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in
>> this case) and builds the crosstable in html, which I copy into the
>> tournament report. It only works for Swiss (and maybe Round Robin)
>> tournaments. It works while the tournament is still running, though
>> only between rounds.I could build a current crosstable each round
>> during a tournament if there is any demand for it.
>>
>>
>>
>> ___ Computer-go mailing
>> list Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iQIcBAEBAgAGBQJWFPljAAoJEInWdHg+Znf4pMwQAI/4c2HhXA1DgAQEcOpulRAg
> SSrDGCLyAypTQb3p1YDx45bvIIE9r0qBL+Ze+GRO/gD9peYmbayQWO7pQiL+2A/i
> 2NSkr5gj6SR2JKW924Ba7NnBBIOil+bVX4Jku8PZJkz7MWwsP7LxFKhX2hA81Iis
> EFdLAJ74atr7LSbERuhDYpwn/FcS9ag2h9k1zcwXD6R/OSuB46+OyR6dWw5NCrAL
> FhjHo22rXig741ZTHtAxx7VXRwMdn6RV2oqkMbajFa7CLHFTcLRLMv3ix2TFyyd9
> JXwYKzJLxCeUNBWQ8WI5wvEqf7BPRjZcg0jujfR29zpg0AEooglrnnyIwRee9DPy
> BGAxR8FmW/5kC1tygvM2c+shxvdhGrIB+1f8UoKIMp/IdhSLGuHc3Dq077+jCjG/
> QYA766C9tg+mqEPRp3nzqTP4G6cdTGlPfPLxMGGz6r1ltdlUwcqAy+Q/x6vztRJz
> HH7ThXZgNpruhKqoDkltxW6udGpdeUiRW5u2JDCHVWPI6S+AjhdbtpI4EG+7Awwq
> GUqk2LFlh3PItxl3UkYBpfevdHnUKXXG5UDxODEIQx43QgyYqLRjMsMXfaJGCbce
> TBsJ9CvvyadAN2JhOzeIVRVZoWbzyk3t/+Pkyg4erhyi517jhmaXJApBQiKl1lOL
> EqL1TfDx1ZJm+Ow0y18D
> 

Re: [Computer-go] KGS Tournament Registration

[Gonçalo Mendes Ferreira]

This is what happens when you reuse an e-mail from someone instead of 
writing a new one...


On 10/07/2015 01:40 PM, Gonçalo Mendes Ferreira wrote:

Hi, I'd like to register my program to enter the November computer Go
tournament.

Bot name in KGS: matilda
Bot real name: matilda
Authors: Gonçalo Mendes Ferreira (gonmf at KGS)
Processor power: CPU Intel(R) Core(TM) i7 3930K @ 3.20GHz, RAM: G.Skill
DDR3 4x8GB @ 1.6GHz, doesn't use the GPU

Thank you,
Gonçalo
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] KGS Tournament Registration

[Gonçalo Mendes Ferreira]

Hi, I'd like to register my program to enter the November computer Go 
tournament.


Bot name in KGS: matilda
Bot real name: matilda
Authors: Gonçalo Mendes Ferreira (gonmf at KGS)
Processor power: CPU Intel(R) Core(TM) i7 3930K @ 3.20GHz, RAM: G.Skill 
DDR3 4x8GB @ 1.6GHz, doesn't use the GPU


Thank you,
Gonçalo
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Stefan Kaitschick]

>
> 1. I do not see a way to do this but running on same hardware (e.g.
> Amazon EC2 with graphic card). Even this is unfair, as programs might
> be optimized to other configurations (cluster)
>
>
 First, there is the question is fairness is even desirable.
But also, as you say, it is really impossible to make an event "fair", even
with identical hardware, because this will give the reverse advantage to
programs that put a premium on resource conservation in their design. And
with the new CNN developments, it would be a shame to put resource limits
to the top of the agenda.

As a Kibitz, I would have an added trivial wish: force a minimum time
consumption on the first moves of 9*9 games. It's annoying as a spectator
to have the first 8 moves or so just spit out on the board, forcing you to
go back to see what happened there. It would also slightly lessen the
advantage of bots with books.

Stefan
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Erik van der Werf]

On Wed, Oct 7, 2015 at 5:02 PM, Hideki Kato  wrote:
> Erik,
>
> Erik van der Werf: 
> 

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

I'd like to explain my idea more.

The monthly tournaments (and current annual championship (FL class)) are 
not necessary to change.   Just creating one more championship class for 
desktop computers (PC class).  

An entrant has to state so if he/she wants to belong PC class (i.e., all 
entrants automatically belong FL class).  One program can entry both 
classes.

Hideki

Goncalo Mendes Ferreira: <561535a9.7010...@sapo.pt>:
>I think this is a good compromise. Monthly tournaments free for everyone 

>and maybe an yearly one segregated by hardware. Having segregated 

>monthly tournaments would be a bit taxing on the organization and people 

>who would submit their programs for all hardware divisions. Segregation 

>based on consumer grade desktops vs others is simple enough.

>

>Since there is no way to confirm the hardware and KGS tournament results 

>cannot be used to measure program strength in a reliable way, I'd prefer 

>the current theme of casual competitive tournaments.

>

>Gonçalo

>

>On 07/10/2015 15:56, Hideki Kato wrote:

>> Nick & all,

>>

>> Another direction for the hardware.  How about introducing two classes

>> for the Annual Championship?  I.e., no-limit (formula libre) class

>> and personal computer one.  My proposal for the later is very simple;

>> one desktop (i.e., non-server) cpu and one video card.

>>

>> Hideki

>

>___

>Computer-go mailing list

>Computer-go@computer-go.org

>http://computer-go.org/mailman/listinfo/computer-go
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

Nick & all,

1. Although introducing some limitation of cpu power is an intersting 
idea (actually my GPW Cup does), I think it's too early for KGS bot 
tournaments.  

How to utilize computer clusters' power for planning tasks is a common 
and important reseach theme now.  As communication over network is less 
effective than in-box one, playing-strength per (total) cpu-power get 
smaller when using computer clusters.  Zen's root parallelization 
improves up-to 1 or 2 ranks but TDS based algorithms (used for Gommora 
and MP-Fuego) are expected to give more improvements.  Preventing such 
effort must be a bad idea, I strongly believe.  So, at least, a simple 
sum of cpu power of all node computers is not acceptable.  (Some 
discouting could be?)

Cpu power of each SMP or NUMA box can be computed by 
number-of-(physical-)cores times clock-frequency (although Erik used 
logical-cores).  Using number-of-threads instead might be better.  For 
more fairness, some factors can be defined for processor arichitectures 
or manufacturers, because some participants have to use non-Intel 
processors due to their environments.  

For GPUs, I have no concrete idea now.  Simple flops is not enough and 
more discussion is necessary, I believe.

Another idea, you (or we?) can define some benchmark program(s).

2. I don't understand this at all.  It's just a record of fact.  
Intentional omitting of information must be a bad idea.

3. Watching the crosstable in real-time should be a fan for all 
observers.

Hideki

Nick Wedd: :
>I am thinking of making some small changes to the way I run bot tournaments
>on KGS.  If you have ever taken part in a KGS bot tournament, I would like
>to hear your opinions on three things.
>
>
>1.  Limit on processor power?
>
>This is the main point on which I want your opinions.  The other two are
>trivial.
>
>Several people have suggested to me that these events would be fairer if
>there were a limit on the computing power of the entrants. I would like to
>do this, but I don't know how. I have little understanding of the
>terminology, I don't know how *e.g.* multiple cores in one computer compare
>with multiple computers on one network, and I don't know how to count a
>graphics card.  *If* someone can find a way to specify an upper limit to
>permitted power which is clear and easy to understand, and *if* most
>entrants would favor imposing such a limit, I will discuss what it should
>be, and apply it.  I am not able to check what entrants are really running
>on, but I will trust people.
>
>
>2. Zeroes in the "Annual Championship" table.
>
>The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a
>cell where a program competed but did not score, and a blank where it did
>not compete (at least it should do, I sometimes get it wrong). I would
>prefer to omit these zeroes, they seem a bit rude. Also there is no clear
>distinction between competing and not competing - how should I treat a
>program which crashes and disappears after two rounds, or one (like AyaMC
>last Sunday) which plays in every round but is broken and has no chance of
>winning?  I realise that the zeroes some convey information that may be of
>interest.  Should I continue to use them, or just leave those cells blank?
>
>
>3. Live crosstable
>
>When I write up my reports, I include a crosstable, like the one near the
>top of http://www.weddslist.com/kgs/past/116/index.html .  This is easy for
>me, I run a script which reads the data from the KGS page (
>http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and
>builds the crosstable in html, which I copy into the tournament report. It
>only works for Swiss (and maybe Round Robin) tournaments. It works while
>the tournament is still running, though only between rounds.I could build a
>current crosstable each round during a tournament if there is any demand
>for it.
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

Nick & all,

Another direction for the hardware.  How about introducing two classes 
for the Annual Championship?  I.e., no-limit (formula libre) class 
and personal computer one.  My proposal for the later is very simple; 
one desktop (i.e., non-server) cpu and one video card.

Hideki

Nick Wedd: :
>I am thinking of making some small changes to the way I run bot tournaments
>on KGS.  If you have ever taken part in a KGS bot tournament, I would like
>to hear your opinions on three things.
>
>
>1.  Limit on processor power?
>
>This is the main point on which I want your opinions.  The other two are
>trivial.
>
>Several people have suggested to me that these events would be fairer if
>there were a limit on the computing power of the entrants. I would like to
>do this, but I don't know how. I have little understanding of the
>terminology, I don't know how *e.g.* multiple cores in one computer compare
>with multiple computers on one network, and I don't know how to count a
>graphics card.  *If* someone can find a way to specify an upper limit to
>permitted power which is clear and easy to understand, and *if* most
>entrants would favor imposing such a limit, I will discuss what it should
>be, and apply it.  I am not able to check what entrants are really running
>on, but I will trust people.
>
>
>2. Zeroes in the "Annual Championship" table.
>
>The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a
>cell where a program competed but did not score, and a blank where it did
>not compete (at least it should do, I sometimes get it wrong). I would
>prefer to omit these zeroes, they seem a bit rude. Also there is no clear
>distinction between competing and not competing - how should I treat a
>program which crashes and disappears after two rounds, or one (like AyaMC
>last Sunday) which plays in every round but is broken and has no chance of
>winning?  I realise that the zeroes some convey information that may be of
>interest.  Should I continue to use them, or just leave those cells blank?
>
>
>3. Live crosstable
>
>When I write up my reports, I include a crosstable, like the one near the
>top of http://www.weddslist.com/kgs/past/116/index.html .  This is easy for
>me, I run a script which reads the data from the KGS page (
>http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and
>builds the crosstable in html, which I copy into the tournament report. It
>only works for Swiss (and maybe Round Robin) tournaments. It works while
>the tournament is still running, though only between rounds.I could build a
>current crosstable each round during a tournament if there is any demand
>for it.
-- 
Hideki Kato 
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

Erik,

Erik van der Werf: 

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Gonçalo Mendes Ferreira]

I think this is a good compromise. Monthly tournaments free for everyone 
and maybe an yearly one segregated by hardware. Having segregated 
monthly tournaments would be a bit taxing on the organization and people 
who would submit their programs for all hardware divisions. Segregation 
based on consumer grade desktops vs others is simple enough.


Since there is no way to confirm the hardware and KGS tournament results 
cannot be used to measure program strength in a reliable way, I'd prefer 
the current theme of casual competitive tournaments.


Gonçalo

On 07/10/2015 15:56, Hideki Kato wrote:

Nick & all,

Another direction for the hardware.  How about introducing two classes
for the Annual Championship?  I.e., no-limit (formula libre) class
and personal computer one.  My proposal for the later is very simple;
one desktop (i.e., non-server) cpu and one video card.

Hideki


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Hideki Kato]

Erik van der Werf: 
:
>On Wed, Oct 7, 2015 at 5:02 PM, Hideki Kato  wrote:
>> Erik,
>>
>> Erik van der Werf: 
>> 

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Erik van der Werf]

Although I agree on the research argument (setting no limits
encourages work on massive parallel distributed architectures), I do
find it a bit funny to see this argument coming from team Zen. As far
as I know team Zen does not publish their research findings (or did I
miss some papers?).

Erik


On Wed, Oct 7, 2015 at 4:20 PM, Hideki Kato  wrote:
> Nick & all,
>
> 1. Although introducing some limitation of cpu power is an intersting
> idea (actually my GPW Cup does), I think it's too early for KGS bot
> tournaments.
>
> How to utilize computer clusters' power for planning tasks is a common
> and important reseach theme now.  As communication over network is less
> effective than in-box one, playing-strength per (total) cpu-power get
> smaller when using computer clusters.  Zen's root parallelization
> improves up-to 1 or 2 ranks but TDS based algorithms (used for Gommora
> and MP-Fuego) are expected to give more improvements.  Preventing such
> effort must be a bad idea, I strongly believe.  So, at least, a simple
> sum of cpu power of all node computers is not acceptable.  (Some
> discouting could be?)
>
> Cpu power of each SMP or NUMA box can be computed by
> number-of-(physical-)cores times clock-frequency (although Erik used
> logical-cores).  Using number-of-threads instead might be better.  For
> more fairness, some factors can be defined for processor arichitectures
> or manufacturers, because some participants have to use non-Intel
> processors due to their environments.
>
> For GPUs, I have no concrete idea now.  Simple flops is not enough and
> more discussion is necessary, I believe.
>
> Another idea, you (or we?) can define some benchmark program(s).
>
> 2. I don't understand this at all.  It's just a record of fact.
> Intentional omitting of information must be a bad idea.
>
> 3. Watching the crosstable in real-time should be a fan for all
> observers.
>
> Hideki
>
> Nick Wedd: 
> :
>>I am thinking of making some small changes to the way I run bot tournaments
>>on KGS.  If you have ever taken part in a KGS bot tournament, I would like
>>to hear your opinions on three things.
>>
>>
>>1.  Limit on processor power?
>>
>>This is the main point on which I want your opinions.  The other two are
>>trivial.
>>
>>Several people have suggested to me that these events would be fairer if
>>there were a limit on the computing power of the entrants. I would like to
>>do this, but I don't know how. I have little understanding of the
>>terminology, I don't know how *e.g.* multiple cores in one computer compare
>>with multiple computers on one network, and I don't know how to count a
>>graphics card.  *If* someone can find a way to specify an upper limit to
>>permitted power which is clear and easy to understand, and *if* most
>>entrants would favor imposing such a limit, I will discuss what it should
>>be, and apply it.  I am not able to check what entrants are really running
>>on, but I will trust people.
>>
>>
>>2. Zeroes in the "Annual Championship" table.
>>
>>The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a
>>cell where a program competed but did not score, and a blank where it did
>>not compete (at least it should do, I sometimes get it wrong). I would
>>prefer to omit these zeroes, they seem a bit rude. Also there is no clear
>>distinction between competing and not competing - how should I treat a
>>program which crashes and disappears after two rounds, or one (like AyaMC
>>last Sunday) which plays in every round but is broken and has no chance of
>>winning?  I realise that the zeroes some convey information that may be of
>>interest.  Should I continue to use them, or just leave those cells blank?
>>
>>
>>3. Live crosstable
>>
>>When I write up my reports, I include a crosstable, like the one near the
>>top of http://www.weddslist.com/kgs/past/116/index.html .  This is easy for
>>me, I run a script which reads the data from the KGS page (
>>http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and
>>builds the crosstable in html, which I copy into the tournament report. It
>>only works for Swiss (and maybe Round Robin) tournaments. It works while
>>the tournament is still running, though only between rounds.I could build a
>>current crosstable each round during a tournament if there is any demand
>>for it.
> --
> Hideki Kato 
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Gonçalo Mendes Ferreira]

I think cluster renting is a little more complex than Rémi makes it 
seem, because behind the few hours of tournament play will be many more 
hours of testing. There are also other reasons why programs may only 
target personal computers, for instance if they're commercial for 
personal use.


If we're thinking of categories, we could adopt an honor system like:
Category A0 - If your computing power is similar to that of a Raspberry Pi
and so on, instead of actually specifying how "computing power" can be 
compared across GPUs/clusters. I'd still prefer very little categories 
(2 or 3) though.



So finally, my actual proposal, is that programs with advanced hardware
configurations handicap themselves.  For example, Zen could unilaterally
announce a lower time limit that it intends to adhere to.


As long as this isn't enforced, I think it's great if a program 
handicaps itself and still wins, it adds excitement to the tournaments.


Gonçalo
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Rémi Coulom]

Hi Nick,

I don’t care much about having a limit on processing power. I’d be happy either 
way.

Cloud computing platforms like Amazon EC2 allows to rent powerful servers at a 
low price. The machine I used for the tournament cost me 0.3$/hour or so. So 
the argument that only rich or academic people can get powerful hardware is not 
good. A cluster of 8 such machines would still be quite cheap. And making an 
efficient distributed search algorithm is an interesting and challenging 
technical problem. So I feel it is interesting to allow big clusters.

Thanks for organizing the KGS tournaments, by the way.

Rémi


On 7 oct. 2015, at 12:27, Nick Wedd  wrote:

> I am thinking of making some small changes to the way I run bot tournaments 
> on KGS.  If you have ever taken part in a KGS bot tournament, I would like to 
> hear your opinions on three things.
> 
> 
> 1.  Limit on processor power?
> 
> This is the main point on which I want your opinions.  The other two are 
> trivial.
> 
> Several people have suggested to me that these events would be fairer if 
> there were a limit on the computing power of the entrants. I would like to do 
> this, but I don't know how. I have little understanding of the terminology, I 
> don't know how e.g. multiple cores in one computer compare with multiple 
> computers on one network, and I don't know how to count a graphics card.  If 
> someone can find a way to specify an upper limit to permitted power which is 
> clear and easy to understand, and if most entrants would favor imposing such 
> a limit, I will discuss what it should be, and apply it.  I am not able to 
> check what entrants are really running on, but I will trust people.
> 
> 
> 2. Zeroes in the "Annual Championship" table.
> 
> The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a cell 
> where a program competed but did not score, and a blank where it did not 
> compete (at least it should do, I sometimes get it wrong). I would prefer to 
> omit these zeroes, they seem a bit rude. Also there is no clear distinction 
> between competing and not competing - how should I treat a program which 
> crashes and disappears after two rounds, or one (like AyaMC last Sunday) 
> which plays in every round but is broken and has no chance of winning?  I 
> realise that the zeroes some convey information that may be of interest.  
> Should I continue to use them, or just leave those cells blank?
> 
> 
> 3. Live crosstable
> 
> When I write up my reports, I include a crosstable, like the one near the top 
> of http://www.weddslist.com/kgs/past/116/index.html .  This is easy for me, I 
> run a script which reads the data from the KGS page 
> (http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and 
> builds the crosstable in html, which I copy into the tournament report. It 
> only works for Swiss (and maybe Round Robin) tournaments. It works while the 
> tournament is still running, though only between rounds.I could build a 
> current crosstable each round during a tournament if there is any demand for 
> it.
> 
> -- 
> Nick Wedd  mapr...@gmail.com
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Rémi Coulom]

Hi Nick, 


I don’t care much about having a limit on processing power. I’d be happy either 
way. 


Cloud computing platforms like Amazon EC2 allows to rent powerful servers at a 
low price. The machine I used for the tournament cost me 0.3$/hour or so. So 
the argument that only rich or academic people can get powerful hardware is not 
good. A cluster of 8 such machines would still be quite cheap. And making an 
efficient distributed search algorithm is an interesting and challenging 
technical problem. So I feel it is interesting to allow big clusters. 

I can beat them on a single machine anyway ;-) 



Thanks for organizing the KGS tournaments, by the way. 


Rémi 





On 7 oct. 2015, at 12:27, Nick Wedd < mapr...@gmail.com > wrote: 



I am thinking of making some small changes to the way I run bot tournaments on 
KGS. If you have ever taken part in a KGS bot tournament, I would like to hear 
your opinions on three things. 




1. Limit on processor power? 


This is the main point on which I want your opinions. The other two are 
trivial. 


Several people have suggested to me that these events would be fairer if there 
were a limit on the computing power of the entrants. I would like to do this, 
but I don't know how. I have little understanding of the terminology, I don't 
know how e.g. multiple cores in one computer compare with multiple computers on 
one network, and I don't know how to count a graphics card. If someone can find 
a way to specify an upper limit to permitted power which is clear and easy to 
understand, and if most entrants would favor imposing such a limit, I will 
discuss what it should be, and apply it. I am not able to check what entrants 
are really running on, but I will trust people. 




2. Zeroes in the "Annual Championship" table. 


The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a cell 
where a program competed but did not score, and a blank where it did not 
compete (at least it should do, I sometimes get it wrong). I would prefer to 
omit these zeroes, they seem a bit rude. Also there is no clear distinction 
between competing and not competing - how should I treat a program which 
crashes and disappears after two rounds, or one (like AyaMC last Sunday) which 
plays in every round but is broken and has no chance of winning? I realise that 
the zeroes some convey information that may be of interest. Should I continue 
to use them, or just leave those cells blank? 




3. Live crosstable 


When I write up my reports, I include a crosstable, like the one near the top 
of http://www.weddslist.com/kgs/past/116/index.html . This is easy for me, I 
run a script which reads the data from the KGS page ( 
http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and builds 
the crosstable in html, which I copy into the tournament report. It only works 
for Swiss (and maybe Round Robin) tournaments. It works while the tournament is 
still running, though only between rounds.I could build a current crosstable 
each round during a tournament if there is any demand for it. 


-- 

Nick Wedd mapr...@gmail.com ___ 
Computer-go mailing list 
Computer-go@computer-go.org 
http://computer-go.org/mailman/listinfo/computer-go 


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Dave Dyer]

How about handicapping the hardware based on time.  Programs running
on more powerful hardware would get less time.  

On the other hand, improving the software includes making use of more
powerful hardware.  Handicapping (or banning) powerful hardware would
discourage that.

On third hand, development teams get the most information out of competitions
if the race is close, so it's in their interest to not crush competitors.

So finally, my actual proposal, is that programs with advanced hardware
configurations handicap themselves.  For example, Zen could unilaterally
announce a lower time limit that it intends to adhere to.

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] KGS bot tournaments - what are your opinions?

[Sebastian Scheib]

I know, this is a lot of work, but what about "caegories" ?

2015-10-07 14:06 GMT-03:00 Rémi Coulom :

> Hi Nick,
>
> I don’t care much about having a limit on processing power. I’d be happy
> either way.
>
> Cloud computing platforms like Amazon EC2 allows to rent powerful servers
> at a low price. The machine I used for the tournament cost me 0.3$/hour or
> so. So the argument that only rich or academic people can get powerful
> hardware is not good. A cluster of 8 such machines would still be quite
> cheap. And making an efficient distributed search algorithm is an
> interesting and challenging technical problem. So I feel it is interesting
> to allow big clusters.
>
> I can beat them on a single machine anyway ;-)
>
> Thanks for organizing the KGS tournaments, by the way.
>
> Rémi
>
>
> On 7 oct. 2015, at 12:27, Nick Wedd  wrote:
>
> I am thinking of making some small changes to the way I run bot
> tournaments on KGS.  If you have ever taken part in a KGS bot tournament, I
> would like to hear your opinions on three things.
>
>
> 1.  Limit on processor power?
>
> This is the main point on which I want your opinions.  The other two are
> trivial.
>
> Several people have suggested to me that these events would be fairer if
> there were a limit on the computing power of the entrants. I would like to
> do this, but I don't know how. I have little understanding of the
> terminology, I don't know how *e.g.* multiple cores in one computer
> compare with multiple computers on one network, and I don't know how to
> count a graphics card.  *If* someone can find a way to specify an upper
> limit to permitted power which is clear and easy to understand, and *if* most
> entrants would favor imposing such a limit, I will discuss what it should
> be, and apply it.  I am not able to check what entrants are really running
> on, but I will trust people.
>
>
> 2. Zeroes in the "Annual Championship" table.
>
> The table at http://www.weddslist.com/kgs/annual/index.html has a 0 in a
> cell where a program competed but did not score, and a blank where it did
> not compete (at least it should do, I sometimes get it wrong). I would
> prefer to omit these zeroes, they seem a bit rude. Also there is no clear
> distinction between competing and not competing - how should I treat a
> program which crashes and disappears after two rounds, or one (like AyaMC
> last Sunday) which plays in every round but is broken and has no chance of
> winning?  I realise that the zeroes some convey information that may be of
> interest.  Should I continue to use them, or just leave those cells blank?
>
>
> 3. Live crosstable
>
> When I write up my reports, I include a crosstable, like the one near the
> top of http://www.weddslist.com/kgs/past/116/index.html .  This is easy
> for me, I run a script which reads the data from the KGS page (
> http://www.gokgs.com/tournEntrants.jsp?sort=s=990 in this case) and
> builds the crosstable in html, which I copy into the tournament report. It
> only works for Swiss (and maybe Round Robin) tournaments. It works while
> the tournament is still running, though only between rounds.I could build a
> current crosstable each round during a tournament if there is any demand
> for it.
>
> --
> Nick Wedd  mapr...@gmail.com
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
>
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>



-- 
Dracux
*http://www.dracux.com *
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go