(This is about the computer-computer tournament, not the Kim-MoGo
match.)
Results of the Computer Go tournament at the 2008 US Go Congress in
Portland, OR, USA can temporarily be found at:
http://svcs.cs.pdx.edu/cgo2008
I would like to thank: Hierarchical Systems Research Foundation for
providing the bulk of the $1250 prize and travel expense money (the
rest was donated anonymously); Bart Massey, Kathi Lee, and everyone at
PSU for providing a physical venue and helping with technical setup;
Bill Shubert and everyone at KGS for providing a virtual venue; and
all of the programmers and operators involved.
The tournament was a double round-robin tournament, so each program
played as W and B against each opponent. All games were 19x19, 45
minutes per side sudden death, Chinese rules, 7.5 komi. Here are the
results in traditional round-robin format:
1 2 3 4 5 6 7 | Total
-----------------------------------
1. GNU Go XX 11 01 11 11 11 11 | 11
2. Many Faces 00 XX 11 11 11 11 11 | 10
3. Leela 10 00 XX 11 11 11 11 | 9
4. House Bot 00 00 00 XX 11 01 11 | 5
5. First Go 00 00 00 00 XX 11 11 | 4
6. Orego 00 00 00 10 00 XX 11 | 3
7. Butter Bot 00 00 00 00 00 00 XX | 0
NOTABLE EVENTS:
The tournament was surprisingly smooth. Every game but one ended in
resignation.
SlugGo was not able to attend due to numerous problems encountered
while assembling a new hardware cluster. We decided to enter GNU Go
(3.7.10, level 12) instead, as GNU Go had only refrained from entering
due to the expected presence of SlugGo.
The version of ManyFaces used includes Monte Carlo search. In fact, we
believe GNU Go was the only non-MC program in the tournament.
In ManyFaces-Leela (that is, the game with ManyFaces as white and
Leela as black), David Fotland discovered that his T61 laptop was
unplugged, so ManyFaces was running at half speed to conserve power.
The laptop was properly plugged in mid-game and ManyFaces went on to
win the game anyway.
In Leela-GNU, KGS reported a win for GNU under Japanese rules, because
we had failed to set one of our KgsGtp configuration files to specify
Chinese rules. Since both programs believed that they were playing
under Chinese rules, we re-scored under Chinese rules and found the
game was a win for Leela (as Leela had reported).
In FirstGo-GNU, FirstGo played out a ladder.
In Leela-FirstGo, FirstGo played out a short ladder. It began to run
when caught in a second, much longer broken ladder; 13 moves were
played before Leela abandoned the chase.
In FirstGo-Orego, Orego resigned after 76 moves -- probably a bit
premature.
After the event, Fotland had hew new multithreaded version of
ManyFaces up and running. It played three games against GNU Go and won
all of them.
LESSONS FOR NEXT YEAR'S TOURNAMENT:
We definitely want to do this again at next year's Go Congress in
Washington, DC. Is there anyone in that area willing to direct the
tournament?
Double round-robin ran extremely smoothly. We were able complete our
games in less time than in a tournament with fixed rounds, because a
new game could start whenever one ended. In fact, we were able to run
some programs on multiple machines, thus completing some games in
parallel. If there are a lot of entrants, perhaps they could be
filtered by a Swiss tournament or the results of previous tournaments
before playing round-robin among the strong programs.
The TD's program (Orego) was ineligible to win to avoid any appearance
of conflict of interest. In the future, we would do what has been done
in the past: appoint a deputy director or committee to make any
decisions regarding the TD's program.
If the prize money is very large, it may be important to require that
source code be made available for inspection by the TD to avoid any
question of plagiarism.
We should do a better job of making rules explicit, e.g., all results
stand, even if it is discovered afterward that a program was running
with incorrect parameters. (We discovered that FirstGo was running
with the incorrect time setting; the result stood.)
It would be good to have some talks by the programmers involved.
The expense of travel is a serious impediment to attendance. Some of
our prize money was used to defray such costs. It would be better to
clarify the distribution of prize money in advance. Perhaps there
could be one pool of prize money for everyone (with more money for
programs that place higher, of course), and a second pool for those
programmers who travel, with appropriate consideration to prevent a
trivial program being entered just to collect travel money while the
"programmer" just attends the Congress.
Our intent is to encourage interesting interactions between
programmers. There were useful conversations in the room as the
tournament ran. It was great when Anders Kierulf dropped in even
though his program SmartGo was not entered.
Peter Drake
http://www.lclark.edu/~drake/
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
