Thanks for your responses guys. I was aware of the limitation of "custom hardware" that such a tournament would have, but I discounted it because to the best of my (albeit entirely second-hand) knowledge, the computer go community hasn't really gone there yet. All of the programs I see discussed on the list are entirely software-based and in fact a significant number of the top ones are available for download and purchase by anyone with the right operating system, which tells me that they're not being terribly clever with the hardware. I would love for this to be wrong, either at the moment or in the future.
I don't think there's any risk to this becoming the "standard format for all go tournaments" but I think some people would like to see *at least one* that controls for this particular confounding variable, because I don't think any of the existing ones do. I would love to be wrong about this too :) Also, Steve, I think your perception of AWS might be outdated by now... there are certainly EC2 clusters that I would not describe as "low-weight PCs" by any definition. Their new "high-performance" instances have the following stats: 23 GB of memory 33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core “Nehalem” architecture) 1690 GB of instance storage 64-bit platform I/O Performance: Very High (10 Gigabit Ethernet) Note that one EC2 Compute Unit provides "the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor". And a single instance of this has 33.5 of them. And you can spin up as many as you want, and put them into a cluster. My point is, we can definitely provide the kind of circumstances you describe, for the programs that can take advantage of them. Again, I realize that this sort of thing is limiting the kind of creative solutions (like Deep Blue) that could happen theoretically at some point in the future... but honestly, how many of today's competitive programs would not be adaptable to this kind of a tournament? I can't think of any, and even if I could, there's plenty of 'conventional' programs that *do* fit into this model that would probably like to see how they'd perform on a perfectly even playing field. Cheers, Adrian On Tue, Sep 28, 2010 at 11:25 AM, steve uurtamo <[email protected]> wrote: > i agree with almost all of what don said. > > i'd like to point out that uniform, homogenous environments do not allow > for super tricky and super cool uses of technology to necessarily take > place. some of which might give what you'd describe as unfair massive > advantages, but which would still be interesting to see. > > i mean, honestly, if you can get access to a supercomputer, and still lose, > great. if you win, great. if you had to work for 4 months to code > specifically for that hardware and ended up winning by a mile, even better. > > the homogenous environment simulates what it's like to have a network of > low-weight pc's. this isn't the only kind of hardware out there, though. > there are massive memory machines, machines with ridiculously fast networks, > machines with tons of cpus, and lots of combinations of the above. some of > these simply cannot be simulated in a contest. > > s. > > > On Tue, Sep 28, 2010 at 9:52 AM, Don Dailey <[email protected]> wrote: > >> >> On Tue, Sep 28, 2010 at 3:04 AM, Adrian Petrescu <[email protected]>wrote: >> >>> I'm not much of a participant in the field of computer go, but I am an >>> avid observer, so it puzzles me when I see things like the recent 9x9 "World >>> Championships" being plagued by issues of operator error, hardware >>> malfunction, network outages, etc. Even when everything goes smoothly, it's >>> hard to take the results too seriously when some programs are running on a >>> 16-core dedicated machines, and others are running on the developer's >>> personal laptop. >>> >>> I propose a tournament be run (or an old one adapted) that uses Amazon >>> Web Services' EC2 as the hardware platform for the competitors. The >>> advantages are several: >>> >>> * Completely predictable and uniform hardware across each competitor. We >>> can even make several divisions at different levels of hardware performance >>> in order to reward those programs which "scale" better >>> * It is very easy to (even programatically) create instances on demand >>> and bring them down when the round is over, so that the costs of running >>> such a thing can be minimized. >>> * The chance of hardware or network failure is negligibly close to zero. >>> * The virtual machines are completely flexible, so almost any strange >>> combination of software dependencies can be accommodated >>> >>> It seems that such a thing would go a long way towards lending more >>> legitimacy and consistency to computer go competitions. If a lot of people >>> like the idea, I would be glad to help set up the infrastructure and assist >>> developers in creating an AMI to host their bots. >>> >>> I also suspect that since AWS often give grants and discounts to worthy >>> causes, they could be persuaded to donate the one or two days' worth of CPU >>> hours needed for such a tournament. >>> >> >> This is a can of worms you are opening. This is a very old issue going >> back decades in computer chess too. The primary issue is whether this is >> just a contest of programming talent or whether it's a contest to encourage >> the best possible artificial go player. It cannot help but be a >> programmer contest even if there are not restrictions. >> >> Another issue is that if you limit the hardware, you could effectively be >> shutting out some competitors. No everyone is comfortable with whatever >> uniform hardware is chosen. Someone will surely produce some go playing >> hardware, the equivalent of a "deep blue" but for GO and it would be tragic >> to shut out this possibility. >> >> This is probably a good thing however if it's not pushed too far and >> becomes the standard format for all go tournaments. There is nothing wrong >> with programming contests. >> >> >> >> >>> >>> Cheers, >>> Adrian >>> >>> _______________________________________________ >>> Computer-go mailing list >>> [email protected] >>> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go >>> >> >> >> _______________________________________________ >> Computer-go mailing list >> [email protected] >> http://dvandva.org/cgi-bin/mailman/listinfo/computer-go >> > > > _______________________________________________ > Computer-go mailing list > [email protected] > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go >
_______________________________________________ Computer-go mailing list [email protected] http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
