[computer-go] CGOS 19x19 anchor
Hi, CGOS 19x19 has stopped for a while. Is server down? If there is no enough anchor resource, I can run anchor (gnugo). Regards, Hiroshi Yamashita ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] CGOS 19x19 anchor
I restarted the 19x19 server. Speaking of anchors ... For the new server I'm thinking about making some specified version of fuego the anchor on all venues. It seems to qualify as it has the following characteristics: 1. open source. 2. high strength to cpu resource utilization ratio I have not looked into the memory requirements yet - but presumably that can either be configured, compiled in, or is not a problem even now.It needs to be low memory and low cpu in order that it can be run unobtrusively, perhaps even on the server itself. Now it's a matter of determining which version of fuego to use. I need to know: 1. the relative strengths of each version. 2. resource utilization of each version. 3. stability of each version - bugs. To keep things simple, I will use fuego on all venues. The number of playouts will probably be in proportion to the time control (or perhaps the number of points on the board.) - Don 2009/6/23 Hiroshi Yamashita y...@bd.mbn.or.jp Hi, CGOS 19x19 has stopped for a while. Is server down? If there is no enough anchor resource, I can run anchor (gnugo). Regards, Hiroshi Yamashita ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] CGOS 19x19 anchor
I restarted the 19x19 server. Thank you. I started my bot. I'm thinking about making some specified version of fuego I think using Fuego for anchor is good idea. One problem is maybe latest Fuego will be overrated from weak Fuego anchor. Regards, Hiroshi Yamashita ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] CGOS 19x19 anchor
On Tue, Jun 23, 2009 at 10:10 AM, Hiroshi Yamashita y...@bd.mbn.or.jpwrote: I restarted the 19x19 server. Thank you. I started my bot. I'm thinking about making some specified version of fuego I think using Fuego for anchor is good idea. One problem is maybe latest Fuego will be overrated from weak Fuego anchor. Can you explain this? I don't understand what you are saying. - Don Regards, Hiroshi Yamashita ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] CGOS 19x19 anchor
Fuego gets an advantage because when it plays the anchor, it is playing a version of itself. That's bad for the same reason that it's bad to test against a version of your own program -- inflated results. But I don't think it's a big deal. What about using both Fuego and Mogo as anchors? Don't they both satisfy those requirements? Don Dailey wrote: On Tue, Jun 23, 2009 at 10:10 AM, Hiroshi Yamashita y...@bd.mbn.or.jp mailto:y...@bd.mbn.or.jp wrote: I restarted the 19x19 server. Thank you. I started my bot. I'm thinking about making some specified version of fuego I think using Fuego for anchor is good idea. One problem is maybe latest Fuego will be overrated from weak Fuego anchor. Can you explain this? I don't understand what you are saying. - Don Regards, Hiroshi Yamashita ___ computer-go mailing list computer-go@computer-go.org mailto:computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] CGOS 19x19 anchor
Can you explain this? I don't understand what you are saying. Once I ran both 1 core and 2 cores Aya on 19x19 CGOS, 2 cores Aya got high rating. But without 1 core Aya, 2 cores Aya could not get such a high rating. Remi also reported same phenomenon. [computer-go] CGOS Deflation or Self-Play delusion? http://computer-go.org/pipermail/computer-go/2008-February/013995.html Regards, Hiroshi Yamashita - Original Message - From: Don Dailey dailey@gmail.com To: computer-go computer-go@computer-go.org Sent: Tuesday, June 23, 2009 11:12 PM Subject: Re: [computer-go] CGOS 19x19 anchor On Tue, Jun 23, 2009 at 10:10 AM, Hiroshi Yamashita y...@bd.mbn.or.jpwrote: I restarted the 19x19 server. Thank you. I started my bot. I'm thinking about making some specified version of fuego I think using Fuego for anchor is good idea. One problem is maybe latest Fuego will be overrated from weak Fuego anchor. Can you explain this? I don't understand what you are saying. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] CGOS 19x19 anchor
On Tue, Jun 23, 2009 at 10:22 AM, Michael Williams michaelwilliam...@gmail.com wrote: Fuego gets an advantage because when it plays the anchor, it is playing a version of itself. That's bad for the same reason that it's bad to test against a version of your own program -- inflated results. But I don't think it's a big deal. What about using both Fuego and Mogo as anchors? Don't they both satisfy those requirements? Ok, I understand. I cannot be too anal about this since this would be an issue with any program I choose. I would prefer not to have multiple different anchors simply due to the extra complexity. I would have to calibrate both of them, find servers to run both of them on, and basically all the issue are doubled.Mogo is also a problem because it's not open source as far as I know. I want there to be this kind of transparency. I don't mind running multiple copies of identical anchors though. Anyway, we could decide later to do something like this, no need to decide now. - Don Don Dailey wrote: On Tue, Jun 23, 2009 at 10:10 AM, Hiroshi Yamashita y...@bd.mbn.or.jpmailto: y...@bd.mbn.or.jp wrote: I restarted the 19x19 server. Thank you. I started my bot. I'm thinking about making some specified version of fuego I think using Fuego for anchor is good idea. One problem is maybe latest Fuego will be overrated from weak Fuego anchor. Can you explain this? I don't understand what you are saying. - Don Regards, Hiroshi Yamashita ___ computer-go mailing list computer-go@computer-go.org mailto:computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] CGOS 19x19 anchor
Is there statistical proof that this is a major issue? I have not reviewed the reference to the forum post but I would like to say this: If you expect something to happen, you will notice it when it does even if what you think is happening really isn't.I'm not saying it didn't or doesn't happen, but caution is in order. To be sure that this really is what you think, you must play a huge number of games.You must also look at the head to head against the 2 versions in question.To illustrate the magnitude of the problem, there is about a 50/50 chance you will see this phenomenon to some degree even if it doesn't exist.Even to get the error bars under 10 ELO you have to play something like 3000 games between the 2 versions in question and then a similar number of games between other programs with BOTH versions. I have no doubt this is somewhat of an issue, even between MCTS programs in general but I doubt it's major (I could be wrong - depending on how you define major.) To quantify it you must play tens of thousands of games in order to nail this down to within 10 or 20 ELO.You could get by on less games if the problem is bigger of course. If this really is a problem I can minimize the impact of this - at the sacrifice of diversity. In other words, if I increase the diversity with server adjustments, then if you have a strong program, you will have to play weaker opponents more often. This will also make the ratings less stable which could cause people to have false observations (and I'm not claiming this is a false observation, but is there proof that it's major?) Anway, I provide a digest of all results (and the SGF games are available) in order for anyone who wishes to scrutinize the results and show that it's statistically improbable (which it might be.) - Don On Tue, Jun 23, 2009 at 10:30 AM, Hiroshi Yamashita y...@bd.mbn.or.jpwrote: Can you explain this? I don't understand what you are saying. Once I ran both 1 core and 2 cores Aya on 19x19 CGOS, 2 cores Aya got high rating. But without 1 core Aya, 2 cores Aya could not get such a high rating. Remi also reported same phenomenon. [computer-go] CGOS Deflation or Self-Play delusion? http://computer-go.org/pipermail/computer-go/2008-February/013995.html Regards, Hiroshi Yamashita - Original Message - From: Don Dailey dailey@gmail.com To: computer-go computer-go@computer-go.org Sent: Tuesday, June 23, 2009 11:12 PM Subject: Re: [computer-go] CGOS 19x19 anchor On Tue, Jun 23, 2009 at 10:10 AM, Hiroshi Yamashita y...@bd.mbn.or.jp wrote: I restarted the 19x19 server. Thank you. I started my bot. I'm thinking about making some specified version of fuego I think using Fuego for anchor is good idea. One problem is maybe latest Fuego will be overrated from weak Fuego anchor. Can you explain this? I don't understand what you are saying. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] CGOS 19x19 anchor
Don, One possibility would be to have two open-source anchors (fuego and gnugo?) and ensure that a full-strength version would never be paired with it's own limited-strength anchor version. Ben. From: Don Dailey dailey@gmail.com To: computer-go computer-go@computer-go.org Sent: Tuesday, June 23, 2009 10:10:09 AM Subject: Re: [computer-go] CGOS 19x19 anchor Is there statistical proof that this is a major issue? I have not reviewed the reference to the forum post but I would like to say this: If you expect something to happen, you will notice it when it does even if what you think is happening really isn't.I'm not saying it didn't or doesn't happen, but caution is in order. To be sure that this really is what you think, you must play a huge number of games.You must also look at the head to head against the 2 versions in question.To illustrate the magnitude of the problem, there is about a 50/50 chance you will see this phenomenon to some degree even if it doesn't exist.Even to get the error bars under 10 ELO you have to play something like 3000 games between the 2 versions in question and then a similar number of games between other programs with BOTH versions. I have no doubt this is somewhat of an issue, even between MCTS programs in general but I doubt it's major (I could be wrong - depending on how you define major.) To quantify it you must play tens of thousands of games in order to nail this down to within 10 or 20 ELO.You could get by on less games if the problem is bigger of course. If this really is a problem I can minimize the impact of this - at the sacrifice of diversity. In other words, if I increase the diversity with server adjustments, then if you have a strong program, you will have to play weaker opponents more often. This will also make the ratings less stable which could cause people to have false observations (and I'm not claiming this is a false observation, but is there proof that it's major?) Anway, I provide a digest of all results (and the SGF games are available) in order for anyone who wishes to scrutinize the results and show that it's statistically improbable (which it might be.) - Don On Tue, Jun 23, 2009 at 10:30 AM, Hiroshi Yamashita y...@bd.mbn.or.jp wrote: Can you explain this? I don't understand what you are saying. Once I ran both 1 core and 2 cores Aya on 19x19 CGOS, 2 cores Aya got high rating. But without 1 core Aya, 2 cores Aya could not get such a high rating. Remi also reported same phenomenon. [computer-go] CGOS Deflation or Self-Play delusion? http://computer-go.org/pipermail/computer-go/2008-February/013995.html Regards, Hiroshi Yamashita - Original Message - From: Don Dailey dailey@gmail.com To: computer-go computer-go@computer-go.org Sent: Tuesday, June 23, 2009 11:12 PM Subject: Re: [computer-go] CGOS 19x19 anchor On Tue, Jun 23, 2009 at 10:10 AM, Hiroshi Yamashita y...@bd.mbn.or.jpwrote: I restarted the 19x19 server. Thank you. I started my bot. I'm thinking about making some specified version of fuego I think using Fuego for anchor is good idea. One problem is maybe latest Fuego will be overrated from weak Fuego anchor. Can you explain this? I don't understand what you are saying. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] CGOS 19x19 anchor
The server has no way of knowing which programs are related. The user is free to choose any name and I don' t want to additional complexity to this. - Don 2009/6/23 Ben Shoemaker plan...@rocketmail.com Don, One possibility would be to have two open-source anchors (fuego and gnugo?) and ensure that a full-strength version would never be paired with it's own limited-strength anchor version. Ben. -- *From:* Don Dailey dailey@gmail.com *To:* computer-go computer-go@computer-go.org *Sent:* Tuesday, June 23, 2009 10:10:09 AM *Subject:* Re: [computer-go] CGOS 19x19 anchor Is there statistical proof that this is a major issue? I have not reviewed the reference to the forum post but I would like to say this: If you expect something to happen, you will notice it when it does even if what you think is happening really isn't.I'm not saying it didn't or doesn't happen, but caution is in order. To be sure that this really is what you think, you must play a huge number of games.You must also look at the head to head against the 2 versions in question.To illustrate the magnitude of the problem, there is about a 50/50 chance you will see this phenomenon to some degree even if it doesn't exist.Even to get the error bars under 10 ELO you have to play something like 3000 games between the 2 versions in question and then a similar number of games between other programs with BOTH versions. I have no doubt this is somewhat of an issue, even between MCTS programs in general but I doubt it's major (I could be wrong - depending on how you define major.) To quantify it you must play tens of thousands of games in order to nail this down to within 10 or 20 ELO.You could get by on less games if the problem is bigger of course. If this really is a problem I can minimize the impact of this - at the sacrifice of diversity. In other words, if I increase the diversity with server adjustments, then if you have a strong program, you will have to play weaker opponents more often. This will also make the ratings less stable which could cause people to have false observations (and I'm not claiming this is a false observation, but is there proof that it's major?) Anway, I provide a digest of all results (and the SGF games are available) in order for anyone who wishes to scrutinize the results and show that it's statistically improbable (which it might be.) - Don On Tue, Jun 23, 2009 at 10:30 AM, Hiroshi Yamashita y...@bd.mbn.or.jpwrote: Can you explain this? I don't understand what you are saying. Once I ran both 1 core and 2 cores Aya on 19x19 CGOS, 2 cores Aya got high rating. But without 1 core Aya, 2 cores Aya could not get such a high rating. Remi also reported same phenomenon. [computer-go] CGOS Deflation or Self-Play delusion? http://computer-go.org/pipermail/computer-go/2008-February/013995.html Regards, Hiroshi Yamashita - Original Message - From: Don Dailey dailey@gmail.com To: computer-go computer-go@computer-go.org Sent: Tuesday, June 23, 2009 11:12 PM Subject: Re: [computer-go] CGOS 19x19 anchor On Tue, Jun 23, 2009 at 10:10 AM, Hiroshi Yamashita y...@bd.mbn.or.jp wrote: I restarted the 19x19 server. Thank you. I started my bot. I'm thinking about making some specified version of fuego I think using Fuego for anchor is good idea. One problem is maybe latest Fuego will be overrated from weak Fuego anchor. Can you explain this? I don't understand what you are saying. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
