All sounds good to me =) My only concern was that filtering out
players with low numbers of games would affect the ratings, but I
think you said in a separate mail that all the *games* would be considered, but just
not all the bots displayed. This sounds ideal.
I should have the beta3 release of the lite 9x9 engine pack out
tomorrow. Have been alpha testing all week, just trying to tweak the
number of playouts Mogo uses to get maximum rating but still be able
to run on an old machine.
Thanks for implementing the comma suffix clone merge feature =)
Raffles
On 18/11/2010 20:37, Don Dailey wrote:
I think I agree with you on the comma bots, allow
anyone to make them.
Copying from bayeselo is tricky because if a bot has only
played one or two games it may not make sense to constantly copy
over the rating. So I should probably change to make the copy
happen only after computing the ratings. Bayeselo has the
desirable characteristic that just 1 or 2 wins or losses will
not get ridiculously high or low ratings because it starts with
some kind of a priori assumption that all bots are equal (I
think it does the equivalent of granting a pseudo draws to all
match-ups or something similar.)
Furthermore I think I will change the bayeselo page to only
display programs who have played at least 100 games.
The final issue is whether to copy ratings over for bots who
have played less than N games. It makes sense to do this since
these players do not have established ratings and we really want
them to get realistic ratings as quickly as possible. So I
could do this even if the bots are not comma bots.
Don
On Thu, Nov 18, 2010 at 3:19 PM, Brian
Sheppard <[email protected]>
wrote:
Sure, let anyone create a
comma bot. Users get faster testing
from having multiple bots, and the community gets
more rating stability.
Everyone wins.
I think that there should
be a minimum number of games for
copying Bayeselo, but I don't feel strongly about
any particular number. Maybe only
30 games for Bayeselo to be as stable as online
updating.
The "last 24 hours"
restriction doesn't seem to matter
much. If a bot hasn't played since the last copying,
then it has no more new
data, so the only changes are from second-order
effects.
Brian
There are now several comma
bots on the server. This
should have a good impact on bringing the ELO
ratings in line with the Bayeselo
page. I'm currently copying over their ratings
every 2 hours.
So now we have a couple of
questions to ask the group -
primarily for those actually using CGOS.
One possible use of comma
bots is for users to have control
over how their programs get rated. It was my
original intentions to
forbid general usage of this mechanism, but
it occurred to me
that it's possible to just open this up to
everyone - you create a comma
bot if you want the bayeselo rating to be copied
over (every 2 hours or so.)
This effectively gives your bot the most
accurate rating as it's
based on your exact performance over all the
games you have ever played.
It would also have a positive affect on the
rating pool accuracy.
Another possibility is to
discourage this and reserve it for
clone bots designed to add stability to the
rating system.
As a separate issue it is
possible to use this copy
mechanism for bots who have played less than 100
or 200 games.
Since that is probably a huge number of bots
over time it would be
restricted to bots that have recent
activity, perhaps within the last
24 hours.
Any thoughts on either of
these points?
_______________________________________________
Computer-go mailing list
[email protected]
http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
_______________________________________________
Computer-go mailing list
[email protected]
http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
|
_______________________________________________
Computer-go mailing list
[email protected]
http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
