Don, I forgot to mention one additional consideration. My top-level driver does check rules for suicide and superko, even though the engine may or may not. At the top-level, if the engine chooses a bad move, then the driver will use the next best move instead. (Repeat as necessary) So it will not lose by rules and (hopefully) the second best move move is still reasonable.
However, I am talking about the actual performance of my engine when doing random playouts for MC. I do know the ELO cost of detection, using a valid heuristic. The real ELO benefit of knowing the validity of moves is just a wild guess. Yet, I stand by my analysis. Michael Wing > I think you are off on the relative importance of superko and suicide > and it seems that your values are rather arbitrary - just made up. > > First of all, we are only talking about detection in the play-outs, not > in the tree search portion. > > In the play-outs, it is very important to avoid moves that are nearly > always horrible. This clearly includes suicide. I don't know why > you estimate that it is worth only 1 elo weakness. > > If you implement a program that doesn't understand superko, you will > occasionally lose a game due to this - but most of the time it won't be > an issue. Nevertheless, it happens often enough that it is probably > worth a few ELO points because your program will LOSE on CGOS if it > fails to realize that it is about to play superko. I am guessing that > this would amount to perhaps 20 ELO, I'm just guessing. > > HOWEVER, if your program simply avoids superko moves, without > understanding them, it probably subtracts almost nothing from your > rating. In monte carlo UCT you can STILL include positional superko > in the tree search and get 99% of the benefit and simply leave this out > of the random play-outs. Including PSK in the play-outs will have no > measurable impact on the quality of the play-outs. > > My conclusion is different than yours. If you leave PSK out of the > play-outs you lose NOTHING that is likely to be measurable. If you > let your program play suicide moves in the play-outs, I'm quite you > lose many ELO rating points (if speed isn't a consideration.) > > Of course speed IS a consideration too and that can change the > formula. In your program there is not question that you should detect > suicide and not play it, because this is only 1.5 percent for you. But > evidently some program benefit substantially (in speed) by accepting > suicide. > > - Don > > > > > > [EMAIL PROTECTED] wrote: > > We can use math to shed some light on the topic: > > > > * Assume that doubling the speed of a machine > > increases the rank of a program by 100 ELO, > > as Don has previously concluded. > > > > * Then we have the following table of approximate > > costs, which comes from the equation y = 100 * 2^x > > cost -> lost ELO > > ---------------- > > 1% -> 1.5 ELO > > 2% -> 3.0 ELO > > 3% -> 4.5 ELO > > 4% -> 6.0 ELO > > 5% -> 7.5 ELO > > 6% -> 9.0 ELO > > 10% -> 15.0 ELO > > > > * In my program (which implements undo), the cost of > > for suicide detection is around 1%, which means it > > would lose 1.5 ELO points. > > > > * If I wanted to know whether it was worth it, I would > > want to measure the ELO benefit by making better > > decisions concerning suicide. It is a small but > > real amount, probably at least 1 ELO (using my > > finger in the breeze). > > > > * Thus the issue of whether you detect suicide may > > be a complete wash. > > > > * On the other hand detecting superko costs more like > > 6% or so, which costs 9 or more ELO. So a benefit > > of 1 ELO for doing superko right may not be worth > > the cost. > > > > Conclusions > > > > * The effect of suicide detection is *very* small in > > the scheme of things, and is probably not worth > > arguing over. Superko is also small, but might be > > worth a tiny amount of effort. > > > > * Some kind of study to measuring the ELO cost of bad > > suicide and ko decisions would be useful. > > > > * I plan to detect both suicide and superko on principle, > > confident that it doesn't make much difference. > > > > Michael Wing > > > > Don Dailey <[EMAIL PROTECTED]> said: > > > > > >>> There are two reasons to consider suicide and its detection.. > >>> > >>> 1) Some rule sets allow suicide. In such a rule set a suicide can > >>> be the best move because it can be a huge ko threat. > >>> > >>> 2) As David Fotland has pointed out many times, when competing > >>> under rules that allow suicide, some programs will do one just to > >>> see if your program refuses to play when you detect its suicide. > >>> > >> But there are very few arguments for putting suicide in the play-outs. > >> You can still design your program to accept and even play suicide > >> without putting these moves in the play-outs. > >> > >> The play-outs are imperfect by nature - they try to take a statistical > >> sample of many possible ways the game might proceed. The path to > >> improve the quality of this statistical sample is to not play moves that > >> represent very UNLIKELY continuations. Adding these moves randomly to > >> the play-outs doesn't improve it's ability to statically measure the > >> likely outcome. > >> > >> For instance since is "legal" to resign, we could randomly include this > >> possibility in the play-outs, but it would not increase the resolving > >> power of the play-outs. > >> > >> Moving into 1 point eyes is also legal, but virtually all Monte Carlo > >> programs forbid this as it's well known to be incredibly stupid in the > >> vast majority of cases. But in some rare cases it is actually good - > >> but we still would not want to add it to our play-outs. > >> > >> Because of the 1 point eye rule, suicide in the play-outs probably isn't > >> THAT bad. You are probably only suiciding a group that is already > >> dead - but you are weakening the play-outs. It may be worth it if you > >> get enough speed in return. > >> > >> In my program I am always looking for an excuse to veto moves that are > >> obviously bad. If I had such an obvious class of position like > >> suicide, I would jump on the opportunity to remove them from the play- outs! > >> > >> - Don > >> > >> > >> > >>> Cheers, > >>> David > >>> > >>> > >>> > >>> On 16, Jan 2008, at 5:52 AM, Don Dailey wrote: > >>> > >>> > >>>> I think suicide is insane myself. But I think the reason programs > >>>> might use it is only for a speedup - it's faster with some > >>>> implementations to allow suicide even though it makes the games longer. > >>>> > >>>> Of course you are right about point B. If suicide is illegal in the > >>>> actual game, there can be no point in allowing it in the play-outs. > >>>> It's almost certainly wrong to allow it in the play-outs even if you are > >>>> playing by suicide rules - a lot of work has gone into finding good > >>>> moves in the play-outs and this would be one of the prime candidates for > >>>> removal! > >>>> > >>>> - Don > >>>> > >>>> > >>>> Jacques Basaldúa wrote: > >>>> > >>>>> Gian-Carlo Pascutto wrote: > >>>>> > >>>>> > >>>>>> Multi-stone suicide is allowed, single stone not. > >>>>>> > >>>>> I hadn't even considered suicide.(It would be a major change for me, > >>>>> as neither my Gui nor my board system allow such moves.) > >>>>> > >>>>> The question is Why do you do it? > >>>>> > >>>>> a. Just in case you wanted the entire program to support suicide go > >>>>> > >>>>> or > >>>>> > >>>>> b. Because that has some advantage as a random playout. > >>>>> > >>>>> If it was b, can anyone explain why suicide is a better evaluation for > >>>>> a normal (non suicide) game. > >>>>> > >>>>> Jacques. > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> computer-go mailing list > >>>>> [email protected] > >>>>> http://www.computer-go.org/mailman/listinfo/computer-go/ > >>>>> > >>>>> > >>>> _______________________________________________ > >>>> computer-go mailing list > >>>> [email protected] > >>>> http://www.computer-go.org/mailman/listinfo/computer-go/ > >>>> > >>> _______________________________________________ > >>> computer-go mailing list > >>> [email protected] > >>> http://www.computer-go.org/mailman/listinfo/computer-go/ > >>> > >>> > >> _______________________________________________ > >> computer-go mailing list > >> [email protected] > >> http://www.computer-go.org/mailman/listinfo/computer-go/ > >> > >> > > > > > > > > > _______________________________________________ > computer-go mailing list > [email protected] > http://www.computer-go.org/mailman/listinfo/computer-go/ > -- _______________________________________________ computer-go mailing list [email protected] http://www.computer-go.org/mailman/listinfo/computer-go/
