Re: [computer-go] simple MC reference bot and specification
sorry to be pedantic, but: 13. Chinese scoring. s. On Sat, Oct 11, 2008 at 9:11 AM, Don Dailey [EMAIL PROTECTED] wrote: On Sat, 2008-10-11 at 13:33 +0100, Claus Reinke wrote: I have a rough idea of what that might be. And I suspect that keeping this de facto standard implicit has been hiding some actual differences in what different people think that standard is. Some of my questions arise from trying to pin down where and why different authors have different ideas of what the standard is. If there has been some explicit standardisation since those papers were published, I'd be interested in a pointer to that standard and its rationale. I'm going to publish a real simple java reference program and some docs to go with it and a program to test it for black box conformance. (Actually, it will test 2 or more and compare them.) I would like to get someone who writes better than I do to write up the standard in less casual language but it goes something like this: 1. A complete game playing program so it can also be tested in real games. 2. Play uniformly random moves except to 1 pt eyes and avoiding simple ko. When a move is otherwise not possible, pass. 3. Playout ends after 2 consecutive pass moves (1 for each side.) 4. 1 pt eye is an empty point surrounded by friendly stones for the side to move. Additionally, we have 2 cases. If the stone is NOT on any edge (where the corner is an edge) there must be no more than one diagonal enemy stone.If the point in question is on the edge, there must be NO diagonal enemy stones. 5. In the playouts, statistics are taken on moves played during the playouts. If the move is played FIRST (during the playout) by the side to move it is one data point and the win loss record is maintained. 6. The move with the highest statistical win rate is the one selected for move in the actual game. 7. In the case of moves with even scores a random selection is made. 8. Pass move are never selected as the final move to play unless no other non-eye filling move is possible. 9. Random number generator is unspecified - your program should simply pass the black box test and as a further optional test your program should score close to 50% against other properly implemented programs. 10. Suicide not allowed in the playouts or in games it plays. 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. 12. If a move has NO STATS taken (which is highly unlikely unless you do very few playouts) it is ignored for move selection. Did I miss anything? I would like to get feedback and agreement on this. Please note - a few GTP commands will be added in order to instrument any conforming programs. Haven't figured those out yet, but it will be designed so that it can report number of nodes, number of playouts, average score of playouts, etc. So the tester may set up some position and a ko, and ask for statistics based on the number of specified playouts. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
I think I already add a 13, so you must mean 14 :-) - Don On Mon, 2008-10-13 at 09:48 -0400, steve uurtamo wrote: sorry to be pedantic, but: 13. Chinese scoring. s. On Sat, Oct 11, 2008 at 9:11 AM, Don Dailey [EMAIL PROTECTED] wrote: On Sat, 2008-10-11 at 13:33 +0100, Claus Reinke wrote: I have a rough idea of what that might be. And I suspect that keeping this de facto standard implicit has been hiding some actual differences in what different people think that standard is. Some of my questions arise from trying to pin down where and why different authors have different ideas of what the standard is. If there has been some explicit standardisation since those papers were published, I'd be interested in a pointer to that standard and its rationale. I'm going to publish a real simple java reference program and some docs to go with it and a program to test it for black box conformance. (Actually, it will test 2 or more and compare them.) I would like to get someone who writes better than I do to write up the standard in less casual language but it goes something like this: 1. A complete game playing program so it can also be tested in real games. 2. Play uniformly random moves except to 1 pt eyes and avoiding simple ko. When a move is otherwise not possible, pass. 3. Playout ends after 2 consecutive pass moves (1 for each side.) 4. 1 pt eye is an empty point surrounded by friendly stones for the side to move. Additionally, we have 2 cases. If the stone is NOT on any edge (where the corner is an edge) there must be no more than one diagonal enemy stone.If the point in question is on the edge, there must be NO diagonal enemy stones. 5. In the playouts, statistics are taken on moves played during the playouts. If the move is played FIRST (during the playout) by the side to move it is one data point and the win loss record is maintained. 6. The move with the highest statistical win rate is the one selected for move in the actual game. 7. In the case of moves with even scores a random selection is made. 8. Pass move are never selected as the final move to play unless no other non-eye filling move is possible. 9. Random number generator is unspecified - your program should simply pass the black box test and as a further optional test your program should score close to 50% against other properly implemented programs. 10. Suicide not allowed in the playouts or in games it plays. 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. 12. If a move has NO STATS taken (which is highly unlikely unless you do very few playouts) it is ignored for move selection. Did I miss anything? I would like to get feedback and agreement on this. Please note - a few GTP commands will be added in order to instrument any conforming programs. Haven't figured those out yet, but it will be designed so that it can report number of nodes, number of playouts, average score of playouts, etc. So the tester may set up some position and a ko, and ask for statistics based on the number of specified playouts. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
[computer-go] simple MC reference bot and specification
-- Don SAID : -- David Fotland suggested another rule, tell me what you think. His rule is to stop the game at or beyond move N*N*2 where N is the board size and score the board no matter what. But in the play-outs he first plays 1 move before making this test. If a game actually lasts that long, the play-outs are not worth much but the game is probably over anyway and the board gets scored as is. My own implementation depends on there being no eyes bigger than 1 point, because it's fast and easy to score a board that has only 1 point eyes. But a simple modification makes it work without any appreciable speed sacrifice: If a game ends beyond move N*N*2 then use the slow scoring routine that doesn't assume the board is complete. It should be very rare that this routine has to be used. - Don - Answer : - I like the principle, however the value you propose seems a bit low ... that would mean 81*2=162 for 9x9 right ? I traced the histogram of game size, counting pass On the left the length of the game on the right the number of games that fall into this category. I think you can use a higher value then like N*N*3. Wich may work for larger board as well. (maybe i should try out to get the distribution out of 19x19 ? :) ) -- There are two reason why i do not like counting one stone captures. The first one is that i'm not 100% sure it'll set things right. Obviously limiting game size can hardly have pathological behavior to it, in term of missing an infinite loop case. The second things, is that i tryed at one point to implement some board implementation with bitmaps. I don't remember all the details so maybe i'm wrong. But i'm quite sure i had a way of notplaying ko wrong, while i didn't counted the number of stone captured. Anyway the implementation at this time didn't showed much promise it was like 3k sim/sec while the goal at the time was to get a way to produce between 100K and 200k simulations/sec per processor. Anyway i do not think i really exhausted all the possibilities with bits maps, and i want to give it another shot in the future. Then when considering the size problem, i think it definitely is easier to get an implementation which is 9x9 tuned with bitMaps :) So my suggestion is to make the black box test aimed at 9x9. It may also be a bit easier to give hard values to help people make sure they are bug-free before they try the black box test suite. -- FOR 9X9 : mean score =2.164072737304306 85775.89950308551 Playout/sec Time=11.663847372 Number of playout=1000477.0 Mean moves per sim 111.06255716023456 Game length : number of games in that category 74 : 1 75 : 5 76 : 14 77 : 31 78 : 82 79 : 159 80 : 333 81 : 596 82 : 1063 83 : 1676 84 : 2352 85 : 3529 86 : 4708 87 : 6323 88 : 7840 89 : 9899 90 : 11773 91 : 14314 92 : 16021 93 : 18832 94 : 20376 95 : 22754 96 : 24179 97 : 26590 98 : 27447 99 : 29852 100 : 29532 101 : 31527 102 : 30818 103 : 32565 104 : 3 105 : 31795 106 : 29896 107 : 30761 108 : 28561 109 : 28786 110 : 26408 111 : 26316 112 : 23620 113 : 22981 114 : 20407 115 : 20034 116 : 17571 117 : 16794 118 : 14478 119 : 13761 120 : 11977 121 : 10898 122 : 9505 123 : 8833 124 : 7626 125 : 7494 126 : 6306 127 : 6766 128 : 5661 129 : 7020 130 : 5862 131 : 7856 132 : 6427 133 : 9115 134 : 7349 135 : 10021 136 : 8230 137 : 10263 138 : 8299 139 : 9514 140 : 7923 141 : 8382 142 : 7106 143 : 7183 144 : 5908 145 : 5667 146 : 4894 147 : 4545 148 : 3855 149 : 3424 150 : 2856 151 : 2524 152 : 2022 153 : 1673 154 : 1505 155 : 1295 156 : 1061 157 : 908 158 : 724 159 : 644 160 : 510 161 : 427 162 : 345 163 : 286 164 : 255 165 : 205 166 : 156 167 : 125 168 : 109 169 : 76 170 : 66 171 : 62 172 : 54 173 : 37 174 : 30 175 : 25 176 : 27 177 : 19 178 : 19 179 : 15 180 : 8 181 : 6 182 : 6 183 : 3 184 : 6 185 : 3 187 : 2 188 : 1 189 : 1 193 : 1 FOR 19x19 --- mean score =1.907221916712493 17337.6306463181 Playout/sec Time=57.686659752 Number of playout=1000150.0 Mean moves per sim 455.3779143128531 Komi = 0,5 Amaf Result : ||,000 ||,493||,503||,500||,503||,504||,504||,503||,503||,504||,504||,504||,503||,503||,504||,504||,503||,501||,500||,492 ||,502||,510||,513||,513||,515||,514||,514||,515||,514||,514||,515||,514||,514||,514||,514||,514||,512||,510||,501 ||,500||,512||,515||,516||,516||,516||,516||,517||,516||,515||,517||,516||,516||,517||,517||,516||,514||,512||,500 ||,502||,515||,516||,517||,517||,518||,517||,517||,516||,517||,517||,517||,517||,517||,517||,516||,517||,514||,503 ||,503||,514||,517||,517||,518||,518||,517||,516||,516||,516||,516||,515||,516||,517||,517||,519||,515||,513||,504 ||,504||,514||,517||,518||,517||,517||,516||,516||,516||,516||,516||,516||,517||,516||,517||,517||,516||,513||,503
[computer-go] simple MC reference bot and specification
On Sat, 2008-10-11 at 13:33 +0100, Claus Reinke wrote: I have a rough idea of what that might be. And I suspect that keeping this de facto standard implicit has been hiding some actual differences in what different people think that standard is. Some of my questions arise from trying to pin down where and why different authors have different ideas of what the standard is. If there has been some explicit standardisation since those papers were published, I'd be interested in a pointer to that standard and its rationale. I'm going to publish a real simple java reference program and some docs to go with it and a program to test it for black box conformance. (Actually, it will test 2 or more and compare them.) I would like to get someone who writes better than I do to write up the standard in less casual language but it goes something like this: 1. A complete game playing program so it can also be tested in real games. 2. Play uniformly random moves except to 1 pt eyes and avoiding simple ko. When a move is otherwise not possible, pass. 3. Playout ends after 2 consecutive pass moves (1 for each side.) 4. 1 pt eye is an empty point surrounded by friendly stones for the side to move. Additionally, we have 2 cases. If the stone is NOT on any edge (where the corner is an edge) there must be no more than one diagonal enemy stone.If the point in question is on the edge, there must be NO diagonal enemy stones. 5. In the playouts, statistics are taken on moves played during the playouts. If the move is played FIRST (during the playout) by the side to move it is one data point and the win loss record is maintained. 6. The move with the highest statistical win rate is the one selected for move in the actual game. 7. In the case of moves with even scores a random selection is made. 8. Pass move are never selected as the final move to play unless no other non-eye filling move is possible. 9. Random number generator is unspecified - your program should simply pass the black box test and as a further optional test your program should score close to 50% against other properly implemented programs. 10. Suicide not allowed in the playouts or in games it plays. 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. 12. If a move has NO STATS taken (which is highly unlikely unless you do very few playouts) it is ignored for move selection. Did I miss anything? I would like to get feedback and agreement on this. Please note - a few GTP commands will be added in order to instrument any conforming programs. Haven't figured those out yet, but it will be designed so that it can report number of nodes, number of playouts, average score of playouts, etc. So the tester may set up some position and a ko, and ask for statistics based on the number of specified playouts. - Don signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
RE: [computer-go] simple MC reference bot and specification
If you don't have sueperko, I think you need a maximum moves stopping criteria too. -Original Message- From: [EMAIL PROTECTED] [mailto:computer-go- [EMAIL PROTECTED] On Behalf Of Don Dailey Sent: Saturday, October 11, 2008 6:11 AM To: computer-go Subject: [computer-go] simple MC reference bot and specification On Sat, 2008-10-11 at 13:33 +0100, Claus Reinke wrote: I have a rough idea of what that might be. And I suspect that keeping this de facto standard implicit has been hiding some actual differences in what different people think that standard is. Some of my questions arise from trying to pin down where and why different authors have different ideas of what the standard is. If there has been some explicit standardisation since those papers were published, I'd be interested in a pointer to that standard and its rationale. I'm going to publish a real simple java reference program and some docs to go with it and a program to test it for black box conformance. (Actually, it will test 2 or more and compare them.) I would like to get someone who writes better than I do to write up the standard in less casual language but it goes something like this: 1. A complete game playing program so it can also be tested in real games. 2. Play uniformly random moves except to 1 pt eyes and avoiding simple ko. When a move is otherwise not possible, pass. 3. Playout ends after 2 consecutive pass moves (1 for each side.) 4. 1 pt eye is an empty point surrounded by friendly stones for the side to move. Additionally, we have 2 cases. If the stone is NOT on any edge (where the corner is an edge) there must be no more than one diagonal enemy stone.If the point in question is on the edge, there must be NO diagonal enemy stones. 5. In the playouts, statistics are taken on moves played during the playouts. If the move is played FIRST (during the playout) by the side to move it is one data point and the win loss record is maintained. 6. The move with the highest statistical win rate is the one selected for move in the actual game. 7. In the case of moves with even scores a random selection is made. 8. Pass move are never selected as the final move to play unless no other non-eye filling move is possible. 9. Random number generator is unspecified - your program should simply pass the black box test and as a further optional test your program should score close to 50% against other properly implemented programs. 10. Suicide not allowed in the playouts or in games it plays. 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. 12. If a move has NO STATS taken (which is highly unlikely unless you do very few playouts) it is ignored for move selection. Did I miss anything? I would like to get feedback and agreement on this. Please note - a few GTP commands will be added in order to instrument any conforming programs. Haven't figured those out yet, but it will be designed so that it can report number of nodes, number of playouts, average score of playouts, etc. So the tester may set up some position and a ko, and ask for statistics based on the number of specified playouts. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
I found one specified issue that must be handled and requires an additional specification. What to do to limit the length of the playout games? As was discussed in the last couple of days or so there are possible cases with multiple ko threats. Please not that the solution I am about to propose doesn't pretend to be a perfect solution since this is just a benchmark (like th eye rule, it isn't perfect) - but I think it does solve the problem. My idea is this: Allow up to N consecutive 1 point captures and then automatically terminate the game.So here is rule 13 in my list: 13. the play-outs terminate after the fifth consecutive 1 stone capture. If someone has a rationale for using a different number let's consider it. I picked 5 for no particular reason. And yes, I know that this might terminate games too early, but the only problem I want to solve is to prevent infinite games. There is of course an argument for using a much bigger number such as 10 or more - 5 single stone captures doesn't prove any kind of ko situation. But I prefer a very simple termination rule that is trivial to implement to something anal and complicated. I don't want to scare anybody away from building this benchmark. Would that guarantee no infinite games?Is there a reason to use a higher number than 5? Should we have an even simpler termination rule such as no playout shall last more than BOARDSIZE * BOARDSIZE * 3 moves? In addition I think it's good to require that integer komi be allowed. - Don On Sat, 2008-10-11 at 09:11 -0400, Don Dailey wrote: I'm going to publish a real simple java reference program and some docs to go with it and a program to test it for black box conformance. (Actually, it will test 2 or more and compare them.) I would like to get someone who writes better than I do to write up the standard in less casual language but it goes something like this: 1. A complete game playing program so it can also be tested in real games. 2. Play uniformly random moves except to 1 pt eyes and avoiding simple ko. When a move is otherwise not possible, pass. 3. Playout ends after 2 consecutive pass moves (1 for each side.) 4. 1 pt eye is an empty point surrounded by friendly stones for the side to move. Additionally, we have 2 cases. If the stone is NOT on any edge (where the corner is an edge) there must be no more than one diagonal enemy stone.If the point in question is on the edge, there must be NO diagonal enemy stones. 5. In the playouts, statistics are taken on moves played during the playouts. If the move is played FIRST (during the playout) by the side to move it is one data point and the win loss record is maintained. 6. The move with the highest statistical win rate is the one selected for move in the actual game. 7. In the case of moves with even scores a random selection is made. 8. Pass move are never selected as the final move to play unless no other non-eye filling move is possible. 9. Random number generator is unspecified - your program should simply pass the black box test and as a further optional test your program should score close to 50% against other properly implemented programs. 10. Suicide not allowed in the playouts or in games it plays. 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. 12. If a move has NO STATS taken (which is highly unlikely unless you do very few playouts) it is ignored for move selection. Did I miss anything? I would like to get feedback and agreement on this. Please note - a few GTP commands will be added in order to instrument any conforming programs. Haven't figured those out yet, but it will be designed so that it can report number of nodes, number of playouts, average score of playouts, etc. So the tester may set up some position and a ko, and ask for statistics based on the number of specified playouts. - Don signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Sat, Oct 11, 2008 at 8:11 AM, Don Dailey [EMAIL PROTECTED] wrote: I'm going to publish a real simple java reference program and some docs to go with it and a program to test it for black box conformance. (Actually, it will test 2 or more and compare them.) I would like to get someone who writes better than I do to write up the standard in less casual language but it goes something like this: 1. A complete game playing program so it can also be tested in real games. 2. Play uniformly random moves except to 1 pt eyes and avoiding simple ko. When a move is otherwise not possible, pass. 3. Playout ends after 2 consecutive pass moves (1 for each side.) 4. 1 pt eye is an empty point surrounded by friendly stones for the side to move. Additionally, we have 2 cases. If the stone is NOT on any edge (where the corner is an edge) there must be no more than one diagonal enemy stone.If the point in question is on the edge, there must be NO diagonal enemy stones. 5. In the playouts, statistics are taken on moves played during the playouts. If the move is played FIRST (during the playout) by the side to move it is one data point and the win loss record is maintained. Wouldn't it make more sense to run an equal number of playouts for each root move? 6. The move with the highest statistical win rate is the one selected for move in the actual game. 7. In the case of moves with even scores a random selection is made. I think maybe this should be something deterministic. 8. Pass move are never selected as the final move to play unless no other non-eye filling move is possible. 9. Random number generator is unspecified - your program should simply pass the black box test and as a further optional test your program should score close to 50% against other properly implemented programs. This is too vague IMO--I think specifying the RNG would be better. I like Marsaglia's--that's what I use now. 10. Suicide not allowed in the playouts or in games it plays. 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. That's fine for this spec, but I don't do this and don't plan on it. 12. If a move has NO STATS taken (which is highly unlikely unless you do very few playouts) it is ignored for move selection. Did I miss anything? I would like to get feedback and agreement on this. Please note - a few GTP commands will be added in order to instrument any conforming programs. Haven't figured those out yet, but it will be designed so that it can report number of nodes, number of playouts, average score of playouts, etc. So the tester may set up some position and a ko, and ask for statistics based on the number of specified playouts. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. positional superko (as opposed to situational superko). - Dave Hillis -Original Message- From: Don Dailey [EMAIL PROTECTED] To: computer-go computer-go@computer-go.org Sent: Sat, 11 Oct 2008 9:11 am Subject: [computer-go] simple MC reference bot and specification On Sat, 2008-10-11 at 13:33 +0100, Claus Reinke wrote: I have a rough idea of what that might be. And I suspect that keeping this de facto standard implicit has been hiding some actual differences in what different people think that standard is. Some of my questions arise from trying to pin down where and why different authors have different ideas of what the standard is. If there has been some explicit standardisation since those papers were published, I'd be interested in a pointer to that standard and its rationale. I'm going to publish a real simple java reference program and some docs to go with it and a program to test it for black box conformance. (Actually, it will test 2 or more and compare them.) I would like to get someone who writes better than I do to write up the standard in less casual language but it goes something like this: 1. A complete game playing program so it can also be tested in real games. 2. Play uniformly random moves except to 1 pt eyes and avoiding simple ko. When a move is otherwise not possible, pass. 3. Playout ends after 2 consecutive pass moves (1 for each side.) 4. 1 pt eye is an empty point surrounded by friendly stones for the side to move. Additionally, we have 2 cases. If the stone is NOT on any edge (where the corner is an edge) there must be no more than one diagonal enemy stone.If the point in question is on the edge, there must be NO diagonal enemy stones. 5. In the playouts, statistics are taken on moves played during the playouts. If the move is played FIRST (during the playout) by the side to move it is one data point and the win loss record is maintained. 6. The move with the highest statistical win rate is the one selected for move in the actual game. 7. In the case of moves with even scores a random selection is made. 8. Pass move are never selected as the final move to play unless no other non-eye filling move is possible. 9. Random number generator is unspecified - your program should simply pass the black box test and as a further optional test your program should score close to 50% against other properly implemented programs. 10. Suicide not allowed in the playouts or in games it plays. 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. 12. If a move has NO STATS taken (which is highly unlikely unless you do very few playouts) it is ignored for move selection. Did I miss anything? I would like to get feedback and agreement on this. Please note - a few GTP commands will be added in order to instrument any conforming programs. Haven't figured those out yet, but it will be designed so that it can report number of nodes, number of playouts, average score of playouts, etc. So the tester may set up some position and a ko, and ask for statistics based on the number of specified playouts. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Oct 11, 2008, at 2:40 PM, [EMAIL PROTECTED] wrote: 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. positional superko (as opposed to situational superko). Whatever is used by the rules... I know CGOS and KGS Chinesee use positional super ko. I don't know which rule sets would use situational. Situational makes more sense to me. - Dave Hillis -Original Message- From: Don Dailey [EMAIL PROTECTED] To: computer-go computer-go@computer-go.org Sent: Sat, 11 Oct 2008 9:11 am Subject: [computer-go] simple MC reference bot and specification On Sat, 2008-10-11 at 13:33 +0100, Claus Reinke wrote: I have a rough idea of what that might be. And I suspect that keeping this de facto standard implicit has been hiding some actual differences in what different people think that standard is. Some of my questions arise from trying to pin down where and why different authors have different ideas of what the standard is. If there has been some explicit standardisation since those papers were published, I'd be interested in a pointer to that standard and its rationale. I'm going to publish a real simple java reference program and some docs to go with it and a program to test it for black box conformance. (Actually, it will test 2 or more and compare them.) I would like to get someone who writes better than I do to write up the standard in less casual language but it goes something like this: 1. A complete game playing program so it can also be tested in real games. 2. Play uniformly random moves except to 1 pt eyes and avoiding simple ko. When a move is otherwise not possible, pass. 3. Playout ends after 2 consecutive pass moves (1 for each side.) 4. 1 pt eye is an empty point surrounded by friendly stones for the side to move. Additionally, we have 2 cases. If the stone is NOT on any edge (where the corner is an edge) there must be no more than one diagonal enemy stone.If the point in question is on the edge, there must be NO diagonal enemy stones. 5. In the playouts, statistics are taken on moves played during the playouts. If the move is played FIRST (during the playout) by the side to move it is one data point and the win loss record is maintained. 6. The move with the highest statistical win rate is the one selected for move in the actual game. 7. In the case of moves with even scores a random selection is made. 8. Pass move are never selected as the final move to play unless no other non-eye filling move is possible. 9. Random number generator is unspecified - your program should simply pass the black box test and as a further optional test your program should score close to 50% against other properly implemented programs. 10. Suicide not allowed in the playouts or in games it plays. 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. 12. If a move has NO STATS taken (which is highly unlikely unless you do very few playouts) it is ignored for move selection. Did I miss anything? I would like to get feedback and agreement on this. Please note - a few GTP commands will be added in order to instrument any conforming programs. Haven't figured those out yet, but it will be designed so that it can report number of nodes, number of playouts, average score of playouts, etc. So the tester may set up some position and a ko, and ask for statistics based on the number of specified playouts. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ McCain or Obama? Stay updated on coverage of the Presidential race while you browse - Download Now! ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Sat, 2008-10-11 at 13:34 -0500, Zach Wegner wrote: Wouldn't it make more sense to run an equal number of playouts for each root move? No, because with AMAF you will get different game counts ANYWAY. And also I believe most programs are viewing the level in playouts as a fixed number of play-outs, not a fixed number of play-outs per root move choice and I would want to conform to what most people are doing. However I'm not opposed to doing it that way, especially if that's what most people are already doing anyway. But I'm just trying to nail down what most people are already doing so that only minor changes would be necessary to anyone wanting to implement this. - Don signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Sat, 2008-10-11 at 13:34 -0500, Zach Wegner wrote: 7. In the case of moves with even scores a random selection is made. I think maybe this should be something deterministic. That of course is clearly a possibility. However I'm trying to approach this with a certain consistent methodology which is to specify the BEHAVIOR, not the implementation. I want complete freedom in how you implement the specified behavior. Also, if you chose a different generator, it would not be possible to prove it if it were a good one. Not unless we also got heavy handed about how it's initialized, and how you select moves with a random distribution.In short, I fear that we would be asking too much to build a bunch of clones that are completely deterministic. If you look at pseudo random number generators you will see there is this same implicit methodology, though not stated. The behavior specified is to appear random and test are designed to see how well this behavior is simulated.In some sense, people consider the best random number generators to be the ones that fool the Diehard tests. They fool these diehard tests into thinking they are really random. I want this to work that way. We will build tests and I don't care if you can find a way to fool the tests. However the tester will be designed to be difficult to fool and it will not present you with predictable positions. I don't care if you don't follow the spec 100% if you can fool the tests. If you can do this, you will probably have found something useful to all of us. This is not going to be a formal thing like the Computer Benchmarks Game is. We will just build a tester and publish the spec and you can do what you want with it. Maybe put entries on Sensei's page. - Don signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Sat, 2008-10-11 at 13:34 -0500, Zach Wegner wrote: 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. That's fine for this spec, but I don't do this and don't plan on it. You don't have to do it, it's just a provision to allow your bot to play legal games. The tester probably won't require it since it will just be looking for statistics from specified (random) positions. A more comprehensive test is to play games with these reference bots and check to see if any are straying too far from 50% scores against the others.Your program could still participate, although it would probably lose slightly more than the others due to the occasionally superko loss. Hopefully, this comprehensive tester could identify just from the game statistics that something is wrong. - Don signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Sat, 2008-10-11 at 14:40 -0400, [EMAIL PROTECTED] wrote: 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. positional superko (as opposed to situational superko). Thank you. I forgot to specify which. - Don - Dave Hillis -Original Message- From: Don Dailey [EMAIL PROTECTED] To: computer-go computer-go@computer-go.org Sent: Sat, 11 Oct 2008 9:11 am Subject: [computer-go] simple MC reference bot and specification On Sat, 2008-10-11 at 13:33 +0100, Claus Reinke wrote: I have a rough idea of what that might be. And I suspect that keeping this de facto standard implicit has been hiding some actual differences in what different people think that standard is. Some of my questions arise from trying to pin down where and why different authors have different ideas of what the standard is. If there has been some explicit standardisation since those papers were published, I'd be interested in a pointer to that standard and its rationale. I'm going to publish a real simple java reference program and some docs to go with it and a program to test it for black box conformance. (Actually, it will test 2 or more and compare them.) I would like to get someone who writes better than I do to write up the standard in less casual language but it goes something like this: 1. A complete game playing program so it can also be tested in real games. 2. Play uniformly random moves except to 1 pt eyes and avoiding simple ko. When a move is otherwise not possible, pass. 3. Playout ends after 2 consecutive pass moves (1 for each side.) 4. 1 pt eye is an empty point surrounded by friendly stones for the side to move. Additionally, we have 2 cases. If the stone is NOT on any edge (where the corner is an edge) there must be no more than one diagonal enemy stone.If the point in question is on the edge, there must be NO diagonal enemy stones. 5. In the playouts, statistics are taken on moves played during the playouts. If the move is played FIRST (during the playout) by the side to move it is one data point and the win loss record is maintained. 6. The move with the highest statistical win rate is the one selected for move in the actual game. 7. In the case of moves with even scores a random selection is made. 8. Pass move are never selected as the final move to play unless no other non-eye filling move is possible. 9. Random number generator is unspecified - your program should simply pass the black box test and as a further optional test your program should score close to 50% against other properly implemented programs. 10. Suicide not allowed in the playouts or in games it plays. 11. When selecting moves to play in the actual game (not playouts) superko is checked and forbidden. 12. If a move has NO STATS taken (which is highly unlikely unless you do very few playouts) it is ignored for move selection. Did I miss anything? I would like to get feedback and agreement on this. Please note - a few GTP commands will be added in order to instrument any conforming programs. Haven't figured those out yet, but it will be designed so that it can report number of nodes, number of playouts, average score of playouts, etc. So the tester may set up some position and a ko, and ask for statistics based on the number of specified playouts. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ __ McCain or Obama? Stay updated on coverage of the Presidential race while you browse - Download Now! signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Sat, 2008-10-11 at 14:48 -0400, Jason House wrote: Whatever is used by the rules... I know CGOS and KGS Chinesee use positional super ko. I don't know which rule sets would use situational. Situational makes more sense to me. I've always personally preferred situation super ko (probably because of my chess background) but Position Super Ko seems to be the more popular standard and CGOS uses it as well as KGS. What are most people using? - Don signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
[computer-go] simple MC reference bot and specification
On point 13. 1 stone captures, may eventually be hard for some implementation. I think using game length as a criterion gives more freedom. Then you have to specify what to do with those pathological games anyway. Do you score them as they are, or do you drop them. I do count them. The rule for counting is probably quite standard too. I give a point for each stone present. Then for each empty stone, i give a point to the side who has all orthogonal control of it, if any. I usually implements super-ko checking very late, if i do at all. I often start by selecting either the first or the last best move, rather than picking one at random. Although picking at random makes me more comfortable somehow. It could be interesting to see how much difference it makes. Finally, about the size. Is it supposed to be 9x9 only ? 9x9 only gives more freedom to fine tune for this size. It may make the implementations less useful, if someone knows another sizes the light-policy could succeed on. (like maybe studying 7x7 and such ?) _ Email envoyé avec Windows Live Hotmail. Dites adieux aux spam et virus, passez à Hotmail ! C'est gratuit ! http://www.windowslive.fr/hotmail/default.asp___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Sat, 2008-10-11 at 21:46 +0200, Denis fidaali wrote: On point 13. 1 stone captures, may eventually be hard for some implementation. I think using game length as a criterion gives more freedom. I definitely considered that. My own program returns the number of stones captured (and a list of those stones if I ask for it) and so it's easy for me, but wonder if there are programs where this would be difficult to detect? My arguments in favor of the rule is that it's not quite as artificial as setting some arbitrary limit and I think this would not unduly delay the terminations of games. With a limit of hundreds of moves, some games would go hundreds of moves beyond what is necessary. Are there any MC programs out that cannot detect whether they made a one stone capture and it would be unduly difficult to fix this? Then you have to specify what to do with those pathological games anyway. Do you score them as they are, or do you drop them. I do count them. You score them as is. You may not get the correct score, but after all these ARE random games and we expect this to be extremely rare. The rule for counting is probably quite standard too. I give a point for each stone present. Then for each empty stone, i give a point to the side who has all orthogonal control of it, if any. Yes. Tromp/Taylor which are Chinese rules. I usually implements super-ko checking very late, if i do at all. I often start by selecting either the first or the last best move, rather than picking one at random. Although picking at random makes me more comfortable somehow. It could be interesting to see how much difference it makes. Of course everyone is free to implement any variation they want, controlled by switches and such.The idea is to have an extremely reliable benchmark and this will also be very helpful for new developers as this also would serve as a good first program for new developers and they could have confidence that they got all the details right. Finally, about the size. Is it supposed to be 9x9 only ? It can be anything you want, but I suggest that you develop a program capable of playing on any board size. Having said that, you can get a little more speed if you code to just 1 specific board size. Lazarus compiles to any ODD board size but it's fixed at compile time and I have to recompile to get other board sizes. I doubt this optimization is worth very much and it's annoying that I have to fetch a different binary in order to play other board sizes. Also, I don't really keep up with how new processor families change the rules. I know that I USED to get a nice speedup several years ago by using as many constants as possible (as opposed to variables) but I have no idea whether (or how much) this is true today. 9x9 only gives more freedom to fine tune for this size. It may make the implementations less useful, if someone knows another sizes the light-policy could succeed on. (like maybe studying 7x7 and such ?) How do you define succeed?It works on all board sizes, it just works better on smaller boards. You could equally say it doesn't work on any board size since there are ways to build stronger programs. - Don _ Email envoyé avec Windows Live Hotmail. Dites adieux aux spam et virus, passez à Hotmail ! C'est gratuit ! http://www.windowslive.fr/hotmail/default.asp___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
Don Dailey wrote: Are there any MC programs out that cannot detect whether they made a one stone capture and it would be unduly difficult to fix this? They would have to detect that to detect simple Ko. Speaking of which, wouldn't it be better to limit consecutive Kos instead of limiting consecutive 1-stone captures? ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Sat, 2008-10-11 at 16:32 -0400, Michael Williams wrote: Don Dailey wrote: Are there any MC programs out that cannot detect whether they made a one stone capture and it would be unduly difficult to fix this? They would have to detect that to detect simple Ko. That's a good point. Speaking of which, wouldn't it be better to limit consecutive Kos instead of limiting consecutive 1-stone captures? Wouldn't this definitely slow down most if not all light implementations and require code changes to existing programs? My own programs do not really know that a ko exists until it decides to make that specific move. Then when it tries to play that move the make() routine discovers that it is was a ko and reject it, returning an illegal move value. To fix this I would have to test every move just to see if there is a ko situation (although there are no doubt some shortcuts.) So I don't think that is a practical rule although it is without doubt slightly superior. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
Don Dailey wrote: Speaking of which, wouldn't it be better to limit consecutive Kos instead of limiting consecutive 1-stone captures? Wouldn't this definitely slow down most if not all light implementations and require code changes to existing programs? My own programs do not really know that a ko exists until it decides to make that specific move. Then when it tries to play that move the make() routine discovers that it is was a ko and reject it, returning an illegal move value. To fix this I would have to test every move just to see if there is a ko situation (although there are no doubt some shortcuts.) I detect simple Ko at the time of the legal capture. I'm curious how you do it at the time of the illegal capture attempt. How do you tell a legal 1-stone capture from an illegal one? ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Sat, 2008-10-11 at 16:55 -0400, Don Dailey wrote: To fix this I would have to test every move just to see if there is a ko situation (although there are no doubt some shortcuts.) One shortcut is to simply not test for this until after you make a 1 stone capture, then test all possible moves for the previous position. But of course this greatly complicates most implementations I fear. An extension of this shortcut is to delay ALL testing until the number of 1 stone captures indicate that you need to test N previous positions for KO. This would not be easy for me unless I were to accept the slowdown of using my slow move maker in the playouts. My slow move maker maintains full state between moves. - Don signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
On Sat, 2008-10-11 at 17:01 -0400, Michael Williams wrote: Don Dailey wrote: Speaking of which, wouldn't it be better to limit consecutive Kos instead of limiting consecutive 1-stone captures? Wouldn't this definitely slow down most if not all light implementations and require code changes to existing programs? My own programs do not really know that a ko exists until it decides to make that specific move. Then when it tries to play that move the make() routine discovers that it is was a ko and reject it, returning an illegal move value. To fix this I would have to test every move just to see if there is a ko situation (although there are no doubt some shortcuts.) I detect simple Ko at the time of the legal capture. I'm curious how you do it at the time of the illegal capture attempt. How do you tell a legal 1-stone capture from an illegal one? An illegal one stone capture is a capture of the same stone that was just used to make a 1 stone capture itself. I think I got that right. Whenever there is a 1 stone capture I flag that and remember it's location (in the move list stack) but a one stone capture does not imply a KO and that takes time to test - so I don't test. So I don't know that a capture move is a KO and most of the time I don't need to know. The next player selects a random move, which MIGHT be a capture attempt which is illegal due to simple KO. If it is, only then do I know that the previous move was a KO. I'm slightly confused by the language ko what is a ko? To me a ko is illegal so it never happens but I suppose my definition is wrong and it's any board position where an illegal repetition is possible due to whatever ko rule is in effect. - Don ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] simple MC reference bot and specification
David Fotland suggested another rule, tell me what you think. His rule is to stop the game at or beyond move N*N*2 where N is the board size and score the board no matter what. But in the play-outs he first plays 1 move before making this test. If a game actually lasts that long, the play-outs are not worth much but the game is probably over anyway and the board gets scored as is. My own implementation depends on there being no eyes bigger than 1 point, because it's fast and easy to score a board that has only 1 point eyes. But a simple modification makes it work without any appreciable speed sacrifice: If a game ends beyond move N*N*2 then use the slow scoring routine that doesn't assume the board is complete. It should be very rare that this routine has to be used. - Don On Sat, 2008-10-11 at 16:15 -0400, Don Dailey wrote: On Sat, 2008-10-11 at 21:46 +0200, Denis fidaali wrote: On point 13. 1 stone captures, may eventually be hard for some implementation. I think using game length as a criterion gives more freedom. I definitely considered that. My own program returns the number of stones captured (and a list of those stones if I ask for it) and so it's easy for me, but wonder if there are programs where this would be difficult to detect? My arguments in favor of the rule is that it's not quite as artificial as setting some arbitrary limit and I think this would not unduly delay the terminations of games. With a limit of hundreds of moves, some games would go hundreds of moves beyond what is necessary. Are there any MC programs out that cannot detect whether they made a one stone capture and it would be unduly difficult to fix this? Then you have to specify what to do with those pathological games anyway. Do you score them as they are, or do you drop them. I do count them. You score them as is. You may not get the correct score, but after all these ARE random games and we expect this to be extremely rare. The rule for counting is probably quite standard too. I give a point for each stone present. Then for each empty stone, i give a point to the side who has all orthogonal control of it, if any. Yes. Tromp/Taylor which are Chinese rules. I usually implements super-ko checking very late, if i do at all. I often start by selecting either the first or the last best move, rather than picking one at random. Although picking at random makes me more comfortable somehow. It could be interesting to see how much difference it makes. Of course everyone is free to implement any variation they want, controlled by switches and such.The idea is to have an extremely reliable benchmark and this will also be very helpful for new developers as this also would serve as a good first program for new developers and they could have confidence that they got all the details right. Finally, about the size. Is it supposed to be 9x9 only ? It can be anything you want, but I suggest that you develop a program capable of playing on any board size. Having said that, you can get a little more speed if you code to just 1 specific board size. Lazarus compiles to any ODD board size but it's fixed at compile time and I have to recompile to get other board sizes. I doubt this optimization is worth very much and it's annoying that I have to fetch a different binary in order to play other board sizes. Also, I don't really keep up with how new processor families change the rules. I know that I USED to get a nice speedup several years ago by using as many constants as possible (as opposed to variables) but I have no idea whether (or how much) this is true today. 9x9 only gives more freedom to fine tune for this size. It may make the implementations less useful, if someone knows another sizes the light-policy could succeed on. (like maybe studying 7x7 and such ?) How do you define succeed?It works on all board sizes, it just works better on smaller boards. You could equally say it doesn't work on any board size since there are ways to build stronger programs. - Don _ Email envoyé avec Windows Live Hotmail. Dites adieux aux spam et virus, passez à Hotmail ! C'est gratuit ! http://www.windowslive.fr/hotmail/default.asp___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ signature.asc Description: This is a digitally signed message part ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
