Re: [computer-go] Roadmap 2020 - using analysis mode to improve programs
I analyzed these positions with Valkyria, and it has no problems seeing what is happening in these positions. Also going back some moves valkyria for example proposes Ws8 instead of Wp12 for move 218, which clearly kills the black group and reduces it no eyes, and showes that Valkyria does know what it is doing (it is not perfect but play acceptable with a few seconds of thinking timew most of time). My point here is just that playing strongly in one type of semeai does not mean it will crush you. But you are right in that analyzing positions like this can identify systematic weaknesse of programs. The problem is that at least for how valkyria works there is not one single fix, the problem is often that a special situation in the playout has to be safely identified and a particulrly bad move need to be pruned, or a deterministic response has to be made to some special kind of threat. But these changes rarely apply to more than a small proportion of the positions encountered in games. The main playing strength come from efficient search in general, and the knowledge that apply to common shapes rather than special tricky situations. Nethertheless I discoverd a nice way of finding tricky situations where evaluations goes very wrong. I run long tests whith 50-500 playouts per move. I also allow Valkyria to resign in these tests. This makes testing go faster and as side effect one sometimes get very long games because the losing colour did not understand it was losing. So I play a 1000 such test games and look for unusually long games. Often it was just a complicated even fight but sometimes one finds some huge misevaluation to fix. Still fixing those things rarely give a measurable boost to playing strength. But I hope they will add up in the long run. Best Magnus Quoting terry mcintyre terrymcint...@yahoo.com: I haven't got a ladder example at the moment, but here's an instance where Leela does not realize it is in terrible trouble. I ( with my 8 kyu AGA rating) know with certainty by move 223 (T5) that Black has captured a large white group. A stronger player could read this out sooner than I. This fight is too big to lose for either side; nothing else on the board matters. ( anyone? how early is this outcome pre-ordained? ) Based on the results of its analysis mode, Leela does not recognize the outcome of this semeai until the large white group in the bottom right is down to two liberties. The problem is even more stark in example2 -- similar board, black has foolishly played one of his own liberties for illustrative purposes. It is black's play, black has three liberties, white has three. Black must take away a liberty from white to win the capturing race, or make two eyes at T8. Black has only four playable moves; any other choice fails. Leela proposes - even after several minutes of analysis and a million nodes - that Black should tennuki at H14. That would snatch defeat from the jaws of certain victory; White would dive into T8 and win the race. I started this thread with the contention that analysis mode can help developers find problems, I hope this example explains why. My theory is that if a program could reliably recognize the outcome of such capturing races five or ten moves sooner, it could crush the likes of me. :D Terry McIntyre terrymcint...@yahoo.com Government is an association of men who do violence to the rest of us. - Leo Tolstoy From: Michael Williams michaelwilliam...@gmail.com To: computer-go computer-go@computer-go.org Sent: Tuesday, April 21, 2009 1:57:54 PM Subject: Re: [computer-go] Reply to Lukasz and Don + Roadmap 2020 Mention the program so that the author can either refute your claim or fix the bug. terry mcintyre wrote: Is it reasonable to expect pro players to use 6-dan programs as a tool for analysis? The pro players are markedly better - at a rough guess, a pro player could give a 6 dan amateur human or program a 3 stone handicap. On the other end of the scale, beginning players and mid kyu players could indeed make good use of an analysis mode by a program which is better than themselves. Lastly, an analysis mode would be helpful to developers, methinks. After winning a game, I like to back up a few moves and find out when the program realized that it was behind. This often happens several moves after the fatal blow has already been struck. I know the feeling too well, when stronger players deftly skewer my group and I only discover the problem five moves later. What do they know that I don't? What do they know that the program doesn't? We have a saying, you learn the most from reviewing games which you have lost. An analysis mode can help developers to discover when their pride and joy first begins to miss the target. Lately, I have been playing quite
Re: [computer-go] Digital Mars
Hi Łukasz , It's fixed now. Thanks a lot! laptop:/u/SW/src/lukaszlew-libego-e4acac7545770fe008c1ff30cf99f874fd7e9272$ build/example/opt/ego = Benchmarking, please wait ... = 20 playouts in 2.83218 seconds 70.6171 kpps 34.8904 kpps/GHz (clock independent) 105316/94359 (black wins / white wins) = 20 playouts in 2.82418 seconds 70.8171 kpps 35.324 kpps/GHz (clock independent) 104924/94746 (black wins / white wins) = 20 playouts in 2.81217 seconds 71.1193 kpps 35.4917 kpps/GHz (clock independent) 105097/94582 (black wins / white wins) = 20 playouts in 2.80818 seconds 71.2206 kpps 35.518 kpps/GHz (clock independent) 105139/94547 (black wins / white wins) = 20 playouts in 2.81618 seconds 71.0183 kpps 35.5761 kpps/GHz (clock independent) 104896/94794 (black wins / white wins) = Try 'help' ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Digital Mars
On Thu, Apr 23, 2009 at 01:25, elife elife2...@gmail.com wrote: On my Intel(R) Core(TM)2 Duo CPU T7200 @ 2.00GHz, using linux and the exact compiler libego was tuned for, I get 70 kpps/GHz. = 20 playouts in 2.85618 seconds 70.0236 kpps -154.124 kpps/GHz (clock independent) I found this kind of garbage associated with 64 bit systems. It's probably because of my poor assembly, but I thought I fixed it recently. Can you check newest version? http://github.com/lukaszlew/libego/zipball/master Lukasz 104896/94794 (black wins / white wins) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Libego benchmarking
This is because there is no getrusage function on windows, so I just return (and divide) by 0. I will try to fix it in next week. Lukasz On Thu, Apr 23, 2009 at 07:28, Petri Pitkanen petri.t.pitka...@gmail.com wrote: Because your time measurement has gone wrong. You get 0 seconds in time hence kpssa in infinity. Petri 2009/4/23 Michael Williams michaelwilliam...@gmail.com: Here is my full set of numbers. I wonder why the known kpps/GHz but unknown kpps. = Benchmarking, please wait ... = 20 playouts in 0 seconds 1.#INF kpps 40.0245 kpps/GHz (clock independent) 105316/94359 (black wins / white wins) -- Petri Pitkänen e-mail: petri.t.pitka...@gmail.com ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Roadmap 2020 - using analysis mode to improve programs
I appreciate that Go programs are complex and not easy to tune. Thinking over Magnus' excellent automated method ( play many games, allow early resignation, inspect long games ), and my own experiences, I'd like to suggest an additional method: when a game ends with a large loss, determine retroactively a) how far back the program was doomed but deluded about the outcome, and b) what the correct earlier plays would have been. Tune and test with a regression suite of difficult positions. This is like troubleshooting any other large and complex program; a subtle error in one portion may only reveal itself when a perfect storm of circumstances arises - but the bug is there all the time. As Magnus and Valkyria pointed out, proper play at an earlier point in my example game would have destroyed Black's position. I don't feel so proud now, lol. At my level of play, it is distressingly common to mis-read capturing races -- it's possible that a good understanding of that topic would improve Go programs. The difficulty cannot be understated - others have indicated that Hunter's Counting Liberties and Winning Capturing Races book, valuable as it may be, misses some cases. As Hunter observes, even dan-level players sometimes make mistakes in capturing races. Programs which get semeai and seki right every time might be a few stones stronger. They'd certainly be more valuable as teaching tools. In the game above, a stronger program would have exploited my earlier weakness; this would have encouraged me to make better moves. Back to Roadmap: 2020, I'd love a status map showing groups which are certainly alive, groups which are unstable, and groups which are certainly dead ( assuming proper play ). That would be quite a feedback tool. When an approach move or throw-in threatens the status of a group, the group marker would change from green to blinking yellow. When the attack succeeds, the group marker would change to red. To be useful, this tool would have to be accurate. If not 100% accurate, it should at least give some indication of its level of confidence. Even better, especially for double-digit-kyu players, would be an exposition of why a group is live, dead, seki, unstable, etc. Terry McIntyre terrymcint...@yahoo.com Government is an association of men who do violence to the rest of us. - Leo Tolstoy ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Re: Analysis mode for human use
But odd move numbers always mean black to move. That becomes second nature very quickly and I personally prefer the less verbose syntax. - Don On Thu, Apr 23, 2009 at 1:47 AM, Darren Cook dar...@dcook.org wrote: translated to Ishi-go B 1 Q4 W 2 R16 B 3 C4 W 4 F3 ... *** modified Ishi-go 1. q4 2. r16 3. c4 4. f3 ... from the west - my modified Ishi-go-format should be even better. (The repetitive B W are a bit annoying in Ishi-go, and small letters are better to read than capital ones.) I find the B/W very useful: when playing out a long list of moves it is very easy to lose track where I am. Most moves are equally likely for both sides. Darren -- Darren Cook, Software Researcher/Developer http://dcook.org/mlsn/ (English-Japanese-German-Chinese-Arabic open source dictionary/semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
[computer-go] Re: Analysis mode for human use
Don Dailey wrote: But odd move numbers always mean black to move. That becomes second nature very quickly and I personally prefer the less verbose syntax. Darren Cook wrote: I find the B/W very useful: when playing out a long list of moves it is very easy to lose track where I am. Most moves are equally likely for both sides. You may have a look at the notation on the gameserver http://www.littlegolem.net . There no B and W are shown, but the moves themselves are in black and white, on a grey background. Example http://www.littlegolem.net/jsp/game/game.jsp?gid=933716 Ingo. -- Neu: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate + Telefonanschluss für nur 17,95 Euro/mtl.!* http://dslspecial.gmx.de/freedsl-surfflat/?ac=OM.AD.PD003K11308T4569a ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
RE: [computer-go] Roadmap 2020 - using analysis mode to improve programs
Many faces will show group status, but with letters on the stones, not colors. From: computer-go-boun...@computer-go.org [mailto:computer-go-boun...@computer-go.org] On Behalf Of terry mcintyre Sent: Thursday, April 23, 2009 7:04 AM To: computer-go Subject: Re: [computer-go] Roadmap 2020 - using analysis mode to improve programs Back to Roadmap: 2020, I'd love a status map showing groups which are certainly alive, groups which are unstable, and groups which are certainly dead ( assuming proper play ). That would be quite a feedback tool. When an approach move or throw-in threatens the status of a group, the group marker would change from green to blinking yellow. When the attack succeeds, the group marker would change to red. To be useful, this tool would have to be accurate. If not 100% accurate, it should at least give some indication of its level of confidence. Even better, especially for double-digit-kyu players, would be an exposition of why a group is live, dead, seki, unstable, etc. Terry McIntyre terrymcint...@yahoo.com Government is an association of men who do violence to the rest of us. - Leo Tolstoy ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Digital Mars
I have two benchmarks: On an: Intel(R) Core(TM)2 CPU T7200 @ 2.00GHz stepping 06 g++ --version g++ (GCC) 4.1.2 20070925 (Red Hat 4.1.2-33) I had to modify SConstruct to refer to the default g++, not g++.4.2 and had to remove -march=native = Benchmarking, please wait ... = 20 playouts in 2.72759 seconds 73.3249 kpps 36.5657 kpps/GHz (clock independent) 105316/94359 (black wins / white wins) = 20 playouts in 2.73858 seconds 73.0304 kpps 36.4108 kpps/GHz (clock independent) 104924/94746 (black wins / white wins) = 20 playouts in 2.72858 seconds 73.2981 kpps 36.5291 kpps/GHz (clock independent) 105097/94582 (black wins / white wins) = 20 playouts in 2.76258 seconds 72.3961 kpps 36.1141 kpps/GHz (clock independent) 105139/94547 (black wins / white wins) = 20 playouts in 2.74358 seconds 72.8974 kpps 36.3124 kpps/GHz (clock independent) 104896/94794 (black wins / white wins) = Try 'help' on an: Intel(R) Core(TM)2 Quad CPUQ9650 @ 3.00GHz stepping 0a g++ --version g++ (GCC) 4.3.0 20080428 (Red Hat 4.3.0-8) (with -march=native flag) = Benchmarking, please wait ... = 20 playouts in 1.65575 seconds 120.791 kpps 40.2566 kpps/GHz (clock independent) 105316/94359 (black wins / white wins) = 20 playouts in 1.65275 seconds 121.011 kpps 40.3069 kpps/GHz (clock independent) 104924/94746 (black wins / white wins) = 20 playouts in 1.65375 seconds 120.937 kpps 40.2789 kpps/GHz (clock independent) 105097/94582 (black wins / white wins) = 20 playouts in 1.65475 seconds 120.864 kpps 40.2917 kpps/GHz (clock independent) 105139/94547 (black wins / white wins) = 20 playouts in 1.65175 seconds 121.084 kpps 40.3084 kpps/GHz (clock independent) 104896/94794 (black wins / white wins) = Try 'help' I'd be curious g++ 4.4 what gives? Cheers, Adrian ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Digital Mars
I get g++-4.1 35 kpps/GHz g++-4.2 45 kpps/GHz g++-4.3 40 kpps/GHz I'm happy it's quite consistent on core2 I'm curious about 4.4 as well. Lukasz PS On Fri, Apr 24, 2009 at 01:29, Adrian Grajdeanu adria...@cox.net wrote: I have two benchmarks: On an: Intel(R) Core(TM)2 CPU T7200 @ 2.00GHz stepping 06 g++ --version g++ (GCC) 4.1.2 20070925 (Red Hat 4.1.2-33) I had to modify SConstruct to refer to the default g++, not g++.4.2 and had to remove -march=native = Benchmarking, please wait ... = 20 playouts in 2.72759 seconds 73.3249 kpps 36.5657 kpps/GHz (clock independent) 105316/94359 (black wins / white wins) = 20 playouts in 2.73858 seconds 73.0304 kpps 36.4108 kpps/GHz (clock independent) 104924/94746 (black wins / white wins) = 20 playouts in 2.72858 seconds 73.2981 kpps 36.5291 kpps/GHz (clock independent) 105097/94582 (black wins / white wins) = 20 playouts in 2.76258 seconds 72.3961 kpps 36.1141 kpps/GHz (clock independent) 105139/94547 (black wins / white wins) = 20 playouts in 2.74358 seconds 72.8974 kpps 36.3124 kpps/GHz (clock independent) 104896/94794 (black wins / white wins) = Try 'help' on an: Intel(R) Core(TM)2 Quad CPU Q9650 @ 3.00GHz stepping 0a g++ --version g++ (GCC) 4.3.0 20080428 (Red Hat 4.3.0-8) (with -march=native flag) = Benchmarking, please wait ... = 20 playouts in 1.65575 seconds 120.791 kpps 40.2566 kpps/GHz (clock independent) 105316/94359 (black wins / white wins) = 20 playouts in 1.65275 seconds 121.011 kpps 40.3069 kpps/GHz (clock independent) 104924/94746 (black wins / white wins) = 20 playouts in 1.65375 seconds 120.937 kpps 40.2789 kpps/GHz (clock independent) 105097/94582 (black wins / white wins) = 20 playouts in 1.65475 seconds 120.864 kpps 40.2917 kpps/GHz (clock independent) 105139/94547 (black wins / white wins) = 20 playouts in 1.65175 seconds 121.084 kpps 40.3084 kpps/GHz (clock independent) 104896/94794 (black wins / white wins) = Try 'help' I'd be curious g++ 4.4 what gives? Cheers, Adrian ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
RE: [computer-go] Roadmap 2020 - using analysis mode to improve programs
For example1, Many Faces' Game Score Graph shows the fight is over around move 208. From: computer-go-boun...@computer-go.org [mailto:computer-go-boun...@computer-go.org] On Behalf Of terry mcintyre Sent: Wednesday, April 22, 2009 6:27 PM To: computer-go Subject: Re: [computer-go] Roadmap 2020 - using analysis mode to improve programs I haven't got a ladder example at the moment, but here's an instance where Leela does not realize it is in terrible trouble. I ( with my 8 kyu AGA rating) know with certainty by move 223 (T5) that Black has captured a large white group. A stronger player could read this out sooner than I. This fight is too big to lose for either side; nothing else on the board matters. ( anyone? how early is this outcome pre-ordained? ) Based on the results of its analysis mode, Leela does not recognize the outcome of this semeai until the large white group in the bottom right is down to two liberties. The problem is even more stark in example2 -- similar board, black has foolishly played one of his own liberties for illustrative purposes. It is black's play, black has three liberties, white has three. Black must take away a liberty from white to win the capturing race, or make two eyes at T8. Black has only four playable moves; any other choice fails. Leela proposes - even after several minutes of analysis and a million nodes - that Black should tennuki at H14. That would snatch defeat from the jaws of certain victory; White would dive into T8 and win the race. I started this thread with the contention that analysis mode can help developers find problems, I hope this example explains why. My theory is that if a program could reliably recognize the outcome of such capturing races five or ten moves sooner, it could crush the likes of me. :D Terry McIntyre terrymcint...@yahoo.com Government is an association of men who do violence to the rest of us. - Leo Tolstoy _ From: Michael Williams michaelwilliam...@gmail.com To: computer-go computer-go@computer-go.org Sent: Tuesday, April 21, 2009 1:57:54 PM Subject: Re: [computer-go] Reply to Lukasz and Don + Roadmap 2020 Mention the program so that the author can either refute your claim or fix the bug. terry mcintyre wrote: Is it reasonable to expect pro players to use 6-dan programs as a tool for analysis? The pro players are markedly better - at a rough guess, a pro player could give a 6 dan amateur human or program a 3 stone handicap. On the other end of the scale, beginning players and mid kyu players could indeed make good use of an analysis mode by a program which is better than themselves. Lastly, an analysis mode would be helpful to developers, methinks. After winning a game, I like to back up a few moves and find out when the program realized that it was behind. This often happens several moves after the fatal blow has already been struck. I know the feeling too well, when stronger players deftly skewer my group and I only discover the problem five moves later. What do they know that I don't? What do they know that the program doesn't? We have a saying, you learn the most from reviewing games which you have lost. An analysis mode can help developers to discover when their pride and joy first begins to miss the target. Lately, I have been playing quite a bit with a commercially available program. An almost-ladder which has an extra liberty will apparently be evaluated the same as a true ladder, and the program can be tricked into trying to capture my ladder-like position. This sort of predictable flaw might provide a clue to improve the next version. Terry McIntyre terrymcint...@yahoo.com Government is an association of men who do violence to the rest of us. - Leo Tolstoy ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Roadmap 2020 - using analysis mode to improve programs
2009/4/23 terry mcintyre terrymcint...@yahoo.com: Programs which get semeai and seki right every time might be a few stones stronger. They'd certainly be more valuable as teaching tools. In the game above, a stronger program would have exploited my earlier weakness; this would have encouraged me to make better moves. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ Actually not. It seems so to a human who every now and then avoids loss by being better at these. But close semeais are rare. Sekis are rarer and program do not fail on theses every time. Go is a game where you can excel by making steady progress throughout the game without any brilliant moves. Also it is quite okay to compensate with other skill, I just played Mogo in KGS and got slaughtered after a careless cut. Well Killing and almost killing a group is where MC programs excel (relative to their strength) and those situations occur in almost every game.. I think semeai problem is easier to solve with: - Preanalysis by a classical go-algorithm. To my understanding this is what MFOG does - When we have even more CPU we can have even heavier playouts. Still an open issue whether smarter playout or more playouts is way to go. Although as I remember there were some mailing were it was mentioned cases where a smart playout could even hurt. -- Petri Pitkänen e-mail: petri.t.pitka...@gmail.com ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
