Hi!
On Wed, Feb 03, 2016 at 10:24:51AM +0100, Robert Jasiek wrote:
> AlphaGo is said to predict 57% of professionals' moves. How is this number
> measured and from which sample?
>
> At some turns, there is only one correct move - at other turns, strong go
> players would say that there are several valid supposedly correct moves.
> This is one of the reasons why 100% cannot be the optimum but a smaller
> percentage must be the best.
>
> Pro players, or players of the database sample (incl. real world 3d players
> being 9d on KGS), make mistakes. A neural net learns from a sample and
> therefore also learns the mistakes. This is the most important reason why
> 100% cannot be the optimum but a smaller percentage must be the best.
>
> (Roughly) which percentage is optimal? Why? Is the optimum greater or
> smaller than 57%?
You are right about these questions. This is from the KGS dataset
http://u-go.net/gamerecords/ that contains 7d+ games. Yes, optimum is
very far below 100%; there was some discussion on this mailing list in
the past about checking strong human performance on this dataset, but
AFAIK nothing came out of that yet...
My impression is that current neural networks seem to converge to
about 60%. It would be interesting if humans can still do better.
Another idea would be considering accuracy as a top N measure among
selected moves, whether it has better discernive power for currently
used models.
--
Petr Baudis
If you have good ideas, good data and fast computers,
you can do almost anything. -- Geoffrey Hinton
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
