I am sorry, but I think this discussion is a bit pointless.
While I write these 3 lines and you read them, AlphGo got 20 ELO
points stronger. :-)
Thomas
On Tue, 22 Mar 2016, Lucas, Simon M wrote:
Still an interesting question is how one could make
more powerful inferences by observing the skill of
the players in each action they take rather than just
the final outcome of each game.
If you saw me play a single game of tennis against Federer
you’d have no doubt as to which way the next 100 games would go.
From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of
Álvaro Begué
Sent: 22 March 2016 17:21
To: computer-go <computer-go@computer-go.org>
Subject: Re: [Computer-go] Congratulations to AlphaGo (Statistical significance
of results)
A very simple-minded analysis is that, if the null hypothesis is that AlphaGo
and Lee Sedol are
equally strong, AlphaGo would do as well as we observed or better 15.625% of
the time. That's a
p-value that even social scientists don't get excited about. :)
Álvaro.
On Tue, Mar 22, 2016 at 12:48 PM, Jason House <jason.james.ho...@gmail.com>
wrote:
Statistical significance requires a null hypothesis... I think it's
probably easiest to
ask the question of if I assume an ELO difference of x, how likely it's a
4-1 result?
Turns out that 220 to 270 ELO has a 41% chance of that result.
>= 10% is -50 to 670 ELO
>= 1% is -250 to 1190 ELO
My numbers may be slightly off from eyeballing things in a simple excel
sheet. The idea
and ranges should be clear though
On Mar 22, 2016 12:00 PM, "Lucas, Simon M" <s...@essex.ac.uk> wrote:
Hi all,
I was discussing the results with a colleague outside
of the Game AI area the other day when he raised
the question (which applies to nearly all sporting events,
given the small sample size involved)
of statistical significance - suggesting that on another week
the result might have been 4-1 to Lee Sedol.
I pointed out that in games of skill there's much more to judge
than just the
final
outcome of each game, but wondered if anyone had any better (or
worse :)
arguments - or had even engaged in the same type of
conversation.
With AlphaGo winning 4 games to 1, from a simplistic
stats point of view (with the prior assumption of a fair
coin toss) you'd not be able to claim much statistical
significance, yet most (me included) believe that
AlphaGo is a genuinely better Go player than Lee Sedol.
From a stats viewpoint you can use this approach:
http://www.inference.phy.cam.ac.uk/itprnn/book.pdf
(see section 3.2 on page 51)
but given even priors it won't tell you much.
Anyone know any good references for refuting this
type of argument - the fact is of course that a game of Go
is nothing like a coin toss. Games of skill tend to base their
outcomes on the result of many (in the case of Go many hundreds of)
individual actions.
Best wishes,
Simon
_______________________________________________
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
_______________________________________________
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
_______________________________________________
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
