Re: [Computer-go] Detlef's DCNN data

[Hiroshi Yamashita]

Hi Aja,


Can you measure the prediction accuracy of the mixture (3)? I suspect its
prediction accuracy is lower than 44%.


Result is as follows.
Mix is highest. And CNN is not 44% but 39.3%. Maybe it is because
Detlef made CNN from KGS games, but I tested by pro games.
Christopher Clark's DCNN is 41.4% for pro games, 44.3% for KGS games.
It seems pro games is more difficult for DCNN.
Aya and CNN accuracy is similar, but CNN cumulative accuracy is much
better.
Detlef's CNN uses only black and white, two 19x19 binary. So I think
Aya's tactical features(string capture, ko threat) may help.


From pro 4000 games

   prediction accuracy 
Aya policy   38.8%

CNN policy   39.3%
Mixture  44.1%


Cumulative accuracy

move rank  Aya   CNN  Mixture
   1  38.8  39.3 44.1
   2  51.1  53.6 57.9
   3  57.8  62.0 65.7
   4  62.5  67.7 70.8
   5  65.9  71.9 74.6
   6  68.8  75.2 77.7
   7  71.1  78.0 80.3
   8  73.2  80.2 82.3
   9  74.8  82.1 84.0
  10  76.3  84.0 85.4
  11  77.6  85.4 86.7
  12  78.9  86.7 88.0
  13  80.0  87.7 89.0
  14  81.0  88.7 89.8
  15  82.0  89.6 90.6
  16  82.9  90.3 91.3
  17  83.7  91.0 91.9
  18  84.4  91.6 92.5
  19  85.1  92.2 93.0
  20  85.8  92.7 93.5

Regards,
Hiroshi Yamashita

- Original Message - 
From: "Aja Huang" 

To: 
Sent: Saturday, October 10, 2015 3:44 AM
Subject: Re: [Computer-go] Detlef's DCNN data



Hi Hiroshi,

On Fri, Oct 9, 2015 at 2:10 PM, Hiroshi Yamashita  wrote:


Mix MM gamma with CNN is clery stronger. But using only CNN is still
stronger than Aya without CNN. I had thought too small CNN's
probability was bad for Mix, but it is not.


Aya with CNN vs Aya without CNN, 1 playouts/move

winrate  wins/games min p
0.956 284/297(p < 0.0001 )  p*=50   Mix MM gamma with CNN's p
0.914  85/ 93(p < 0.001  )  p*=50   Mix MM gamma with CNN's p
0.923 275/298(p < 0.01   )  p*=50   Mix MM gamma with CNN's p
0.570 158/277(p < 0.1)  p*=1Use only CNN's p
0.720 206/286(p < 0.1)  p*=10   Use only CNN's p
0.777 226/286(p < 0.1)  p*=100  Use only CNN's p



Thanks for the results. Very interesting. I still don't understand how can
possibly the mixture can do better than CNN alone since CNN policy is much
stronger than Aya policy.

So now, you have three different policies:

1. Aya policy pi_aya, prediction accuracy is 35% if I remember correctly.
2. CNN policy pi_cnn, prediction accuracy is 44%.
3. Mixture of pi_aya and pi_cnn.

Can you measure the prediction accuracy of the mixture (3)? I suspect its
prediction accuracy is lower than 44%.

Aja



___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] MIG26: The Horizon Effect (Part 1)

[djhbrown .]

.
https://www.youtube.com/watch?v=syYFVSz8kNY=PL4y5WtsvtduqNW0AKlSsOdea3Hl1X_v-S=26
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go