Re: [Computer-go] CPU vs GPU

[Detlef Schmicker]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

you can use caffe with time on the command line.

It gives you forward and backward time for a batch.

In my tests the batch size was not too important (I think, because the
net is quite large)...

cuDNN helps a lot in training, I did not test recently but it was 2
times faster end of last year and improved by a factor of 4 during
this year :)

Detlef

Am 02.03.2016 um 10:22 schrieb Rémi Coulom:
> I tried Detlef's 54% NN on my machine. CPU = i7-5930K, GPU = GTX 
> 980 (not using cuDNN).
> 
> On the CPU, I get 176 ms time, and 10 ms on the GPU (IIRC, someone
>  reported 6 ms with cuDNN). But it is using only one core on the 
> CPU, whereas it is using the full GPU.
> 
> If this is correct, then I believe it is still possible to have a 
> very strong CPU-based program.
> 
> Or is it possible to evaluate faster on the GPU by using a batch?
> 
> Rémi
> 
> On 03/02/2016 09:43 AM, Petr Baudis wrote:
>> Also, reading more of that pull request, the guy benchmarking it 
>> had old nvidia driver version which came with about 50% 
>> performance hit.  So I'm not sure what were the final numbers. 
>> (And whether current caffe version can actually match these 
>> numbers, since this pull request wasn't merged.)
>> 
>> On Wed, Mar 02, 2016 at 12:29:41AM -0800, Chaz G. wrote:
>>> Rémi,
>>> 
>>> Nvidia launched the K20 GPU in late 2012. Since then, GPUs and 
>>> their convolution algorithms have improved considerably, while 
>>> CPU performance has been relatively stagnant. I would expect 
>>> about a 10x improvement with 2016 hardware.
>>> 
>>> When it comes to training, it's the difference between running 
>>> a job overnight and running a job for the entire weekend.
>>> 
>>> Best, -Chaz
>>> 
>>> On Tue, Mar 1, 2016 at 1:03 PM, Rémi Coulom 
>>>  wrote:
>>> 
 How tremendous is it? On that page, I find this data:
 
 https://github.com/BVLC/caffe/pull/439
 
 " These are setup details:
 
 * Desktop: CPU i7-4770 (Haswell), 3.5 GHz , DRAM - 16 GB;
 GPU K20. * Ubuntu 12.04; gcc 4.7.3; MKL 11.1.
 
 Test:: imagenet, 100 train iteration (batch = 256).
 
 * GPU: time= 260 sec / memory = 0.8 GB * CPU: time= 752 sec
 / memory = 3.5 GiB //Memory data is from system monitor.
 
 "
 
 This does not look so tremendous to me. What kind of speed 
 difference do you get for Go networks?
 
 Rémi
 
 On 03/01/2016 06:19 PM, Petr Baudis wrote:
 
> On Tue, Mar 01, 2016 at 09:14:39AM -0800, David Fotland 
> wrote:
> 
>> Very interesting, but it should also mention Aya.
>> 
>> I'm working on this as well, but I haven’t bought any 
>> hardware yet.  My goal is not to get 7 dan on expensive 
>> hardware, but to get as much strength as I can on 
>> standard PC hardware.  I'll be looking at much smaller 
>> nets, that don’t need a GPU to run.  I'll have to buy a 
>> GPU for training.
>> 
> But I think most people who play Go are also fans of 
> computer games that often do use GPUs. :-)  Of course,
> it's something totally different from NVidia Keplers, but
> still the step up from a CPU is tremendous.
> 
> Petr Baudis ___
> Computer-go mailing list Computer-go@computer-go.org 
> http://computer-go.org/mailman/listinfo/computer-go
> 
 ___ Computer-go 
 mailing list Computer-go@computer-go.org 
 http://computer-go.org/mailman/listinfo/computer-go
>>> ___ Computer-go 
>>> mailing list Computer-go@computer-go.org 
>>> http://computer-go.org/mailman/listinfo/computer-go
>> 
> 
> ___ Computer-go
> mailing list Computer-go@computer-go.org 
> http://computer-go.org/mailman/listinfo/computer-go
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJW2JVUAAoJEInWdHg+Znf4JGEQAIcuAbKkEihOhjKoH4fXKr1C
zsN0nVX5Viyca9Ca8NDxBb8xL1w1MYQl8oO2yiljwZKD6GQiSGV2d+jQTI42HCXn
xPn88O9nkzOJh6OKyNmLxZvj/A9XQjFTaDkPgaO0NejoAytdQEjXZrZlDB1dLjvJ
5GYtIZCXpCPsmgV25Iz8Hs1lsMFbUy4GrTx6m+htd3iAZiL+ZmaFMh+aYHVVn9ea
j66EAevu3klzCiWoO8A+s0onpvw05LM9gSpC1yPqqR55IOY1ikEAh7MqUSdStz/9
xt7YUdtXm5lrzEIRIMvhDec/FsnVYfmeRE6lKxBus95Y6zcwx9HvwtDZBDWzjIsl
4tgO86N8saK5eG1PqAPmRkg5abp0r3JG7NRCoI/47T9i3NkYC+RtCVZCo0tojzkW
gY28blxHtG5yQscob6D1T/2597VWetXBghzcnXCzLe0+aJS2p+S5klW3Q/B4o3t+
fKqICpMQrNzJO3vagefqYkRCzIUdUVXH4/rAnUnu3+pY75iTPNzqAK1PtdYcAQYt
XDGUIWH0+Ir9Wk20EzM3dXDR/7rKbHgeuBbIEP4+LnEQsNJ7UE6YSFJHhdJTNGGM
5LXeXJv6YQQO3Gj9yQYuMTZFploVhJKAFf4vIloYwG2ZKE1SkU3aBQsudCr5uhc1
5XK2WAy4gKl/Ih0KxZLm
=uWVF
-END PGP SIGNATURE-
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] CPU vs GPU

[David Fotland]

If you are using caffe, the network evaluator is single threaded, but it spends 
almost all of its time in BLAS, which uses one thread per virtual CPU.On a 
somewhat slower i7, I’m seeing about 200 ms.

David

> -Original Message-
> From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf
> Of Rémi Coulom
> Sent: Wednesday, March 02, 2016 1:23 AM
> To: computer-go@computer-go.org
> Subject: Re: [Computer-go] CPU vs GPU
> 
> I tried Detlef's 54% NN on my machine. CPU = i7-5930K, GPU = GTX 980
> (not using cuDNN).
> 
> On the CPU, I get 176 ms time, and 10 ms on the GPU (IIRC, someone
> reported 6 ms with cuDNN). But it is using only one core on the CPU,
> whereas it is using the full GPU.
> 
> If this is correct, then I believe it is still possible to have a very
> strong CPU-based program.
> 
> Or is it possible to evaluate faster on the GPU by using a batch?
> 
> R mi
> 
> On 03/02/2016 09:43 AM, Petr Baudis wrote:
> > Also, reading more of that pull request, the guy benchmarking it had
> > old nvidia driver version which came with about 50% performance hit.
> > So I'm not sure what were the final numbers.  (And whether current
> > caffe version can actually match these numbers, since this pull
> > request wasn't
> > merged.)
> >
> > On Wed, Mar 02, 2016 at 12:29:41AM -0800, Chaz G. wrote:
> >> R mi,
> >>
> >> Nvidia launched the K20 GPU in late 2012. Since then, GPUs and their
> >> convolution algorithms have improved considerably, while CPU
> >> performance has been relatively stagnant. I would expect about a 10x
> >> improvement with
> >> 2016 hardware.
> >>
> >> When it comes to training, it's the difference between running a job
> >> overnight and running a job for the entire weekend.
> >>
> >> Best,
> >> -Chaz
> >>
> >> On Tue, Mar 1, 2016 at 1:03 PM, R mi Coulom 
> wrote:
> >>
> >>> How tremendous is it? On that page, I find this data:
> >>>
> >>> https://github.com/BVLC/caffe/pull/439
> >>>
> >>> "
> >>> These are setup details:
> >>>
> >>>   * Desktop: CPU i7-4770 (Haswell), 3.5 GHz , DRAM - 16 GB; GPU K20.
> >>>   * Ubuntu 12.04; gcc 4.7.3; MKL 11.1.
> >>>
> >>> Test:: imagenet, 100 train iteration (batch = 256).
> >>>
> >>>   * GPU: time= 260 sec / memory = 0.8 GB
> >>>   * CPU: time= 752 sec / memory = 3.5 GiB //Memory data is from
> system
> >>> monitor.
> >>>
> >>> "
> >>>
> >>> This does not look so tremendous to me. What kind of speed
> >>> difference do you get for Go networks?
> >>>
> >>> R mi
> >>>
> >>> On 03/01/2016 06:19 PM, Petr Baudis wrote:
> >>>
>  On Tue, Mar 01, 2016 at 09:14:39AM -0800, David Fotland wrote:
> 
> > Very interesting, but it should also mention Aya.
> >
> > I'm working on this as well, but I haven t bought any hardware
> > yet.  My goal is not to get 7 dan on expensive hardware, but to
> > get as much strength as I can on standard PC hardware.  I'll be
> > looking at much smaller nets, that don t need a GPU to run.  I'll
> have to buy a GPU for training.
> >
>  But I think most people who play Go are also fans of computer games
>  that often do use GPUs. :-)  Of course, it's something totally
>  different from NVidia Keplers, but still the step up from a CPU is
> tremendous.
> 
>   Petr Baudis
>  ___
>  Computer-go mailing list
>  Computer-go@computer-go.org
>  http://computer-go.org/mailman/listinfo/computer-go
> 
> >>> ___
> >>> Computer-go mailing list
> >>> Computer-go@computer-go.org
> >>> http://computer-go.org/mailman/listinfo/computer-go
> >> ___
> >> Computer-go mailing list
> >> Computer-go@computer-go.org
> >> http://computer-go.org/mailman/listinfo/computer-go
> >
> 
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Deep Learning learning resources?

[David Fotland]

I got the basics of Machine learning (including sample neural nets) from Andrew 
Ng's course course, two or three years ago.  I highly recommend it.  Lots of 
practical advice.  The rest came from reading papers and probably some on-line 
searches.  Amazon's Computer Vision team uses deep neural nets.  I've talked to 
some of them and attended some internal presentations.  I'm using caffe, so I 
don’t need to implement the network code itself.  Mostly I looked for practical 
advice about network organizations that work.

David

> -Original Message-
> From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf
> Of Darren Cook
> Sent: Wednesday, March 02, 2016 1:53 AM
> To: computer-go@computer-go.org
> Subject: [Computer-go] Deep Learning learning resources?
> 
> I'm sure quite a few people here have suddenly taken a look at neural
> nets the past few months. With hindsight where have you learnt most?
> Which is the most useful book you've read? Is there a Udacity (or
> similar) course that you recommend? Or perhaps a blog or youtube series
> that was so good you went back and read/viewed all the archives?
> 
> Thanks!
> Darren
> 
> P.S. I was thinking pragmatic, and general, how-to guides for people
> dealing with challenging problems similar to computer go, but if you
> have recommendations for latest academic theories, or for a very
> specific field, I'm sure someone would appreciate hearing it.
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go