Re: [Computer-go] CPU vs GPU
I checked again, and it is using only one single thread for everything in CPU mode. I simply use caffe::Caffe::set_mode(caffe::Caffe::CPU), and it is using only one core (I checked with top in a long loop of calls to the network). Rémi On 03/03/2016 09:19 AM, David Fotland wrote: If you are using caffe, the network evaluator is single threaded, but it spends almost all of its time in BLAS, which uses one thread per virtual CPU.On a somewhat slower i7, I’m seeing about 200 ms. David -Original Message- From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf Of Rémi Coulom Sent: Wednesday, March 02, 2016 1:23 AM To: computer-go@computer-go.org Subject: Re: [Computer-go] CPU vs GPU I tried Detlef's 54% NN on my machine. CPU = i7-5930K, GPU = GTX 980 (not using cuDNN). On the CPU, I get 176 ms time, and 10 ms on the GPU (IIRC, someone reported 6 ms with cuDNN). But it is using only one core on the CPU, whereas it is using the full GPU. If this is correct, then I believe it is still possible to have a very strong CPU-based program. Or is it possible to evaluate faster on the GPU by using a batch? R mi On 03/02/2016 09:43 AM, Petr Baudis wrote: Also, reading more of that pull request, the guy benchmarking it had old nvidia driver version which came with about 50% performance hit. So I'm not sure what were the final numbers. (And whether current caffe version can actually match these numbers, since this pull request wasn't merged.) On Wed, Mar 02, 2016 at 12:29:41AM -0800, Chaz G. wrote: R mi, Nvidia launched the K20 GPU in late 2012. Since then, GPUs and their convolution algorithms have improved considerably, while CPU performance has been relatively stagnant. I would expect about a 10x improvement with 2016 hardware. When it comes to training, it's the difference between running a job overnight and running a job for the entire weekend. Best, -Chaz On Tue, Mar 1, 2016 at 1:03 PM, R mi Coulom <remi.cou...@free.fr> wrote: How tremendous is it? On that page, I find this data: https://github.com/BVLC/caffe/pull/439 " These are setup details: * Desktop: CPU i7-4770 (Haswell), 3.5 GHz , DRAM - 16 GB; GPU K20. * Ubuntu 12.04; gcc 4.7.3; MKL 11.1. Test:: imagenet, 100 train iteration (batch = 256). * GPU: time= 260 sec / memory = 0.8 GB * CPU: time= 752 sec / memory = 3.5 GiB //Memory data is from system monitor. " This does not look so tremendous to me. What kind of speed difference do you get for Go networks? R mi On 03/01/2016 06:19 PM, Petr Baudis wrote: On Tue, Mar 01, 2016 at 09:14:39AM -0800, David Fotland wrote: Very interesting, but it should also mention Aya. I'm working on this as well, but I haven t bought any hardware yet. My goal is not to get 7 dan on expensive hardware, but to get as much strength as I can on standard PC hardware. I'll be looking at much smaller nets, that don t need a GPU to run. I'll have to buy a GPU for training. But I think most people who play Go are also fans of computer games that often do use GPUs. :-) Of course, it's something totally different from NVidia Keplers, but still the step up from a CPU is tremendous. Petr Baudis ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] CPU vs GPU
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 you can use caffe with time on the command line. It gives you forward and backward time for a batch. In my tests the batch size was not too important (I think, because the net is quite large)... cuDNN helps a lot in training, I did not test recently but it was 2 times faster end of last year and improved by a factor of 4 during this year :) Detlef Am 02.03.2016 um 10:22 schrieb Rémi Coulom: > I tried Detlef's 54% NN on my machine. CPU = i7-5930K, GPU = GTX > 980 (not using cuDNN). > > On the CPU, I get 176 ms time, and 10 ms on the GPU (IIRC, someone > reported 6 ms with cuDNN). But it is using only one core on the > CPU, whereas it is using the full GPU. > > If this is correct, then I believe it is still possible to have a > very strong CPU-based program. > > Or is it possible to evaluate faster on the GPU by using a batch? > > Rémi > > On 03/02/2016 09:43 AM, Petr Baudis wrote: >> Also, reading more of that pull request, the guy benchmarking it >> had old nvidia driver version which came with about 50% >> performance hit. So I'm not sure what were the final numbers. >> (And whether current caffe version can actually match these >> numbers, since this pull request wasn't merged.) >> >> On Wed, Mar 02, 2016 at 12:29:41AM -0800, Chaz G. wrote: >>> Rémi, >>> >>> Nvidia launched the K20 GPU in late 2012. Since then, GPUs and >>> their convolution algorithms have improved considerably, while >>> CPU performance has been relatively stagnant. I would expect >>> about a 10x improvement with 2016 hardware. >>> >>> When it comes to training, it's the difference between running >>> a job overnight and running a job for the entire weekend. >>> >>> Best, -Chaz >>> >>> On Tue, Mar 1, 2016 at 1:03 PM, Rémi Coulom >>>wrote: >>> How tremendous is it? On that page, I find this data: https://github.com/BVLC/caffe/pull/439 " These are setup details: * Desktop: CPU i7-4770 (Haswell), 3.5 GHz , DRAM - 16 GB; GPU K20. * Ubuntu 12.04; gcc 4.7.3; MKL 11.1. Test:: imagenet, 100 train iteration (batch = 256). * GPU: time= 260 sec / memory = 0.8 GB * CPU: time= 752 sec / memory = 3.5 GiB //Memory data is from system monitor. " This does not look so tremendous to me. What kind of speed difference do you get for Go networks? Rémi On 03/01/2016 06:19 PM, Petr Baudis wrote: > On Tue, Mar 01, 2016 at 09:14:39AM -0800, David Fotland > wrote: > >> Very interesting, but it should also mention Aya. >> >> I'm working on this as well, but I haven’t bought any >> hardware yet. My goal is not to get 7 dan on expensive >> hardware, but to get as much strength as I can on >> standard PC hardware. I'll be looking at much smaller >> nets, that don’t need a GPU to run. I'll have to buy a >> GPU for training. >> > But I think most people who play Go are also fans of > computer games that often do use GPUs. :-) Of course, > it's something totally different from NVidia Keplers, but > still the step up from a CPU is tremendous. > > Petr Baudis ___ > Computer-go mailing list Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go >>> ___ Computer-go >>> mailing list Computer-go@computer-go.org >>> http://computer-go.org/mailman/listinfo/computer-go >> > > ___ Computer-go > mailing list Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJW2JVUAAoJEInWdHg+Znf4JGEQAIcuAbKkEihOhjKoH4fXKr1C zsN0nVX5Viyca9Ca8NDxBb8xL1w1MYQl8oO2yiljwZKD6GQiSGV2d+jQTI42HCXn xPn88O9nkzOJh6OKyNmLxZvj/A9XQjFTaDkPgaO0NejoAytdQEjXZrZlDB1dLjvJ 5GYtIZCXpCPsmgV25Iz8Hs1lsMFbUy4GrTx6m+htd3iAZiL+ZmaFMh+aYHVVn9ea j66EAevu3klzCiWoO8A+s0onpvw05LM9gSpC1yPqqR55IOY1ikEAh7MqUSdStz/9 xt7YUdtXm5lrzEIRIMvhDec/FsnVYfmeRE6lKxBus95Y6zcwx9HvwtDZBDWzjIsl 4tgO86N8saK5eG1PqAPmRkg5abp0r3JG7NRCoI/47T9i3NkYC+RtCVZCo0tojzkW gY28blxHtG5yQscob6D1T/2597VWetXBghzcnXCzLe0+aJS2p+S5klW3Q/B4o3t+ fKqICpMQrNzJO3vagefqYkRCzIUdUVXH4/rAnUnu3+pY75iTPNzqAK1PtdYcAQYt XDGUIWH0+Ir9Wk20EzM3dXDR/7rKbHgeuBbIEP4+LnEQsNJ7UE6YSFJHhdJTNGGM 5LXeXJv6YQQO3Gj9yQYuMTZFploVhJKAFf4vIloYwG2ZKE1SkU3aBQsudCr5uhc1 5XK2WAy4gKl/Ih0KxZLm =uWVF -END PGP SIGNATURE- ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] CPU vs GPU
If you are using caffe, the network evaluator is single threaded, but it spends almost all of its time in BLAS, which uses one thread per virtual CPU.On a somewhat slower i7, I’m seeing about 200 ms. David > -Original Message- > From: Computer-go [mailto:computer-go-boun...@computer-go.org] On Behalf > Of Rémi Coulom > Sent: Wednesday, March 02, 2016 1:23 AM > To: computer-go@computer-go.org > Subject: Re: [Computer-go] CPU vs GPU > > I tried Detlef's 54% NN on my machine. CPU = i7-5930K, GPU = GTX 980 > (not using cuDNN). > > On the CPU, I get 176 ms time, and 10 ms on the GPU (IIRC, someone > reported 6 ms with cuDNN). But it is using only one core on the CPU, > whereas it is using the full GPU. > > If this is correct, then I believe it is still possible to have a very > strong CPU-based program. > > Or is it possible to evaluate faster on the GPU by using a batch? > > R mi > > On 03/02/2016 09:43 AM, Petr Baudis wrote: > > Also, reading more of that pull request, the guy benchmarking it had > > old nvidia driver version which came with about 50% performance hit. > > So I'm not sure what were the final numbers. (And whether current > > caffe version can actually match these numbers, since this pull > > request wasn't > > merged.) > > > > On Wed, Mar 02, 2016 at 12:29:41AM -0800, Chaz G. wrote: > >> R mi, > >> > >> Nvidia launched the K20 GPU in late 2012. Since then, GPUs and their > >> convolution algorithms have improved considerably, while CPU > >> performance has been relatively stagnant. I would expect about a 10x > >> improvement with > >> 2016 hardware. > >> > >> When it comes to training, it's the difference between running a job > >> overnight and running a job for the entire weekend. > >> > >> Best, > >> -Chaz > >> > >> On Tue, Mar 1, 2016 at 1:03 PM, R mi Coulom <remi.cou...@free.fr> > wrote: > >> > >>> How tremendous is it? On that page, I find this data: > >>> > >>> https://github.com/BVLC/caffe/pull/439 > >>> > >>> " > >>> These are setup details: > >>> > >>> * Desktop: CPU i7-4770 (Haswell), 3.5 GHz , DRAM - 16 GB; GPU K20. > >>> * Ubuntu 12.04; gcc 4.7.3; MKL 11.1. > >>> > >>> Test:: imagenet, 100 train iteration (batch = 256). > >>> > >>> * GPU: time= 260 sec / memory = 0.8 GB > >>> * CPU: time= 752 sec / memory = 3.5 GiB //Memory data is from > system > >>> monitor. > >>> > >>> " > >>> > >>> This does not look so tremendous to me. What kind of speed > >>> difference do you get for Go networks? > >>> > >>> R mi > >>> > >>> On 03/01/2016 06:19 PM, Petr Baudis wrote: > >>> > >>>> On Tue, Mar 01, 2016 at 09:14:39AM -0800, David Fotland wrote: > >>>> > >>>>> Very interesting, but it should also mention Aya. > >>>>> > >>>>> I'm working on this as well, but I haven t bought any hardware > >>>>> yet. My goal is not to get 7 dan on expensive hardware, but to > >>>>> get as much strength as I can on standard PC hardware. I'll be > >>>>> looking at much smaller nets, that don t need a GPU to run. I'll > have to buy a GPU for training. > >>>>> > >>>> But I think most people who play Go are also fans of computer games > >>>> that often do use GPUs. :-) Of course, it's something totally > >>>> different from NVidia Keplers, but still the step up from a CPU is > tremendous. > >>>> > >>>> Petr Baudis > >>>> ___ > >>>> Computer-go mailing list > >>>> Computer-go@computer-go.org > >>>> http://computer-go.org/mailman/listinfo/computer-go > >>>> > >>> ___ > >>> Computer-go mailing list > >>> Computer-go@computer-go.org > >>> http://computer-go.org/mailman/listinfo/computer-go > >> ___ > >> Computer-go mailing list > >> Computer-go@computer-go.org > >> http://computer-go.org/mailman/listinfo/computer-go > > > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] CPU vs GPU
I tried Detlef's 54% NN on my machine. CPU = i7-5930K, GPU = GTX 980 (not using cuDNN). On the CPU, I get 176 ms time, and 10 ms on the GPU (IIRC, someone reported 6 ms with cuDNN). But it is using only one core on the CPU, whereas it is using the full GPU. If this is correct, then I believe it is still possible to have a very strong CPU-based program. Or is it possible to evaluate faster on the GPU by using a batch? Rémi On 03/02/2016 09:43 AM, Petr Baudis wrote: Also, reading more of that pull request, the guy benchmarking it had old nvidia driver version which came with about 50% performance hit. So I'm not sure what were the final numbers. (And whether current caffe version can actually match these numbers, since this pull request wasn't merged.) On Wed, Mar 02, 2016 at 12:29:41AM -0800, Chaz G. wrote: Rémi, Nvidia launched the K20 GPU in late 2012. Since then, GPUs and their convolution algorithms have improved considerably, while CPU performance has been relatively stagnant. I would expect about a 10x improvement with 2016 hardware. When it comes to training, it's the difference between running a job overnight and running a job for the entire weekend. Best, -Chaz On Tue, Mar 1, 2016 at 1:03 PM, Rémi Coulomwrote: How tremendous is it? On that page, I find this data: https://github.com/BVLC/caffe/pull/439 " These are setup details: * Desktop: CPU i7-4770 (Haswell), 3.5 GHz , DRAM - 16 GB; GPU K20. * Ubuntu 12.04; gcc 4.7.3; MKL 11.1. Test:: imagenet, 100 train iteration (batch = 256). * GPU: time= 260 sec / memory = 0.8 GB * CPU: time= 752 sec / memory = 3.5 GiB //Memory data is from system monitor. " This does not look so tremendous to me. What kind of speed difference do you get for Go networks? Rémi On 03/01/2016 06:19 PM, Petr Baudis wrote: On Tue, Mar 01, 2016 at 09:14:39AM -0800, David Fotland wrote: Very interesting, but it should also mention Aya. I'm working on this as well, but I haven’t bought any hardware yet. My goal is not to get 7 dan on expensive hardware, but to get as much strength as I can on standard PC hardware. I'll be looking at much smaller nets, that don’t need a GPU to run. I'll have to buy a GPU for training. But I think most people who play Go are also fans of computer games that often do use GPUs. :-) Of course, it's something totally different from NVidia Keplers, but still the step up from a CPU is tremendous. Petr Baudis ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] CPU vs GPU
Also, reading more of that pull request, the guy benchmarking it had old nvidia driver version which came with about 50% performance hit. So I'm not sure what were the final numbers. (And whether current caffe version can actually match these numbers, since this pull request wasn't merged.) On Wed, Mar 02, 2016 at 12:29:41AM -0800, Chaz G. wrote: > Rémi, > > Nvidia launched the K20 GPU in late 2012. Since then, GPUs and their > convolution algorithms have improved considerably, while CPU performance > has been relatively stagnant. I would expect about a 10x improvement with > 2016 hardware. > > When it comes to training, it's the difference between running a job > overnight and running a job for the entire weekend. > > Best, > -Chaz > > On Tue, Mar 1, 2016 at 1:03 PM, Rémi Coulomwrote: > > > How tremendous is it? On that page, I find this data: > > > > https://github.com/BVLC/caffe/pull/439 > > > > " > > These are setup details: > > > > * Desktop: CPU i7-4770 (Haswell), 3.5 GHz , DRAM - 16 GB; GPU K20. > > * Ubuntu 12.04; gcc 4.7.3; MKL 11.1. > > > > Test:: imagenet, 100 train iteration (batch = 256). > > > > * GPU: time= 260 sec / memory = 0.8 GB > > * CPU: time= 752 sec / memory = 3.5 GiB //Memory data is from system > >monitor. > > > > " > > > > This does not look so tremendous to me. What kind of speed difference do > > you get for Go networks? > > > > Rémi > > > > On 03/01/2016 06:19 PM, Petr Baudis wrote: > > > >> On Tue, Mar 01, 2016 at 09:14:39AM -0800, David Fotland wrote: > >> > >>> Very interesting, but it should also mention Aya. > >>> > >>> I'm working on this as well, but I haven’t bought any hardware yet. My > >>> goal is not to get 7 dan on expensive hardware, but to get as much > >>> strength > >>> as I can on standard PC hardware. I'll be looking at much smaller nets, > >>> that don’t need a GPU to run. I'll have to buy a GPU for training. > >>> > >> But I think most people who play Go are also fans of computer games that > >> often do use GPUs. :-) Of course, it's something totally different from > >> NVidia Keplers, but still the step up from a CPU is tremendous. > >> > >> Petr Baudis > >> ___ > >> Computer-go mailing list > >> Computer-go@computer-go.org > >> http://computer-go.org/mailman/listinfo/computer-go > >> > > > > ___ > > Computer-go mailing list > > Computer-go@computer-go.org > > http://computer-go.org/mailman/listinfo/computer-go > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go -- Petr Baudis If you have good ideas, good data and fast computers, you can do almost anything. -- Geoffrey Hinton ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] CPU vs GPU
Rémi, Nvidia launched the K20 GPU in late 2012. Since then, GPUs and their convolution algorithms have improved considerably, while CPU performance has been relatively stagnant. I would expect about a 10x improvement with 2016 hardware. When it comes to training, it's the difference between running a job overnight and running a job for the entire weekend. Best, -Chaz On Tue, Mar 1, 2016 at 1:03 PM, Rémi Coulomwrote: > How tremendous is it? On that page, I find this data: > > https://github.com/BVLC/caffe/pull/439 > > " > These are setup details: > > * Desktop: CPU i7-4770 (Haswell), 3.5 GHz , DRAM - 16 GB; GPU K20. > * Ubuntu 12.04; gcc 4.7.3; MKL 11.1. > > Test:: imagenet, 100 train iteration (batch = 256). > > * GPU: time= 260 sec / memory = 0.8 GB > * CPU: time= 752 sec / memory = 3.5 GiB //Memory data is from system >monitor. > > " > > This does not look so tremendous to me. What kind of speed difference do > you get for Go networks? > > Rémi > > On 03/01/2016 06:19 PM, Petr Baudis wrote: > >> On Tue, Mar 01, 2016 at 09:14:39AM -0800, David Fotland wrote: >> >>> Very interesting, but it should also mention Aya. >>> >>> I'm working on this as well, but I haven’t bought any hardware yet. My >>> goal is not to get 7 dan on expensive hardware, but to get as much strength >>> as I can on standard PC hardware. I'll be looking at much smaller nets, >>> that don’t need a GPU to run. I'll have to buy a GPU for training. >>> >> But I think most people who play Go are also fans of computer games that >> often do use GPUs. :-) Of course, it's something totally different from >> NVidia Keplers, but still the step up from a CPU is tremendous. >> >> Petr Baudis >> ___ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go >> > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
[Computer-go] CPU vs GPU
How tremendous is it? On that page, I find this data: https://github.com/BVLC/caffe/pull/439 " These are setup details: * Desktop: CPU i7-4770 (Haswell), 3.5 GHz , DRAM - 16 GB; GPU K20. * Ubuntu 12.04; gcc 4.7.3; MKL 11.1. Test:: imagenet, 100 train iteration (batch = 256). * GPU: time= 260 sec / memory = 0.8 GB * CPU: time= 752 sec / memory = 3.5 GiB //Memory data is from system monitor. " This does not look so tremendous to me. What kind of speed difference do you get for Go networks? Rémi On 03/01/2016 06:19 PM, Petr Baudis wrote: On Tue, Mar 01, 2016 at 09:14:39AM -0800, David Fotland wrote: Very interesting, but it should also mention Aya. I'm working on this as well, but I haven’t bought any hardware yet. My goal is not to get 7 dan on expensive hardware, but to get as much strength as I can on standard PC hardware. I'll be looking at much smaller nets, that don’t need a GPU to run. I'll have to buy a GPU for training. But I think most people who play Go are also fans of computer games that often do use GPUs. :-) Of course, it's something totally different from NVidia Keplers, but still the step up from a CPU is tremendous. Petr Baudis ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go