I suppose that your GPU is shared with the monitor? If so, this is normal that you can't raise the value too high. This % is the % of the total GPU memory. Not of the free memory. So if we take too much, then it don't let enought space for cudnn internal stuff.
So mostly, don't use a too high cnmem value :) Fred On Wed, Aug 3, 2016 at 10:05 AM, BorjaH_85 <[email protected]> wrote: > Hi all, > > I have been doing a benchmark regarding the gpu-based execution of Theano > (0.8.2) and Keras (1.0.5) on Windows 10. I am comparing the training time > of a CNN over the MNIST dataset in 3 scenarios: CPU, GPU without cuDNN and > GPU with CUDNN. CUDA v7.5 and cuDNN v5.0 > > First two approaches go well, but I have gone into troubles with the > sysenv variables configuration for the GPU+cuDNN trial. My error source is > the CNMeM configuration. If I run the same python code with CNMeM values > from 0 (disbaled) up to 0.75, I have no problems. Moreover, I obtain a drop > in the training time of more than 50% compared to the general GPU execution. > > But I cannot go behind from 0.8. With 0.8, I receive the same compilation > error. > > RuntimeError: ('The following error happened while compiling the node', > GpuDnnConv{algo='small', inplace=True}(GpuContiguous.0, GpuContiguous.0, > GpuAllocEmpty.0, GpuDnnConvDesc{border_mode='valid', subsample=(1, 1), > conv_mode='conv', precision='float32'}.0, Constant{1.0}, Constant{0.0}), > '\n', 'could not create cuDNN handle: CUDNN_STATUS_INTERNAL_ERROR') > > Best regards, > > Borja > > -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
