Hi Fred, I tested the convnet using floatX= float32, device=gpu theano.tensor.nnet.conv3d2d.conv3d updated theano/sandbox/cuda/blas.py downloaded from https://github.com/Theano/Theano/pull/5050 <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2FTheano%2FTheano%2Fpull%2F5050&sa=D&sntz=1&usg=AFQjCNFR8fDmqOsSdghS2mC-7R90hku9Rw>
The convnet converges: Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul 2 2016, 17:42:40) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://anaconda.org >>> runfile('/home/luca/data/DeepLearningTutorials/Theano-3D-ConvNet-master/convnet3d/core/run_multi_conv_t.py', wdir='/home/luca/data/DeepLearningTutorials/Theano-3D-ConvNet-master/convnet3d/core') Using gpu device 0: GeForce 840M (CNMeM is disabled, cuDNN 5103) /home/luca/data/Theano-master/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module. "downsample module has been moved to the theano.tensor.signal.pool module.") start time: 04/10/2016 16:18:13 Images for training: 316 Images for validation: 56 training @ iter = 0 training cost 0.69672 epoch 1, training batch 316/316, validation error 37.500 % ------ If I make the same test using: floatX = float16 device=gpu theano.tensor.nnet.conv3d2d.conv3d updated theano/sandbox/cuda/blas.py downloaded from https://github.com/Theano/Theano/pull/5050 <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2FTheano%2FTheano%2Fpull%2F5050&sa=D&sntz=1&usg=AFQjCNFR8fDmqOsSdghS2mC-7R90hku9Rw> I have an error running the convnet: Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul 2 2016, 17:42:40) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://anaconda.org >>> runfile('/home/luca/data/DeepLearningTutorials/Theano-3D-ConvNet-master/convnet3d/core/run_multi_conv_t.py', wdir='/home/luca/data/DeepLearningTutorials/Theano-3D-ConvNet-master/convnet3d/core') Using gpu device 0: GeForce 840M (CNMeM is disabled, cuDNN 5103) /home/luca/data/Theano-master/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module. "downsample module has been moved to the theano.tensor.signal.pool module.") Disabling C code for Elemwise{mul,no_inplace} due to unsupported float16 Disabling C code for Elemwise{Cast{float32}} due to unsupported float16 Disabling C code for Elemwise{Cast{float16}} due to unsupported float16 Disabling C code for Elemwise{Cast{float16}} due to unsupported float16 Disabling C code for Alloc due to unsupported float16 Disabling C code for Elemwise{abs_,no_inplace} due to unsupported float16 Disabling C code for Sum{acc_dtype=float32} due to unsupported float16 Disabling C code for mrg_uniform{TensorType(float16, matrix),inplace} due to unsupported float16 Disabling C code for mrg_uniform{TensorType(float16, matrix),inplace} due to unsupported float16 Disabling C code for Elemwise{Composite{(-Cast{float16}((i0 / i1)))}} due to unsupported float16 Disabling C code for Elemwise{Composite{Cast{float16}(Cast{int64}(LT(i0, i1)))}}[(0, 0)] due to unsupported float16 Disabling C code for Elemwise{Composite{Cast{float16}(Cast{int64}(LT(i0, i1)))}}[(0, 0)] due to unsupported float16 Disabling C code for CorrMM{valid, (1, 1), (1, 1)} due to unsupported float16 Disabling C code for DiagonalSubtensor{inplace} due to unsupported float16 Disabling C code for Sum{axis=[3], acc_dtype=float32} due to unsupported float16 Disabling C code for Elemwise{Add}[(0, 0)] due to unsupported float16 Disabling C code for sigmoid due to unsupported float16 Disabling C code for Pool{ds=(3, 3), ignore_border=True, st=(3, 3), padding=(0, 0), mode='max'} due to unsupported float16 Disabling C code for Pool{ds=(1, 3), ignore_border=True, st=(1, 3), padding=(0, 0), mode='max'} due to unsupported float16 Disabling C code for dot due to unsupported float16 Disabling C code for Elemwise{Composite{scalar_sigmoid((i0 + i1))}}[(0, 0)] due to unsupported float16 Disabling C code for Elemwise{mul,no_inplace} due to unsupported float16 Disabling C code for dot due to unsupported float16 Disabling C code for CrossentropySoftmaxArgmax1HotWithBias due to unsupported float16 Disabling C code for CrossentropySoftmax1HotWithBiasDx due to unsupported float16 Disabling C code for Sum{acc_dtype=float32} due to unsupported float16 Disabling C code for Sum{axis=[0], acc_dtype=float32} due to unsupported float16 Disabling C code for dot due to unsupported float16 Disabling C code for dot due to unsupported float16 Disabling C code for Elemwise{Composite{(i0 - (i1 * i2))}}[(0, 0)] due to unsupported float16 Disabling C code for Elemwise{Composite{(i0 - (i1 * ((i2 * i0) + i3 + (i4 * sgn(i0)))))}}[(0, 3)] due to unsupported float16 Disabling C code for Elemwise{Composite{Cast{float16}(((i0 - i1) * i1 * i2 * i3 * i4))}}[(0, 1)] due to unsupported float16 Disabling C code for Elemwise{Sqr}[(0, 0)] due to unsupported float16 Disabling C code for Sum{axis=[0], acc_dtype=float32} due to unsupported float16 Disabling C code for dot due to unsupported float16 Disabling C code for dot due to unsupported float16 Disabling C code for Sum{acc_dtype=float32} due to unsupported float16 Disabling C code for Elemwise{Composite{(i0 - (i1 * i2))}}[(0, 0)] due to unsupported float16 Disabling C code for Elemwise{Composite{(i0 - (i1 * i2))}}[(0, 0)] due to unsupported float16 Disabling C code for Elemwise{Composite{((-Cast{float16}(((-i0) / i1))) + (i2 * i3) + (i2 * i4))}}[(0, 0)] due to unsupported float16 Disabling C code for MaxPoolGrad{ds=(1, 3), ignore_border=True, st=(1, 3), padding=(0, 0), mode='max'} due to unsupported float16 Disabling C code for MaxPoolGrad{ds=(3, 3), ignore_border=True, st=(3, 3), padding=(0, 0), mode='max'} due to unsupported float16 Disabling C code for Elemwise{Composite{Cast{float16}(((i0 - scalar_sigmoid(i1)) * i2 * scalar_sigmoid(i1)))}}[(0, 1)] due to unsupported float16 Disabling C code for Sum{axis=[1, 2, 3], acc_dtype=float32} due to unsupported float16 Disabling C code for Alloc due to unsupported float16 Disabling C code for Elemwise{Composite{(i0 - (i1 * i2))}}[(0, 0)] due to unsupported float16 Disabling C code for IncDiagonalSubtensor due to unsupported float16 Disabling C code for CorrMM_gradWeights{valid, (1, 1), (1, 1)} due to unsupported float16 Disabling C code for Elemwise{Composite{(i0 - (i1 * i2))}}[(0, 0)] due to unsupported float16 Disabling C code for mrg_uniform{TensorType(float16, matrix),inplace} due to unsupported float16 Disabling C code for mrg_uniform{TensorType(float16, matrix),inplace} due to unsupported float16 Disabling C code for CorrMM{valid, (1, 1), (1, 1)} due to unsupported float16 Disabling C code for DiagonalSubtensor{inplace} due to unsupported float16 Disabling C code for Sum{axis=[3], acc_dtype=float32} due to unsupported float16 Disabling C code for Elemwise{Add}[(0, 0)] due to unsupported float16 Disabling C code for Elemwise{ScalarSigmoid}[(0, 0)] due to unsupported float16 Disabling C code for Pool{ds=(3, 3), ignore_border=True, st=(3, 3), padding=(0, 0), mode='max'} due to unsupported float16 Disabling C code for Pool{ds=(1, 3), ignore_border=True, st=(1, 3), padding=(0, 0), mode='max'} due to unsupported float16 Disabling C code for dot due to unsupported float16 Disabling C code for Elemwise{Composite{(Cast{float16}(Cast{int64}(LT(i0, i1))) * scalar_sigmoid((i2 + i3)) * Cast{float16}(Cast{int64}(LT(i4, i1))))}}[(0, 0)] due to unsupported float16 Disabling C code for dot due to unsupported float16 Disabling C code for Elemwise{Add}[(0, 0)] due to unsupported float16 Disabling C code for MaxAndArgmax due to unsupported float16 start time: 04/10/2016 16:20:07 Images for training: 316 Images for validation: 56 Epochs: 100 training @ iter = 0 Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/luca/anaconda2/lib/python2.7/site-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 714, in runfile execfile(filename, namespace) File "/home/luca/anaconda2/lib/python2.7/site-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 81, in execfile builtins.execfile(filename, *where) File "/home/luca/data/DeepLearningTutorials/Theano-3D-ConvNet-master/convnet3d/core/run_multi_conv_t.py", line 32, in <module> run_experiments() File "/home/luca/data/DeepLearningTutorials/Theano-3D-ConvNet-master/convnet3d/core/run_multi_conv_t.py", line 25, in run_experiments Learning_rate=0.001 File "mpr_convnet_class_t.py", line 266, in __init__ training_cost_ij=train_model(a, b) File "/home/luca/data/Theano-master/theano/compile/function_module.py", line 879, in __call__ storage_map=getattr(self.fn, 'storage_map', None)) File "/home/luca/data/Theano-master/theano/gof/link.py", line 325, in raise_with_op reraise(exc_type, exc_value, exc_trace) File "/home/luca/data/Theano-master/theano/compile/function_module.py", line 866, in __call__ self.fn() if output_subset is None else\ File "/home/luca/data/Theano-master/theano/gof/op.py", line 908, in rval r = p(n, [x[0] for x in i], o) File "/home/luca/data/Theano-master/theano/gof/op.py", line 762, in perform "Did you used Theano flags mode=FAST_COMPILE?" theano.gof.utils.MethodNotDefined: ('perform', <class 'theano.tensor.nnet.corr.CorrMM'>, 'CorrMM', 'Did you used Theano flags mode=FAST_COMPILE? You can use optimizer=fast_compile instead.') Apply node that caused the error: CorrMM{valid, (1, 1), (1, 1)}(InplaceDimShuffle{0,x,1,2}.0, Subtensor{::, ::, ::int64, ::int64}.0) Toposort index: 30 Inputs types: [TensorType(float16, (False, True, False, False)), TensorType(float16, (False, True, False, False))] Inputs shapes: [(24, 1, 24, 24), (200, 1, 5, 5)] Inputs strides: [(2, 1152, 48, 1152), (50, 50, -10, -2)] Inputs values: ['not shown', 'not shown'] Outputs clients: [[Reshape{5}(CorrMM{valid, (1, 1), (1, 1)}.0, TensorConstant{[24 40 5 20 20]})]] Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer): File "<stdin>", line 1, in <module> File "/home/luca/anaconda2/lib/python2.7/site-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 714, in runfile execfile(filename, namespace) File "/home/luca/anaconda2/lib/python2.7/site-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 81, in execfile builtins.execfile(filename, *where) File "/home/luca/data/DeepLearningTutorials/Theano-3D-ConvNet-master/convnet3d/core/run_multi_conv_t.py", line 32, in <module> run_experiments() File "/home/luca/data/DeepLearningTutorials/Theano-3D-ConvNet-master/convnet3d/core/run_multi_conv_t.py", line 25, in run_experiments Learning_rate=0.001 File "mpr_convnet_class_t.py", line 169, in __init__ b ) File "cuddn_convnet3d.py", line 90, in __init__ border_mode='valid') HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node. On Monday, October 3, 2016 at 11:02:09 PM UTC+2, nouiz wrote: > > I have a fix in the new back-end about the error: > > https://github.com/Theano/Theano/pull/5050 > <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2FTheano%2FTheano%2Fpull%2F5050&sa=D&sntz=1&usg=AFQjCNFR8fDmqOsSdghS2mC-7R90hku9Rw> > > So you have a few way to get it to work. Use the config as Pascal wrote > (the easiest) or try this new PR. > > > > On Mon, Oct 3, 2016 at 12:11 PM, Pascal Lamblin <[email protected] > <javascript:>> wrote: > >> On Mon, Oct 03, 2016, [email protected] <javascript:> wrote: >> > floatX = float32 >> > device=cuda0 >> > dnn.conv.algo_fwd = time_once >> > dnn.conv.algo_bwd_filter = time_once >> > dnn.conv.algo_bwd_data = time_once >> >> In the .theanorc, you have to use sections, for instance: >> >> [dnn.conv] >> algo_fwd = time_once >> algo_bwd_filter = time_once >> algo_bwd_data = time_once >> >> > >> > Using theano.gpuarray.dnn.dnn_conv the output is: ValueError: >> > ("convolution algo %s can't be used for 3d convolutions", ('small',)) >> > Same output with float16. >> > >> > >> > If I use theano.sandbox.cuda.dnn.dnn_conv3d with Theano flags >> > floatX = float16 >> > device=cuda0 >> > dnn.conv.algo_fwd = time_once >> > dnn.conv.algo_bwd_filter = time_once >> > dnn.conv.algo_bwd_data = time_once >> > >> > the output is: TypeError: CudaNdarrayType only supports dtype float32 >> for >> > now. Tried using dtype float16 for variable None >> > >> > >> > >> > -- >> > >> > --- >> > You received this message because you are subscribed to the Google >> Groups "theano-users" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to [email protected] <javascript:>. >> > For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> Pascal >> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "theano-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
