What I mean is, the output of pooling with mode="sum" is equivalent to the output of pooling with mode="average_inc_pad" multiplied by the number of pixels that are pooled. That would be the product of the elements of pool_size below, or in that specific case, (self.sentence_length_short - self.num_words_to_use_conv) * 1.
It may not matter if you are learning the weights anyway. On Fri, Sep 02, 2016, Tom Ball wrote: > Hi, > > Thanks for the response. > > I have changed from mode='sum' to mode='average_inc_pad' for all layers. > This seems to have fixed the crash! > > This is my first time using Theano and CNNs so i'm still very much a > novice. When you say multiply by the window size, what exactly do you mean > by that? Do you mean multiply the pool size by the window size? > > For example I define a layer as such: > > self.surface_pool1_l = lasagne.layers.Pool2DLayer( > self.surface_conv1_l, > name='surface_pool1', > pool_size=(self.sentence_length_short - self.num_words_to_use_conv, 1), > mode='average_inc_pad', > ) > > > > On Thursday, September 1, 2016 at 4:41:55 PM UTC+1, Pascal Lamblin wrote: > > > > Hi, > > > > The issue is that cuDNN has no "sum" pooling, but Theano still tries to > > optimizer CPU sum-pooling by creating a cuDNN Op, which crashes at the > > creation. > > > > The right solution would be to update the optimization doing that > > (https://github.com/Theano/Theano/issues/4929). > > > > A workaround for you now would be to explicitly use > > "average_inc_padding" and multiply by the window size. > > > > On Wed, Aug 31, 2016, Tom Ball wrote: > > > Hi, > > > > > > Sorry I missed part of the error message when I copied it over. > > > > > > Using gpu device 0: Tesla K40m (CNMeM is enabled with initial size: > > 95.0% > > > of memory, cuDNN 5005) > > > True > > > ++ which python2 > > > + PYTHON=/cm/shared/apps/python/2.7.5/bin/python2 > > > + WIKIPEDIA_DUMP=enwiki-latest-pages-articles.xml > > > + WIKIPEDIA_DOWNLOAD= > > > > > https://dumps.wikimedia.org/enwiki/latest/enwiki-latest-pages-articles.xml.bz2 > > > > > + WIKIPEDIA_W2V=enwiki-vecs > > > + python runner.py --queries /home/t/tj/tjb32/workspace/ > > > CNN_EL/data/data/ace-dev-set-queries.json --surface_count > > > /home/t/tj/tjb32/workspace/CNN_EL/data/data/enwiki-vecs-surface.json > > > --wordvecs /home/t/tj/tjb32/workspace/CNN_EL/data/data/enwiki-vecs.bin > > > --redirects > > /home/t/tj/tjb32/workspace/CNN_EL/data/data/enwiki-vecs-redir.json > > > --wiki_dump > > /home/t/tj/tjb32/workspace/CNN_EL/data/data/enwiki-latest-pages-articles.xml > > > > > > > --raw_output ./data/ace-dev-set.h5 --csv_output ./data/ace-dev-set.csv > > > Using gpu device 0: Tesla K40m (CNMeM is enabled with initial size: > > 95.0% > > > of memory, cuDNN 5005) > > > > > /home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/tensor/signal/downsample.py:6: > > > > > > > UserWarning: downsample module has been moved to the > > > theano.tensor.signal.pool module. > > > "downsample module has been moved to the theano.tensor.signal.pool > > > module.") > > > > > /home/t/tj/tjb32/.local/lib/python2.7/site-packages/lasagne/layers/conv.py:489: > > > > > > > UserWarning: The `image_shape` keyword argument to `tensor.nnet.conv2d` > > is > > > deprecated, it has been renamed to `input_shape`. > > > border_mode=border_mode) > > > ERROR (theano.gof.opt): Optimization failure due to: > > > local_avgpool_dnn_grad_stride > > > ERROR (theano.gof.opt): node: AveragePoolGrad{ds=(5, 1), > > > ignore_border=True, st=(5, 1), padding=(0, 0), > > mode='sum'}(HostFromGpu.0, > > > Rebroadcast{?,?,0,0}.0) > > > ERROR (theano.gof.opt): TRACEBACK: > > > ERROR (theano.gof.opt): Traceback (most recent call last): > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 1820, in process_node > > > replacements = lopt.transform(node) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/sandbox/cuda/dnn.py", > > > > > > > line 3034, in local_avgpool_dnn_grad_stride > > > ret = GpuDnnPoolGrad(mode=mode)(gpu_contiguous(inp), > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/sandbox/cuda/dnn.py", > > > > > > > line 1746, in __init__ > > > assert mode in ('max', 'average_inc_pad', 'average_exc_pad') > > > AssertionError > > > > > > Total queries possible: 0.984514648951, Testing queries possible: > > > 0.949673811743 > > > Number word vectors: 4402369 > > > Finding relevant pages from wikipedia > > > Loading model > > > Traceback (most recent call last): > > > File "runner.py", line 306, in <module> > > > main() > > > File "runner.py", line 241, in main > > > queries_exp = __import__(args.exp_model).queries_exp > > > File "/mnt/nfs2/inf/tjb32/workspace/CNN_EL/nlp-entity- > > > convnet/exp_multi_conv_cosim.py", line 923, in <module> > > > queries_exp = EntityVectorLinkExp() > > > File "/mnt/nfs2/inf/tjb32/workspace/CNN_EL/nlp-entity- > > > convnet/exp_multi_conv_cosim.py", line 51, in __init__ > > > self._setup() > > > File "/mnt/nfs2/inf/tjb32/workspace/CNN_EL/nlp-entity- > > > convnet/exp_multi_conv_cosim.py", line 543, in _setup > > > on_unused_input='ignore', > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/compile/function.py", > > > > > > > line 326, in function > > > output_keys=output_keys) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/compile/pfunc.py", > > > > > > > line 484, in pfunc > > > output_keys=output_keys) > > > File "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/ > > > theano/compile/function_module.py", line 1788, in orig_function > > > output_keys=output_keys).create( > > > File "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/ > > > theano/compile/function_module.py", line 1467, in __init__ > > > optimizer_profile = optimizer(fgraph) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 102, in __call__ > > > return self.optimize(fgraph) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 90, in optimize > > > ret = self.apply(fgraph, *args, **kwargs) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 235, in apply > > > sub_prof = optimizer.optimize(fgraph) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 90, in optimize > > > ret = self.apply(fgraph, *args, **kwargs) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 235, in apply > > > sub_prof = optimizer.optimize(fgraph) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 90, in optimize > > > ret = self.apply(fgraph, *args, **kwargs) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 2262, in apply > > > lopt_change = self.process_node(fgraph, node, lopt) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 1825, in process_node > > > lopt, node) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 1719, in warn_inplace > > > return NavigatorOptimizer.warn(exc, nav, repl_pairs, local_opt, > > node) > > > File > > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > > > line 1705, in warn > > > raise exc > > > AssertionError > > > > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to the Google > > Groups "theano-users" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [email protected] <javascript:>. > > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > Pascal > > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- Pascal -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
