Is there anything else in the error message at all? Typically, there should be a line with the name of the optimization that is crashing.
On Wed, Aug 31, 2016, Tom Ball wrote: > > > Hi, > > > I am having some issues getting theano to run correctly on GPUs using > cuDNN. The same code runs and compiles without any errors when running it > on a CPU node or a GPU node without cuDNN enabled. > > I am a non-root user working on a cluster machine. The cluster has a number > of GPU nodes (Tesla K20m, k40m...). The system is running Scientific Linux > release 6.6 (Carbon). > > The system admin has installed CUDA 5.5 and CUDA 6.5, using cuDNN with > these versions does not work as theano expects a later version of cuDNN > that is not compatible with 6.5. > > I have locally installed the bleeding edge version of Theano. I have also > tried using the current release version and some archived version with no > variation in the errors reported. > > I have locally installed CUDA 7.5 as well as cudNN V5. > > > When running the theano code theano recognises that cuDNN 5005 is > installed, but I later get crashes in the code (that don't occur when > running the code on a CPU or GPU with cuDNN disabled). > > > I was wondering if there was any additional configuration that I need to do > in order to get theano to work correctly with cuDNN? > > > > > This is the crash log that occurs ONLY when cuDNN is enabled. Thanks for > your time. > > > Traceback (most recent call last): > File "runner.py", line 306, in <module> > main() > File "runner.py", line 241, in main > queries_exp = __import__(args.exp_model).queries_exp > File "/mnt/nfs2/inf/tjb32/workspace/CNN_EL/nlp-entity- > convnet/exp_multi_conv_cosim.py", line 923, in <module> > queries_exp = EntityVectorLinkExp() > File "/mnt/nfs2/inf/tjb32/workspace/CNN_EL/nlp-entity- > convnet/exp_multi_conv_cosim.py", line 51, in __init__ > self._setup() > File "/mnt/nfs2/inf/tjb32/workspace/CNN_EL/nlp-entity- > convnet/exp_multi_conv_cosim.py", line 543, in _setup > on_unused_input='ignore', > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/compile/function.py", > > line 326, in function > output_keys=output_keys) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/compile/pfunc.py", > > line 484, in pfunc > output_keys=output_keys) > File "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/ > theano/compile/function_module.py", line 1788, in orig_function > output_keys=output_keys).create( > File "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/ > theano/compile/function_module.py", line 1467, in __init__ > optimizer_profile = optimizer(fgraph) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > line 102, in __call__ > return self.optimize(fgraph) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > line 90, in optimize > ret = self.apply(fgraph, *args, **kwargs) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > line 235, in apply > sub_prof = optimizer.optimize(fgraph) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > line 90, in optimize > ret = self.apply(fgraph, *args, **kwargs) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > line 235, in apply > sub_prof = optimizer.optimize(fgraph) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > line 90, in optimize > ret = self.apply(fgraph, *args, **kwargs) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > line 2262, in apply > lopt_change = self.process_node(fgraph, node, lopt) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > line 1825, in process_node > lopt, node) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > line 1719, in warn_inplace > return NavigatorOptimizer.warn(exc, nav, repl_pairs, local_opt, node) > File > "/home/t/tj/tjb32/.local/lib/python2.7/site-packages/theano/gof/opt.py", > line 1705, in warn > raise exc > AssertionError > > -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- Pascal -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
