[theano-users] Re: Theano GPU results not reproducible
Some operations on GPU are not deterministic. I think some convolution operations and also reduction operations are two examples. See this thread for more info https://groups.google.com/forum/#!searchin/theano-users/atomic$20add%7Csort:relevance/theano-users/g-BF6zwMirM/ojWzbUBPBwAJ On Tuesday, July 18, 2017 at 1:20:41 PM UTC-7, Wenpeng Yin wrote: > > Hi guys, > > I have a long-term problem when running theano code in GPU: even I use two > command windows to run the same program (on the same GPU or different > GPUs), they show different performances. It's hard to say the difference is > small or big, depending on the task. This makes difficult to judge a > program modification is better or worse. > > I can not find the problem, as I notice that I always use the same random > seed, for example "rng = numpy.random.RandomState(23455)", whenever I > create parameters, so they are expected to repeat the process, right? > > The only thing I can think about is that GPU uses 32 bits, not 64, this > will lose precision? > > Thanks for any hints. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[theano-users] Re: Implementing a GPU op
Oh man, so glad I randomly clicked on your your message! I just wrote a post about the same problem, except I was getting the segmentation fault while simply trying to run some old code on the GPU with the new GPU backend. I also use anaconda and using the init_gpu_device flag instead of the ordinary device flag as well as downgrading pygpu and libgpuarray also solved my problem. Not sure if the problem lies with Theano, but seems like this could affect a fair number of people. On Sunday, July 16, 2017 at 4:15:51 AM UTC-7, Christopher Bourez wrote: > > Moreover if you install Theano from scratch : > > conda install theano pygpu > > Fetching package metadata . > Solving package specifications: . > > Package plan for installation in environment /root/miniconda2: > > The following NEW packages will be INSTALLED: > > libgpuarray: 0.6.8-0 > pygpu: 0.6.8-py27_0 > theano: 0.9.0-py27_0 > > > you need to run the following correction after : > > conda install libgpuarray=0.6.4-0 pygpu=0.6.4 > > > And now that works on the server I was initially working on. > > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[theano-users] New GPUArray Related Crash
Theano version 0.90 is crashing python on my system (Ubuntu 14.04) with the following error message: "Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)" . This crash happens when I do theano.test(). Theano also similarly crashes when I call theano.function() in some personal code with device=cuda. The code works fine when device=gpu or device=cpu. Curiously, running the GPU test code from the documentation with device=cuda works fine. It may be useful to know that theano seems to successfully find my graphics card: "Mapped name None to device cuda: GeForce GTX 870M (:01:00.0)." Is there some way to get more detailed information on the error? Any idea what could be causing it? -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
