To update just in case other people have the same problem, by switching to bleeding edge theano I no longer experience nan values with cudnn 5 :)
On Thursday, 10 November 2016 09:10:40 UTC+1, Joe Blair wrote: > > I have recently been forced to upgrade from cudnn 4 to 5 as Im using a > shared server. I am still able to run my network architectures but with > some strange results. > > I am running with lasagne (bleeding edge version) dnn implementations of > 3d and 2d networks (non bleeding edge thenao) and cuda 8.0. I am able to > run my 3d cnn fine with reasonable results. However, when I run my 2d > architecture i am getting a loss of nan. > > This did not happen with cudnn 4 and as far I can see this is the only > change that has been made. Is there any explanation for this? > > Anything I can do to rectify the situation? I should note that for my > first minibatch I have a loss around 0.5, which is reduced to nan > immediatley afterwards. > > Best, > > J > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
