I have recently been forced to upgrade from cudnn 4 to 5 as Im using a shared server. I am still able to run my network architectures but with some strange results.
I am running with lasagne (bleeding edge version) dnn implementations of 3d and 2d networks (non bleeding edge thenao) and cuda 8.0. I am able to run my 3d cnn fine with reasonable results. However, when I run my 2d architecture i am getting a loss of nan. This did not happen with cudnn 4 and as far I can see this is the only change that has been made. Is there any explanation for this? Anything I can do to rectify the situation? I should note that for my first minibatch I have a loss around 0.5, which is reduced to nan immediatley afterwards. Best, J -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
