Hi,
I recently (yesterday) also did ran some code that returned the trained
parameters as nan. The reason I got nan was because my cost function that
looks like this
cost = T.mean(T.exp((0.5 - self.x) * (T.dot(self.x, self.W) + self.b
)))
exploded in my first run, thus resulting in a nan value and the parameters
learnt is nan, which I would attribute it to the exponential function. I
would suggest you try printing out the gradients, I think it can be done
but I don't know how as I'm new to Theano to see do you get nan in the
first run. If your problem is indeed a very large valued cost function,
then perhaps you can keep it small with some epsilon (10e-6)? I did this
for my code, but it didn't work for me; I'm still learning the wrong
parameters.
Hope my comment helps.
zs
On Thursday, 5 January 2017 22:14:37 UTC+8, Cosmo Zhang wrote:
>
> Hi Dear All,
>
> When I construct such a computation
>
> p_1 = 1 / (1 + T.exp(-T.dot(x, (w1-w2)) - (b1-b2)))
>
> w1, w2, b1, b2 are parameters. And I construct a cross-entropy as loss
> function.
> But when I take gradients
> T.grad(loss,[w1, b1, w2, b2]
> All the resulted gradients are nan.
> Is there any possible reasons for this problem? And any solution to it?
>
> Thank you in advance!
>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
