Hi,
thank you for this code, I think it worked for me.
But I did not understand what the [T.arange(y.shape[0]), y] makes?
If I see it right,
T.log(self.p_y_given_x) returns a Tensor with shape
(number_of_unique_classes)x(batch_size). It contains the logarithmic
probabilities for each class
(w1,w2,..) has as many weights as the number of unique classes.
So (T.log(self.p_y_given_x) * w) returns the "weighted probabilities", that
also can be greater than 1.
According to the mathematical definition of the loss-function,
[T.arange(y.shape[0]), y] should actually pick the probability for the
actual correct class in each row of (T.log(self.p_y_given_x) * w).
Is that correct?
Am Dienstag, 9. Juni 2015 23:37:20 UTC+2 schrieb Pascal Lamblin:
>
> On Tue, Jun 09, 2015, HeeHwan Park wrote:
> > Hi, I'm totally new to theano and I'm trying to make DBN for my
> > classification problem by using sample code from deeplearning.net. And
> I
> > want to give weights for each class to negative log likelihood(NLL), but
> I
> > don’t know how to do it in theano.
> > For example, let me assume there are two classes, then likelihood
> function
> > of the prediction of this model is P(y=0|x, theta)*P(y=1|x, theta) for
> each
> > input point x. So NLL is -1*{log(P(y=0|x, theta))+ log(P(y=1|x,
> theta))}.
> > What I want to do is giving weights {w_0, w_1}, such as -1*{log(P(y=0|x,
> > theta))*w_0+ log(P(y=1|x, theta))*w_1}.
> > I spent several days to solve this, but I failed. theano code about NLL
> is
> > following.
> > ------------------------
> > self.p_y_given_x = T.nnet.softmax(T.dot(input, self.W) + self.b)
> > ...
> > -T.mean(T.log(self.p_y_given_x)[T.arange(y.shape[0]), y])
> > -----------------------
> > How should I do?
>
> Assuming you have your weights in a vector w = [w_0, w_1, ...], then you
> can simply multiply it (elementwise) with the log-probability:
>
> -T.mean((T.log(self.p_y_given_x) * w)[T.arange(y.shape[0]), y])
>
>
>
>
> --
> Pascal
>
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
