As the title says, I'm trying to implement a GAN in Theano/Lasagne. I'm
having problems with my discriminator network. I think the following code
should be sufficient to corrrect my mistake.
> # Get the output of the discriminator network wrt actual data and
> generated data
> data_probs = l.layers.get_output(self.out_layer, inputs={self.input_layer:
> self.from_data})
> generated_probs = l.layers.get_output(self.out_layer,
> inputs={self.input_layer: self.from_generator})
>
> # I put a minus in front of the loss because I don't know how to change
> the updates to ascend the gradient.
> loss = -T.mean( T.log(data_probs) + T.log(1 - generated_probs))
>
> # Get all the parameters of the discriminator network
> params = l.layers.get_all_params(self.out_layer)
>
> updates = l.updates.nesterov_momentum(loss, params,
> learning_rate=config['learning_rate'], momentum=config['momentum'])
> self.train_fn = theano.function([self.from_data, self.from_generator],
> loss, updates=updates)
>
So, to get a feeling for whether the network is learning what it should I
made an inference function and run real and generated data through it.
discriminator_fn = theano.function([self.inputs], self.out_layer, self.
> l.layers.get_output(self.out_layer, inputs={self.input_layer: self.inputs}))
> print 'Data batch: {}
> '.format(discriminator_fn(data_batch).flatten().mean())
> print 'Noise batch: {}
> '.format(discriminator_fn(G.forward(noise_batch)).flatten().mean())
What I expect to happen, if I only train the discriminator, is that the
discriminator network should on average the noise batch close to zero and
the data batch close to one.
What actually happens is that the discriminator network learns how to spot
the noise batch (by scoring it close to 0). It does not learn anything
about the real data though, consistently scoring it around 0.5 on average.
If I flip the input to the function, it will score the real data close to 1
but 0.5 on the fake
The network is being run through two times for each training iteration of
the discriminator, once for the real and once for the fake data. It seems
to me, how ever, that the gradients are only calculated for one of run
throughs.
How do I remedy this?
Cheers
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
