How is your updates dictionary being created? Maybe it is only getting
updates from one of the two batches.
On Wednesday, April 5, 2017 at 1:05:27 PM UTC-7, Mikkel wrote:
>
> As the title says, I'm trying to implement a GAN in Theano/Lasagne. I'm
> having problems with my discriminator network. I think the following code
> should be sufficient to corrrect my mistake.
>
>
>
>> # Get the output of the discriminator network wrt actual data and
>> generated data
>> data_probs = l.layers.get_output(self.out_layer,
>> inputs={self.input_layer: self.from_data})
>> generated_probs = l.layers.get_output(self.out_layer,
>> inputs={self.input_layer: self.from_generator})
>>
>> # I put a minus in front of the loss because I don't know how to change
>> the updates to ascend the gradient.
>> loss = -T.mean( T.log(data_probs) + T.log(1 - generated_probs))
>>
>> # Get all the parameters of the discriminator network
>> params = l.layers.get_all_params(self.out_layer)
>>
>> updates = l.updates.nesterov_momentum(loss, params,
>> learning_rate=config['learning_rate'], momentum=config['momentum'])
>> self.train_fn = theano.function([self.from_data, self.from_generator],
>> loss, updates=updates)
>>
>
> So, to get a feeling for whether the network is learning what it should I
> made an inference function and run real and generated data through it.
>
> discriminator_fn = theano.function([self.inputs], self.out_layer, self.
>> l.layers.get_output(self.out_layer, inputs={self.input_layer: self.inputs}))
>> print 'Data batch: {}
>> '.format(discriminator_fn(data_batch).flatten().mean())
>> print 'Noise batch: {}
>> '.format(discriminator_fn(G.forward(noise_batch)).flatten().mean())
>
>
> What I expect to happen, if I only train the discriminator, is that the
> discriminator network should on average the noise batch close to zero and
> the data batch close to one.
>
> What actually happens is that the discriminator network learns how to spot
> the noise batch (by scoring it close to 0). It does not learn anything
> about the real data though, consistently scoring it around 0.5 on average.
>
> If I flip the input to the function, it will score the real data close to
> 1 but 0.5 on the fake
>
> The network is being run through two times for each training iteration of
> the discriminator, once for the real and once for the fake data. It seems
> to me, how ever, that the gradients are only calculated for one of run
> throughs.
> How do I remedy this?
>
> Cheers
>
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
