Hi, Ideally you should be creating two different updates. One for the discriminator, the other for generator with respective parameters alone. The loss function 1) Of the generator should be computed on the output of discriminator (with input to discriminator being output of the generator). Note : For this, the update should be done on that of generator's parameter alone. 2) Of the discriminator should be on the output of discriminator plus what is discussed in 1). Note : Here the update should only done only on the param of discriminator network
Probably because you use only one loss it could be because of that the gradients are calculated for one run. From what I could infer, you don't penalise your discriminator. It just computes output. Ideally it should be evaluating a loss function with a dictionary of update variables. You could have a look here for a good GAN example, https://gist.github.com/f0k/9b0bb51040719eeafec7eba473a9e79b. I hope that helps, hope I haven't made a mistake anywhere as I am also a newbie in GAN :) Cheers, Ramana On Thursday, April 6, 2017 at 6:49:05 AM UTC+5:30, Jesse Livezey wrote: > > How is your updates dictionary being created? Maybe it is only getting > updates from one of the two batches. > > On Wednesday, April 5, 2017 at 1:05:27 PM UTC-7, Mikkel wrote: >> >> As the title says, I'm trying to implement a GAN in Theano/Lasagne. I'm >> having problems with my discriminator network. I think the following code >> should be sufficient to corrrect my mistake. >> >> >> >>> # Get the output of the discriminator network wrt actual data and >>> generated data >>> data_probs = l.layers.get_output(self.out_layer, >>> inputs={self.input_layer: self.from_data}) >>> generated_probs = l.layers.get_output(self.out_layer, >>> inputs={self.input_layer: self.from_generator}) >>> >>> # I put a minus in front of the loss because I don't know how to change >>> the updates to ascend the gradient. >>> loss = -T.mean( T.log(data_probs) + T.log(1 - generated_probs)) >>> >>> # Get all the parameters of the discriminator network >>> params = l.layers.get_all_params(self.out_layer) >>> >>> updates = l.updates.nesterov_momentum(loss, params, >>> learning_rate=config['learning_rate'], momentum=config['momentum']) >>> self.train_fn = theano.function([self.from_data, self.from_generator], >>> loss, updates=updates) >>> >> >> So, to get a feeling for whether the network is learning what it should I >> made an inference function and run real and generated data through it. >> >> discriminator_fn = theano.function([self.inputs], self.out_layer, self. >>> l.layers.get_output(self.out_layer, inputs={self.input_layer: self.inputs})) >>> print 'Data batch: {} >>> '.format(discriminator_fn(data_batch).flatten().mean()) >>> print 'Noise batch: {} >>> '.format(discriminator_fn(G.forward(noise_batch)).flatten().mean()) >> >> >> What I expect to happen, if I only train the discriminator, is that the >> discriminator network should on average the noise batch close to zero and >> the data batch close to one. >> >> What actually happens is that the discriminator network learns how to >> spot the noise batch (by scoring it close to 0). It does not learn anything >> about the real data though, consistently scoring it around 0.5 on average. >> >> If I flip the input to the function, it will score the real data close to >> 1 but 0.5 on the fake >> >> The network is being run through two times for each training iteration of >> the discriminator, once for the real and once for the fake data. It seems >> to me, how ever, that the gradients are only calculated for one of run >> throughs. >> How do I remedy this? >> >> Cheers >> > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
