I also have a question on this. This is how I am using it at the moment for
my convolutional layer: (conv + pool in pool out, pre-activation)
self.mean = theano.shared(value=numpy.zeros((channels,),
dtype=theano.config.floatX), name =
'population_mean', borrow = borrow)
self.var = theano.shared(value=numpy.zeros((nkerns,),
dtype=theano.config.floatX),
name = 'population_var',
borrow=borrow)
batch_norm_out,_,_,self.mean,self.var =
batch_normalization_train(
inputs = pool_out + \
self.b.dimshuffle('x', 0, 'x', 'x'),
gamma = self.gamma,
beta = self.beta,
running_mean = self.mean,
running_var = self.var )
And for inference time, I use the following :
batch_norm_inference = batch_normalization_test (
inputs = pool_out + \
self.b.dimshuffle('x', 0, 'x', 'x'),
gamma = self.gamma,
beta = self.beta,
mean = self.mean,
var = self.var )
The question I have though is on the running mean and variance returned by
the train method. Is it alright to over-write them the way I have done so ?
If not should I create a automatic update for the mean and variance update
such as
updates [self.mean] = (running mean returned by the train method).
On Thursday, February 16, 2017 at 8:17:24 PM UTC-7, David Leon wrote:
>
> I'm using nnet.bn.batch_normalization_train() and
> nnet.bn.batch_normalization_test() for batch normalization, however during
> test phase, nnet.bn.batch_normalization_test() produces wrong results. For
> the time being, I just use nnet.bn.batch_normalization_train() with
> *running_average_factor
> *set to zero for test phase as:
>
> if deterministic is False: # train phase
> normalized, input_mean, input_inv_std, self.mean, self.var =
> T.nnet.bn.batch_normalization_train(input, self.gamma, self.beta, self.axes,
>
> self.epsilon, self.alpha, self.mean, self.var)
> else: # test phase
> # normalized = T.nnet.bn.batch_normalization_test(input, self.gamma,
> self.beta, self.mean, self.var, self.axes, self.epsilon)
> normalized, _, _, _, _ = T.nnet.bn.batch_normalization_train(input,
> self.gamma, self.beta, self.axes, self.epsilon, 0.0, self.mean, self.var)
> return normalized
>
>
>
> My theano version is
> '0.9.0beta1.dev-b2afa088d1cb416b4507348019af34adae908b73', CUDA 8.0 and
> CuDNN 5.1
>
>
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
