Hi Guys,
Just looking back over that myself. And I have a similar question to one
thing you're saying and an answer to another I think :)
*Re givens={self.x : train_set[batch..]}* .... As far as I can see, only
the *bottom* layer RBM's input is *X. *The rest of the RBMs input is
calculated (through the theano symbolic graph) as the output from the
previous layers
i.e. input to layer 2 = self.sigmoid_layers[-1].output
... that I get... but....
I agree with what Nizar is saying in that the output is between 0 and 1 so
would they not have to use gaussian-bernoulli RBMs and use a different free
energy function to compute the upper layer RBMs properly?
All the Best,
Jim
On Friday, 16 October 2015 01:03:08 UTC+1, Nizar Gandy Assaf Layouss wrote:
>
> Well actually I have the same question!
> The input of the upper RBMs are connected to the output of the HiddenLayer
> that's right lower to it and is continuous valued in the range of [0,1].
> Shouldn't the input of the upper RBMs be connected to binomial sampling
> with probability equals that output ??
> In fact, for a problem I'm working on I had to change it that way to see
> the expected results. I'm wondering how come that it works on MNIST problem
> in that way !
>
> BR ,
> Nizar Assaf
>
> El viernes, 14 de agosto de 2015, 19:50:18 (UTC+3), Siavash Malektaji
> escribió:
>>
>> Hi,
>>
>> I have a question about the pre-training phase of DBNs as implemented in
>> theano tutorial code. As I understand and mentioned in the DBN tutorial
>> there are two ways for stacking RBMs
>> 1. use the the mean activations of previous layer's RBM hidden units as
>> the input of the next layer's RBM
>> 2. sample from the hidden layer of the previous layer p(h(i) | h(i-1))
>> and use it as the input for next layer's RBM
>> However the way it is implemented is always use training data as the
>> input layer (for first layer and also deeper layer), which is highlighted
>> in red in the following
>>
>> learning_rate = T.scalar('lr') # learning rate to use
>>
>> # number of batches
>> n_batches = train_set_x.get_value(borrow=True).shape[0] / batch_size
>> # begining of a batch, given `index`
>> batch_begin = index * batch_size
>> # ending of a batch given `index`
>> batch_end = batch_begin + batch_size
>>
>> pretrain_fns = []
>> for rbm in self.rbm_layers:
>>
>> # get the cost and the updates list
>> # using CD-k here (persisent=None) for training each RBM.
>> # TODO: change cost function to reconstruction error
>> cost, updates = rbm.get_cost_updates(learning_rate,
>> persistent=None, k=k)
>>
>> # compile the theano function
>> fn = theano.function(
>> inputs=[index, theano.Param(learning_rate, default=0.1)],
>> outputs=cost,
>> updates=updates,
>> givens={
>> self.x: train_set_x[batch_begin:batch_end]
>> }
>> )
>> # append `fn` to the list of functions
>> pretrain_fns.append(fn)
>>
>> return pretrain_fns
>>
>> I would really appreciate any comments to help me understand, this
>> implementation and also how those two methods of stacking RBMs are
>> implemented in the theano DBN tutorial
>> (http://deeplearning.net/tutorial/DBN.html)
>>
>> Thanks,
>>
>> Siavash Malektaji
>>
>>
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
