Think I agree On Thursday, 3 December 2015 00:10:43 UTC, Daniel Carlin wrote: > > Should the last line of the free energy calc read: > > return -hidden_term + vbias_term > > going off of > https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2012-95.pdf for > instance? > > On Friday, 15 June 2012 05:37:08 UTC-7, Pawel wrote: >> >> >> >> On Thursday, 14 June 2012 22:31:38 UTC+1, dag wrote: >>> >>> I'm starting to work with Theano and I'd like to know if you have some >>> Theano implementation of Gaussian-Bernoulli DBNs. >>> I want to train a phone classifier like in: >>> http://www.cs.utoronto.ca/~gdahl/papers/dbnPhoneRec.pdf >>> >>> So following the tutorial (from the engineering point of view) you >> should: >> >> 1) Inherit the new GBRBM class from RBM and overwrite three functions >> free_energy, sample_v_given_h and reconstruction_cost (you probably want >> MSE as monitoring cost of Gaussian-Bernoulli variant so do no need to worry >> about pseudo likelihood), like these for example: >> >> # here only vbias term is different >> def free_energy(self, v_sample): >> wx_b = T.dot(v_sample, self.W) + self.hbias >> vbias_term = 0.5*T.dot((v_sample-self.vbias), >> (v_sample-self.vbias).T) >> hidden_term = T.sum(T.log(1+T.exp(wx_b)), axis = 1) >> >> return -hidden_term - vbias_term >> >> # and here you sample from normal distribution >> def sample_v_given_h(self, h0_sample): >> pre_sigmoid_v1, v1_mean = self.propdown(h0_sample) >> >> #in fact, you don't need to sample from normal distribution here >> and just use mean instead >> v1_sample = self.theano_rng.normal(size = v1_mean.shape, avg=0.0, >> std=1.0,\ >> dtype = theano.config.floatX) + pre_sigmoid_v1 >> >> return [pre_sigmoid_v1, v1_mean, v1_sample] >> >> Note: Bear in mind you need to normalize your data to zero mean unit >> variance prior training. >> >> 2) In DBN class constructor when you stack RBMs just create a GBRBM in >> the first layer >> >> -- >> Pawel >> >
-- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
