Hi all,
Ok so I am trying to figure out how to implement an ordered dropout.
This is for a 1D vector of nodes (1..N), I would like the frequency of
dropping that node to be dependent on its position in the vector. So for
example node 1 would drop with a probability of 0.01, while node N with a
probability of 0.99
For what I understood the relevant part of the classic dropout can be done
as below. However, the binomial function here receives a constant value and
not a probability vector.
Any help appreciated.
def dropit(srng, weight, drop):
# proportion of probability to retain
retain_prob = 1 - drop
# a masking variable
mask = srng.binomial(n=1, p=retain_prob, size=weight.shape,
dtype='floatX')
# final weight with dropped neurons
return theano.tensor.cast(weight * mask,
theano.config.floatX)
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
