Hello,
I'm trying to use theano.clone to implement dropout in my MLP network.
Because I want to apply dropout at multiple layers, I pass the clone call
multiple key value pairs to its replacement parameter:
replace={layer1:mask*layer1, layer2:mask*layer2, etc} however the graph
that's returned seems to have only actually made one of the replacements.
I suspect this is because clone is doing the replacements sequentially and
once it's done one replacement it generates a new graph for which the other
key value pairs no longer correspond.
Here is some example code that demonstrates the unexpected behavior:
v = T.lscalar()
exp1 = 2*v
exp2 = 4*exp1
exp3 = 6*exp2
exp4 = 8*exp3
print theano.pp(exp4)
exp5 = theano.clone(exp4, replace={exp1:(3*exp1), exp2:(5*exp2),
exp3:(7*exp3)})
print theano.pp(exp5)
t = theano.function(inputs=[v], outputs=exp5)
print t(1)
The output is:
(TensorConstant{8} * (TensorConstant{6} * (TensorConstant{4} *
(TensorConstant{2} * <TensorType(int64, scalar)>))))
(TensorConstant{8} * (TensorConstant{7} * (TensorConstant{6} *
(TensorConstant{4} * (TensorConstant{2} * <TensorType(int64, scalar)>)))))
2688
Although the clone adds the 7 factor to the new graph, it does not add the
3 or 5 factors such that the output for an input value of 1 is 8*7*6*4*2*1
instead of 8! as I would have expected.
I'm guessing this is how the clone function is supposed to work, but does
anyone see how to get the desired behavior I'm looking for? Perhaps I
could apply the replacements one at a time and after each replacement
update the remaining replacement key value pairs to point to corresponding
points in the new graph, but I'm not sure how to find these corresponding
points. Or perhaps there's a function like the clone but that actually
makes the replacements in place so that the other replacement key value
pairs would not be invalidated after the first replacement? Any ideas
would be greatly appreciated!
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
