I'm trying to use theano.clone to implement dropout in my MLP network. 
 Because I want to apply dropout at multiple layers, I pass the clone call 
multiple key value pairs to its replacement parameter: 
replace={layer1:mask*layer1, layer2:mask*layer2, etc} however the graph 
that's returned seems to have only actually made one of the replacements. 
 I suspect this is because clone is doing the replacements sequentially and 
once it's done one replacement it generates a new graph for which the other 
key value pairs no longer correspond.

Here is some example code that demonstrates the unexpected behavior:

    v = T.lscalar()
    exp1 = 2*v
    exp2 = 4*exp1
    exp3 = 6*exp2
    exp4 = 8*exp3

    print theano.pp(exp4)
    exp5 = theano.clone(exp4, replace={exp1:(3*exp1), exp2:(5*exp2), 
    print theano.pp(exp5)
    t = theano.function(inputs=[v], outputs=exp5)
    print t(1) 

The output is:
(TensorConstant{8} * (TensorConstant{6} * (TensorConstant{4} * 
(TensorConstant{2} * <TensorType(int64, scalar)>))))
(TensorConstant{8} * (TensorConstant{7} * (TensorConstant{6} * 
(TensorConstant{4} * (TensorConstant{2} * <TensorType(int64, scalar)>)))))

Although the clone adds the 7 factor to the new graph, it does not add the 
3 or 5 factors such that the output for an input value of 1 is 8*7*6*4*2*1 
instead of 8! as I would have expected.

I'm guessing this is how the clone function is supposed to work, but does 
anyone see how to get the desired behavior I'm looking for?  Perhaps I 
could apply the replacements one at a time and after each replacement 
update the remaining replacement key value pairs to point to corresponding 
points in the new graph, but I'm not sure how to find these corresponding 
points.  Or perhaps there's a function like the clone but that actually 
makes the replacements in place so that the other replacement key value 
pairs would not be invalidated after the first replacement?  Any ideas 
would be greatly appreciated!


You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to