I see, thanks Pascal! Shame map_variables doesn't do the trick in this case. I think I'll go with the manual approach you recommended as it seems the most efficient and relatively straight forward in my case.
On Friday, October 14, 2016 at 5:13:53 PM UTC-7, Pascal Lamblin wrote: > > On Sat, Oct 15, 2016, Pascal Lamblin wrote: > > Another option, still experimental, may be the `map_variables` function > > in scan_modules/scan_utils. > > There seem to be some challenges regarding scalar constants with that > function, but I was able to do the following: > > >>> theano.tensor.basic.constant.enable = False > >>> v = theano.tensor.lscalar('v') > >>> exp1 = 2 * v > >>> exp1.name = 'exp1' > >>> exp2 = 4 * exp1 > >>> exp2.name = 'exp2' > >>> exp3 = 6 * exp2 > >>> exp3.name = 'exp3' > >>> exp4 = 8 * exp3 > >>> exp4.name = 'exp4' > >>> replace_dict = {'exp1': (3*exp1), 'exp2': (5*exp2), 'exp3': (7*exp3)} > >>> def replace(var): > ... return replace_dict.get(var.name, var) > >>> exp5, = theano.scan_module.scan_utils.map_variables(replace, [exp4]) > >>> theano.printing.debugprint(exp5) > Elemwise{mul,no_inplace} [id A] 'exp4' > |TensorConstant{8} [id B] > |Elemwise{mul,no_inplace} [id C] '' > |TensorConstant{7} [id D] > |Elemwise{mul,no_inplace} [id C] '' > > The issue is that it introduced a cycle in the graph: it replaced exp3 > by 7*exp3, where exp3 is the new one... > > I guess that illustrates the challenge of getting replacements right. > > > > > Finally, it is actually possible to replace Apply nodes inputs manually. > > In your case, you could do something like: > > > > >>> exp2.owner.inputs[1] = 3*exp1 > > >>> exp3.owner.inputs[1] = 5*exp2 > > >>> exp4.owner.inputs[1] = 7*exp3 > > >>> print(theano.pp(exp4)) > > (TensorConstant{8} * (TensorConstant{7} * (TensorConstant{6} * > (TensorConstant{5} * (TensorConstant{4} * (TensorConstant{3} * > (TensorConstant{2} * <TensorType(int64, scalar)>))))))) > > >>> exp4.eval({v: 1}) > > array(40320) > > > > But it can get hard to get right if the same expression is re-used > > several times. > > > > On Fri, Oct 14, 2016, John Coolidge wrote: > > > Hello, > > > > > > I'm trying to use theano.clone to implement dropout in my MLP network. > > > Because I want to apply dropout at multiple layers, I pass the clone > call > > > multiple key value pairs to its replacement parameter: > > > replace={layer1:mask*layer1, layer2:mask*layer2, etc} however the > graph > > > that's returned seems to have only actually made one of the > replacements. > > > I suspect this is because clone is doing the replacements > sequentially and > > > once it's done one replacement it generates a new graph for which the > other > > > key value pairs no longer correspond. > > > > > > Here is some example code that demonstrates the unexpected behavior: > > > > > > v = T.lscalar() > > > exp1 = 2*v > > > exp2 = 4*exp1 > > > exp3 = 6*exp2 > > > exp4 = 8*exp3 > > > > > > print theano.pp(exp4) > > > exp5 = theano.clone(exp4, replace={exp1:(3*exp1), exp2:(5*exp2), > > > exp3:(7*exp3)}) > > > print theano.pp(exp5) > > > t = theano.function(inputs=[v], outputs=exp5) > > > print t(1) > > > > > > > > > The output is: > > > (TensorConstant{8} * (TensorConstant{6} * (TensorConstant{4} * > > > (TensorConstant{2} * <TensorType(int64, scalar)>)))) > > > (TensorConstant{8} * (TensorConstant{7} * (TensorConstant{6} * > > > (TensorConstant{4} * (TensorConstant{2} * <TensorType(int64, > scalar)>))))) > > > 2688 > > > > > > Although the clone adds the 7 factor to the new graph, it does not add > the > > > 3 or 5 factors such that the output for an input value of 1 is > 8*7*6*4*2*1 > > > instead of 8! as I would have expected. > > > > > > I'm guessing this is how the clone function is supposed to work, but > does > > > anyone see how to get the desired behavior I'm looking for? Perhaps I > > > could apply the replacements one at a time and after each replacement > > > update the remaining replacement key value pairs to point to > corresponding > > > points in the new graph, but I'm not sure how to find these > corresponding > > > points. Or perhaps there's a function like the clone but that > actually > > > makes the replacements in place so that the other replacement key > value > > > pairs would not be invalidated after the first replacement? Any ideas > > > would be greatly appreciated! > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to the Google > Groups "theano-users" group. > > > To unsubscribe from this group and stop receiving emails from it, send > an email to theano-users...@googlegroups.com <javascript:>. > > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > Pascal > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "theano-users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to theano-users...@googlegroups.com <javascript:>. > > For more options, visit https://groups.google.com/d/optout. > > -- > Pascal > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.