I have tried inline=True, but the compilation did not finish in an hour for a test case so I doubt this is a viable option. Could you elaborate on how to construct a GPU only graph? Could I make a normal graph, then compile it - where the optimizations would move it to gpu and then use that for OfG?
Regarding the shared variable its exactly like I wanted it to be :) Thank you! On Friday, 24 March 2017 04:25:35 UTC+1, Adam Becker wrote: > > OpFromGraph is still under development. If you want to use it on GPU, the > safest way would be setting inline=True at constructor (requires 0.9rc1+). > This will cause more compilation time though. Or you can try constructing a > GPU only graph by hand and build OfG with that, I didn't test that though. > > Yes, a shared variable is "global" in OfG, much like a static variable in > a C function. Using one OfG op to make multiple Apply nodes would use the > same shared variable. If you want separate shared variables, you should > define them outside OfG, and use pure symbolic graph inside OfG. > > On Wednesday, March 22, 2017 at 10:43:09 PM UTC+8, Šarūnas S. wrote: >> >> I have been trying to train an NN with little success and started >> wondering whether all instances of OpFromGraph share the underlying shared >> variables. Ie. when doing gradient updates - are my gradients wrt shared >> variables computed over all OpFromGraph nodes or is it done only locally >> within each OpFromGraph instance? >> >> I would welcome if someone could elaborate on this since the >> documentation on OpFromGraph is very sparse. >> >> On Friday, 17 March 2017 10:29:16 UTC+2, Šarūnas S. wrote: >>> >>> I am building a reinforcement learner and I am wondering how to scale >>> it. >>> >>> At first I initialised a deep neural network in Keras and convert it to >>> theano computational graph which takes state variables as inputs and >>> outputs an action to make. >>> Then, I wrote a simulator in Theano where at decision points I >>> theano.clone the DNN computational graph. Lastly, I do gradient descent on >>> the DNN parameters in order to get a "good" DNN AI. If I use a proper DNN >>> with many layers and parameters the compilation takes forever and >>> iterations are very slow. >>> >>> Then I've tried using OpFromGraph. It seems to reduce my compilation >>> time quite a bit. However, once I looked at the computational graph it >>> seems that OpFromGraph moves everything back to the CPU. >>> >>> Given that the op is a DNN which are very GPU friendly I wonder whether >>> there is a way to avoid that? >>> >>> Please find my graph at >>> https://drive.google.com/open?id=0BzjH-3p3dTNzWU8zS05wMU5STEk >>> >>> -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
