I created an issue to document some tradeoff between compile time and run time. Can you try them:
https://github.com/Theano/Theano/issues/5762 That would be much simpler to use then OpFromGraph as it isn't ready when not used with inline=True. keep us updated on your results. On Fri, Mar 24, 2017 at 4:09 AM Šarūnas S. <[email protected]> wrote: > I have tried inline=True, but the compilation did not finish in an hour > for a test case so I doubt this is a viable option. Could you elaborate on > how to construct a GPU only graph? Could I make a normal graph, then > compile it - where the optimizations would move it to gpu and then use that > for OfG? > > Regarding the shared variable its exactly like I wanted it to be :) Thank > you! > > > On Friday, 24 March 2017 04:25:35 UTC+1, Adam Becker wrote: > > OpFromGraph is still under development. If you want to use it on GPU, the > safest way would be setting inline=True at constructor (requires 0.9rc1+). > This will cause more compilation time though. Or you can try constructing a > GPU only graph by hand and build OfG with that, I didn't test that though. > > Yes, a shared variable is "global" in OfG, much like a static variable in > a C function. Using one OfG op to make multiple Apply nodes would use the > same shared variable. If you want separate shared variables, you should > define them outside OfG, and use pure symbolic graph inside OfG. > > On Wednesday, March 22, 2017 at 10:43:09 PM UTC+8, Šarūnas S. wrote: > > I have been trying to train an NN with little success and started > wondering whether all instances of OpFromGraph share the underlying shared > variables. Ie. when doing gradient updates - are my gradients wrt shared > variables computed over all OpFromGraph nodes or is it done only locally > within each OpFromGraph instance? > > I would welcome if someone could elaborate on this since the documentation > on OpFromGraph is very sparse. > > On Friday, 17 March 2017 10:29:16 UTC+2, Šarūnas S. wrote: > > I am building a reinforcement learner and I am wondering how to scale it. > > At first I initialised a deep neural network in Keras and convert it to > theano computational graph which takes state variables as inputs and > outputs an action to make. > Then, I wrote a simulator in Theano where at decision points I > theano.clone the DNN computational graph. Lastly, I do gradient descent on > the DNN parameters in order to get a "good" DNN AI. If I use a proper DNN > with many layers and parameters the compilation takes forever and > iterations are very slow. > > Then I've tried using OpFromGraph. It seems to reduce my compilation time > quite a bit. However, once I looked at the computational graph it seems > that OpFromGraph moves everything back to the CPU. > > Given that the op is a DNN which are very GPU friendly I wonder whether > there is a way to avoid that? > > Please find my graph at > https://drive.google.com/open?id=0BzjH-3p3dTNzWU8zS05wMU5STEk > > -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
