Hi, When you tell 5x and 25x faster, what do you time?
Fred On Wed, Sep 11, 2013 at 7:51 PM, Guy Parsey <[email protected]> wrote: > Hello, > Due to an up coming conference and a need to get my code in a working > state, I have taken a brute force approach which seems to be working pretty > well (I am in the middle of testing it). I am very much interested in > working with Matt's advice on building up the ODE function from Theano > variables, but due to time constraints and an unfamiliarity with Theano, my > short-term solution is using Sympy theano_function while removing > complicated parts (spline interpolations wrapped as > function_symbols+argument_symbols) and replacing them as other symbols. A > python wrapper takes in an argument list (in the format for > scipy.integrate.odeint for now), a standard lambda function uses mapped > arguments to evaluate each function+argument pair only once per call, the > spline results are then passed to the theano_function function (this also > works for C codegen) as extra arguments. This method has worked with all of > my test cases so far (including a few I was having trouble with before), > but I have some larger simulations soon to come. > I use this method on my system of ODEs along with the jacobian (sympy > haddles the creation of this really well). Though sympy.lambdify so far > seems to be quite a bit faster to generate a callable function, this > wrapper method has been about 5x faster with function calls, 25x faster > with jacobian calls (A good number of the jacobian elements are zero, could > this be what is causing this difference in timing?) and does not encounter > the limits that initially drove me here. It definitely feels like a brute > force solution, but broken code doesn't return results to present :) > With regards to my test example I posted, I feel as though I am missing > something rather simple but it also makes complete sense that creating the > system of equations from theano variables would be a lot more efficient if > Theano evaluation is the final goal. Once the time crunch is not as > aggressive I will return to trying to find a cleaner solution. > Thank you all very much for your help and advice. I have learnt quite lot > with regards to both SymPy and Theano. > Cheers, > Guy > > On Tuesday, September 10, 2013 8:19:11 AM UTC-4, Frédéric Bastien wrote: > >> Hi, >> >> I was in vacation. Have you been able to make this work? >> >> Fred >> Le 22 août 2013 18:30, "Matthew Rocklin" <[email protected]> a écrit : >> >>> Looking at your code it's difficult for me to see what you're doing >>> with the cache. It looks like you're trying to fill it with a particular >>> value so that SymPy's theano_function call latches onto something you've >>> already built. >>> >>> Instead, I recommend that you use theano_code, to transform individual >>> sympy expressions into Theano variables like so >>> >>> theano_inps = [theano_code(inp) for inp in inplist] >>> theano_outs = [theano_code(expr) for expr in exprlist] >>> >>> Then, if you want to build more Theano expressions with your custom >>> Theano op you can do so in standard Theano using these standard Theano >>> variables. I think that this will be simpler than reverse engineering a >>> caching system in order to inject a pre-built theano variable into the >>> graph. >>> >>> In short, only use theano_function if you're only using pure SymPy. If >>> you're doing more complex things then translate your SymPy expressions to >>> Theano expressions and work in pure Theano from then on. >>> >>> >>> >>> On Thu, Aug 22, 2013 at 5:15 PM, Guy Parsey <[email protected]> wrote: >>> >>>> That is precisely what I don't understand. In your example we are >>>> neglecting to give the function all of its inputs and the error message: >>>> >>>> MissingInputError: ('An input of the graph, used to compute >>>> Elemwise{add,no_inplace}(x, y), was not provided and not given a value', y) >>>> >>>> is saying that we have forgotten the y input. In the test case I am >>>> doing, which yields the message: >>>> >>>> MissingInputError: ('An input of the graph, used to compute >>>> TheanoInterpWrapOp.**theanointerp(y), was not provided and not given a >>>> value', y) >>>> >>>> Though the message is practically identical, when running the test case >>>> with my verbose print statements, the following is printed before being >>>> passed to the Op pulled from the TheanoPrinter.cache >>>> children: [y] >>>> child types: [<class 'theano.tensor.basic.**TensorVariable'>] >>>> followed by the return line: >>>> >>>> self.cache[newkey](*children) >>>> >>>> where self.cache[newkey] is the theano op. Doesn't this mean that the >>>> theano variable y is being passed to the theano Op or does this y not carry >>>> a value? Instead of passing the theano variable y to the op, should I be >>>> passing it to a theano.function of the Op? >>>> Cheers, >>>> Guy >>>> >>>> On Thursday, August 22, 2013 6:01:17 PM UTC-4, Matthew wrote: >>>> >>>>> Here is a printout of the error message: >>>>> >>>>> MissingInputError: ('An input of the graph, used to compute >>>>> TheanoInterpWrapOp.**theanointer**p(y), was not provided and not >>>>> given a value', y) >>>>> >>>>> What this says is that some nodes in your graph (TheanoInterpWrapOp.** >>>>> theanointe**rp(y),) weren't given access to all of the inputs that >>>>> they needed. This would happen in Theano if, for example, >>>>> >>>>> x = theano.tensor.vector('x') >>>>> y = theano.tensor.vector('y') >>>>> z = x + y >>>>> f = theano.function([x], [z]) >>>>> >>>>> Notice that we're only giving function x and asking it to compute z. >>>>> This code will produce a similar error to what you're receiving. >>>>> >>>>> >>>>> On Thu, Aug 22, 2013 at 4:58 PM, Guy Parsey <[email protected]>wrote: >>>>> >>>>>> Correction (independent of testing the theano op), 's' is a simple >>>>>> wrapper to the spline function, it is not the sympy wrapper. the sympy >>>>>> wrapper is K and can be evaluated as >>>>>> In [17]: K._imp_(4.0) >>>>>> Out[17]: array(16.0) >>>>>> >>>>>> On Thursday, August 22, 2013 5:44:35 PM UTC-4, Guy Parsey wrote: >>>>>>> >>>>>>> Hey Matt, >>>>>>> I am pretty sure that I have tested the theano Op separately from >>>>>>> Sympy, but again, I am probably missing something silly. >>>>>>> After running the test cases, or evaluating >>>>>>> k = TestInterpOp() >>>>>>> k.CreateSuite() >>>>>>> from the SymPy_Theano_KGM_indep.py file, one can run the following >>>>>>> lines >>>>>>> >>>>>>> In [2]: s,K,Kp = k.SymInterp() >>>>>>> >>>>>>> In [3]: op = k.TheanoInterpOp() >>>>>>> >>>>>>> In [4]: x = theano.tensor.dvector() >>>>>>> >>>>>>> In [5]: f = theano.function([x],op(x)) >>>>>>> >>>>>>> In [6]: s(4.0) >>>>>>> Out[6]: array(16.0) >>>>>>> >>>>>>> In [7]: f([4.0]) >>>>>>> Out[7]: array([ 16.]) >>>>>>> >>>>>>> where 's' is the sympy wrapped spline function (undefined function) >>>>>>> and 'f' is the theano.function of the theano op created around the >>>>>>> spline. >>>>>>> Is this what you mean? >>>>>>> Cheers, >>>>>>> Guy >>>>>>> >>>>>>> On Thursday, August 22, 2013 5:18:49 PM UTC-4, Matthew wrote: >>>>>>>> >>>>>>>> Have you tested your interpolation op in isolation from SymPy? >>>>>>>> >>>>>>>> A quick glance at the error (quick glance means I can easily be >>>>>>>> wrong) leads me to think that this particular issue is localized >>>>>>>> within the >>>>>>>> domain of Theano. If this is the case then I recommend asking about >>>>>>>> your >>>>>>>> spline op on the [email protected] mailing list. >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Aug 22, 2013 at 3:59 PM, Guy Parsey <[email protected]>wrote: >>>>>>>> >>>>>>>>> Hello again everyone, >>>>>>>>> I thought I understood everything I needed to implement a theano >>>>>>>>> Op wrapping a scipy spline function through theanocode, but have been >>>>>>>>> promptly proven wrong (and was on a small family vacation). >>>>>>>>> Firstly, many thanks for the modifications done to theanocode to >>>>>>>>> allow for Piecewise and Undefined functions. I feel as though >>>>>>>>> everything is >>>>>>>>> in place for me to solve my problem, but I am still either lacking or >>>>>>>>> mis-undertsanding something with regards to mapping a custom theano Op >>>>>>>>> through the theanocode.theano_function. >>>>>>>>> >>>>>>>>>> >>>>>>>>> I have created a quick test case to show what I have understood to >>>>>>>>> date which in my mind should have all the pieces necessary to function >>>>>>>>> correctly. I have made a small git repository on GitHub in order to >>>>>>>>> share >>>>>>>>> this example and because I became fed up trying to figure out how to >>>>>>>>> publicly share a BitBucket repository (academic license-where I am >>>>>>>>> hosting >>>>>>>>> my thesis project-which will be made public once functioning >>>>>>>>> correctly). >>>>>>>>> https://github.com/gparsey/**KGM****indep_SympyTheanoOp<https://github.com/gparsey/KGMindep_SympyTheanoOp> >>>>>>>>> >>>>>>>>> Running: >>>>>>>>> >>> ipython SymPy_Theano_KGM_indep.py >>>>>>>>> Evaluates three test cases: f0) simple arithmetic operation, f1) >>>>>>>>> sympy piecewise into theano and f2) sympy undefined function wrapped >>>>>>>>> spline >>>>>>>>> into theano using a custom theano Op >>>>>>>>> Third test case crashes with: >>>>>>>>> <<<MissingInputError: ('An input of the graph, used to compute >>>>>>>>> TheanoInterpWrapOp.**theanointer****p(y), was not provided and >>>>>>>>> not given a value', y) >>>>>>>>> >>>>>>>>> I apologize in advance for: the verbosity of the test cases >>>>>>>>> (trying to figure out what is happening within theanocode) using >>>>>>>>> loc_theanocode (modified sympy.printing.theanocode), the novice >>>>>>>>> nature of >>>>>>>>> my code and whether I included correct references to the SymPy >>>>>>>>> community. I >>>>>>>>> am pretty sure that I am either missing something crucial or doing >>>>>>>>> something silly. Any and all help/comments would be greatly >>>>>>>>> appreciated. >>>>>>>>> Cheers, >>>>>>>>> Guy >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "sympy" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> To post to this group, send email to [email protected]. >>>>>>>>> Visit this group at >>>>>>>>> http://groups.google.com/**group****/sympy<http://groups.google.com/group/sympy> >>>>>>>>> . >>>>>>>>> For more options, visit https://groups.google.com/**grou**** >>>>>>>>> ps/opt_out <https://groups.google.com/groups/opt_out>. >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "sympy" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to sympy+un...@**googlegroups.com. >>>>>> To post to this group, send email to [email protected]. >>>>>> Visit this group at >>>>>> http://groups.google.com/**group**/sympy<http://groups.google.com/group/sympy> >>>>>> . >>>>>> For more options, visit >>>>>> https://groups.google.com/**grou**ps/opt_out<https://groups.google.com/groups/opt_out> >>>>>> . >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "sympy" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to sympy+un...@**googlegroups.com. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at >>>> http://groups.google.com/**group/sympy<http://groups.google.com/group/sympy> >>>> . >>>> For more options, visit >>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>>> . >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "sympy" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to sympy+un...@**googlegroups.com. >>> To post to this group, send email to [email protected]. >>> Visit this group at >>> http://groups.google.com/**group/sympy<http://groups.google.com/group/sympy> >>> . >>> For more options, visit >>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/sympy. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sympy. For more options, visit https://groups.google.com/groups/opt_out.
