About the benchmark, the problem is a known one, Theano isn't very fast to parse Theano input. In your case, it is this parsing that take the most time in theano when n is small. But when n increase, Theano is faster then lambdify:
n = 1 The derivation took 0.154258966446 seconds. Running with theano method. It took 0.515482187271 seconds to compute M and F with theano 1000 times at an average of 0.000515482187271 seconds per computation. Running with lambdify method. It took 0.0773530006409 seconds to compute M and F with lambdify 1000 times at an average of 7.73530006409e-05 seconds per computation. n = 6 The derivation took 6.48748397827 seconds. Running with theano method. It took 1.83713889122 seconds to compute M and F with theano 1000 times at an average of 0.00183713889122 seconds per computation. Running with lambdify method. It took 0.807645082474 seconds to compute M and F with lambdify 1000 times at an average of 0.000807645082474 seconds per computation. n = 11 The derivation took 33.0718569756 seconds. Running with theano method. It took 3.446434021 seconds to compute M and F with theano 1000 times at an average of 0.003446434021 seconds per computation. Running with lambdify method. It took 3.89024996758 seconds to compute M and F with lambdify 1000 times at an average of 0.00389024996758 seconds per computation. n = 16 The derivation took 104.534656048 seconds. Running with theano method. It took 5.41041302681 seconds to compute M and F with theano 1000 times at an average of 0.00541041302681 seconds per computation. Running with lambdify method. It took 11.5178751945 seconds to compute M and F with lambdify 1000 times at an average of 0.0115178751945 seconds per computation. The option to make it faster would be to give less inputs to the theano function. The first step would be to make the constants as theano constant instead of input to the theano function. That will make put the constant in the c code, so it could also speed up even the computation itself. On Tue, Aug 6, 2013 at 11:34 AM, Jason Moore <[email protected]> wrote: > Guy, > > I don't know enough about the details of the codegen module, but > theoretically there should be a mapping from sympy functions to a c-code > equivalent that you can add and/or override. It is just a matter of getting > real familiar with the code gen modules. > > You may want to check out https://github.com/IgnitionProject/ignition. > Andy showed me this at SciPy and setting up your own mappings for various > things seemed more clean than what is in SymPy. > > > Jason > moorepants.info > +01 530-601-9791 > > > On Tue, Aug 6, 2013 at 11:30 AM, Guy Parsey <[email protected]> wrote: > >> Addon to the Codegen section: >> With the example above of having an implemented function in some >> expression in sympy, assuming one can write the c-code equivalent, how >> would one link the c-code to the codegen process as opposed to having >> codegen generate it? >> Cheers, >> Guy >> >> On Tuesday, August 6, 2013 11:04:33 AM UTC-4, Guy Parsey wrote: >>> >>> Hello, >>> >>> Theano/Sympy questions >>> Fred, in terms of replacing sympy.Pieceiwse with a Theano equivalent, >>> since sympy.Piecewise attempts each condition (from ExprCondPair) until one >>> is valid, I would think that the closest equivalent would be a recursive >>> theano ifelse (ie. sympy.Piecewise((expr1,cond1),**(expr2,cond2)) ~ >>> theano.ifelse(cond1,expr1,**theano.ifelse(cond2,expr2,**None))) >>> statement so that the expressions are not evaluated until the condition is >>> achieved. This might be too much of a patch though and I am not sure how to >>> implement it with the same argument structure as sympy.Pieceiwse. >>> >>> With regards to using an externally defined theano graph or op (theano >>> wrapped sympy implementation), how would one pass them to theano_function? >>> It feels as though it would be analogous to the autowrap helpers argument, >>> or would this be an issue of merging graphs before calling theano.function? >>> If I can understand this step, I feel as though both of my problems would >>> be solvable (a wrapping of a piecewise along with a wrapping of an >>> interpolated function) >>> >>> In all of my simplified test cases (no pieceiwse or interpolations, 9 >>> heavily-linked non-linear ODEs), I also show that the lambdify function is >>> faster to evaluate than my created theano function. When passing a list of >>> expressions, if there is a commons subexpression that exists in two >>> expressions (not twice in the same expression), is it treated as such >>> during theano compilation or is each expression handled separately? >>> >>> Codegen questions (maybe a different topic): >>> Jason, I have started playing with the sympy.printing.codegen (C for >>> now, Fortran later) as an alternative to theano (I would very much like to >>> get multiple methods working, and GPU acceleration makes me want to keep >>> theano implementation). There is a size-able speedup relative to lambdify >>> or theano_function, which is to be expected, but I arrive at the same >>> problem as above with regards to interpolated or piecewise functions. The >>> initial crash from using Piecewise functions comes from Routine calling >>> sympy.tensor.index_methods get_contraction_structure which explicitly >>> states no support for Piecewise function types. In principle, if one made a >>> symbolic implemented function (preferably without an analytical >>> representation), could one link the python object to the compiled code? >>> What attributes would said implemented_function need to have? >>> >>> Thank you for your responses, I am glad to hear that at least I am not >>> completely missing something. >>> Cheers, >>> Guy >>> >>> On Tuesday, August 6, 2013 9:37:41 AM UTC-4, Jason Moore wrote: >>>> >>>> Fred, >>>> >>>> I think on_used_input=ignore should be a default in Matthew's theano >>>> printing code or that arg needs to be pushed up to his layer. I hit that >>>> issue too. >>>> >>>> The code I have is here: https://github.com/** >>>> PythonDynamics/pydy-code-gen<https://github.com/PythonDynamics/pydy-code-gen> >>>> >>>> See the results.txt file for basic speed comparisons. I'm generating >>>> the ODEs for an n-link pendulum with mechanics and then see how fast it >>>> generates and simulates with scipy.odeint. >>>> >>>> The code doesn't work at the moment. I haven't touched it in a month >>>> and looks like some things have changed in sympy and/or theano. I'll work >>>> on the bugs now. >>>> >>>> But "python benchmark.py" should run it with sympy master and ?some? >>>> version of Theano. >>>> >>>> >>>> Jason >>>> moorepants.info >>>> +01 530-601-9791 >>>> >>>> >>>> On Tue, Aug 6, 2013 at 9:25 AM, Frédéric Bastien <[email protected]>wrote: >>>> >>>>> Hi, >>>>> >>>>> I don't know what is sympy.functions.elementary.**piecewise. Do Jason >>>>> answered that part? If not, I'll look into it to know how to make Theano >>>>> reproduce it. About converting any Sympy symbol to Theano symbol, when >>>>> there isn't a one to one matching, you can create a one to a full Theano >>>>> graph conversion. When this is possible, it is probably the best, as if >>>>> you >>>>> make a new Theano op, it work work on the GPU. But if you make a Theano >>>>> graph, there is good change that the graph will already work on the GPU. >>>>> >>>>> If it is not possible to make a Theano graph for a sympy symbol, it is >>>>> possible to make a new Theano op that just wrap the sympy implementation. >>>>> Also, if this is a bottleneck, I recently added an example that show how >>>>> to >>>>> use numba with Theano so speed up the python code in a Theano op. >>>>> >>>>> >>>>> Theano do not parallelize on the CPU, except for the call to BLAS, >>>>> when the BLAS library is parallel. On the GPU, it is parallel. >>>>> >>>>> Having on_unused_input=ignore is normal for complicated generated >>>>> code. That is why it was added. But when the code is simpler and not >>>>> generated, but all user manually coded, most of time it mean the user >>>>> didn't do what he wanted. If you know it is normal that you have unused >>>>> input, there is no problem to use that flag. >>>>> >>>>> >>>>> Jason, about the case where Theano is slower, can you send me the >>>>> Theano code? I would like to look at it. I'm very surprised that Theano is >>>>> slower then Sympy in this case and would like to know why it is like this. >>>>> >>>>> Fred >>>>> >>>>> >>>>> On Mon, Aug 5, 2013 at 9:03 PM, Jason Moore <[email protected]>wrote: >>>>> >>>>>> Guy, >>>>>> >>>>>> We're working on the same problem for sympy.physics.mechanics. >>>>>> Matthew Rocklin added support for matrix conversions in the theano code >>>>>> that is in SymPy and I used that, but found that theano was slower that >>>>>> lambdify for most of my cases (I only have two cores, so I'm not taking >>>>>> advantage of the Theano parallel stuff). I think writing specific code >>>>>> gen >>>>>> for ode integration is going to be the best bet. I'm happy to collaborate >>>>>> on this with you. >>>>>> >>>>>> >>>>>> Jason >>>>>> moorepants.info >>>>>> +01 530-601-9791 >>>>>> >>>>>> >>>>>> On Mon, Aug 5, 2013 at 3:13 PM, Guy Parsey <[email protected]>wrote: >>>>>> >>>>>>> Hello Everyone, >>>>>>> Thank you in advance for reading through my problem and for any >>>>>>> input you may have. I must say that I still feel like a novice >>>>>>> programmer >>>>>>> and my problems may be easily solvable from a different mindset. My >>>>>>> present >>>>>>> project entails time-integration of extremely stiff and non-linear ODEs >>>>>>> with regards to chemical kinetics (one derivative equation for each >>>>>>> variable species) and energy equations. Initially we were planning on >>>>>>> using >>>>>>> the sympy.lambdify function to create callable functions for the main >>>>>>> function along with the jacobian and passing said functions to >>>>>>> scipy.integrate.odeint, however this method only works for easier test >>>>>>> cases (fewer species and/or no energy equations) before being limited by >>>>>>> either the list recursion limit or segfaulting due to the limited stack >>>>>>> size. I know that both of these limits can be edited, but that fact >>>>>>> that I >>>>>>> am reaching them makes me feel as though I am doing something extremely >>>>>>> inefficiently. Outside of the documentation of SymPy and Theano, I have >>>>>>> also been heavily using the BlogPost by Matthew Rocklin >>>>>>> http://matthewrocklin.**com/blog/work/2013/03/19/** >>>>>>> SymPy-Theano-part-1/<http://matthewrocklin.com/blog/work/2013/03/19/SymPy-Theano-part-1/>. >>>>>>> >>>>>>> Presently I am trying to use the mapping between Theano and SymPy >>>>>>> (sympy.printing.theanocode theano_function) to make my callable >>>>>>> functions >>>>>>> and take advantage of the optimization routines. I have two major >>>>>>> problems >>>>>>> and a few questions: >>>>>>> >>>>>>> 1st major problem: Though piecewise functions exist in SymPy >>>>>>> (sympy.functions.elementary.**piecewise) there is no counterpart in >>>>>>> Theano. Looking at the source of the inspiration for theanocode ( >>>>>>> https://github.com/nouiz/**theano_sympy/<https://github.com/nouiz/theano_sympy/> >>>>>>> graph_translation.py) I see that some of the SymPy equivalents were >>>>>>> defined as lambda functions. Is there an equivalent way to add Theano >>>>>>> conditional expressions wrapped into a function to add to the mapping >>>>>>> dictionary in theanocode.py? >>>>>>> >>>>>>> 2nd major problem: Similar to the problem above in that I am not >>>>>>> sure that the Theano counterpart is; some of the terms that I use are >>>>>>> interpolated functions (with one ODE variable as input) that we have >>>>>>> wrapped symbolically while providing a numerical implementation (so that >>>>>>> symbolic derivatives can be made, resulting in their own >>>>>>> interpolations). >>>>>>> Is it possible to recreate the interpolation function as a Theano >>>>>>> operation >>>>>>> for use within the system of ODEs? >>>>>>> >>>>>>> Remain questions: >>>>>>> I presently have to flatten my input to theano_function to a list of >>>>>>> expressions and then wrap to return to a form (Jacobian is a matrix not >>>>>>> a >>>>>>> vector); is it possible to have a matrix of different expressions as an >>>>>>> input to theano_function with a vector output? >>>>>>> >>>>>>> I know that a huge amount of Theano speed up is due to >>>>>>> parallelization of matrix operations (which I do not have), should I be >>>>>>> focusing on SymPy Autowrap/Ufuncify or my own code generation instead of >>>>>>> trying to get Theano to play nicely? >>>>>>> >>>>>>> Stupid questions: >>>>>>> Does sympy.printing.theanocode.**theano_function automatically >>>>>>> optimize the compiled graph? >>>>>>> >>>>>>> Minor comment: >>>>>>> Perhaps unnecessary for most uses of the theano_function, but I >>>>>>> needed to modify function inputs so as to be able to use the keyword >>>>>>> argument 'on_unused_input=ignore' as opposed to 'raise' so that I did >>>>>>> not >>>>>>> need to have all symbols in all equations. This may be avoided by having >>>>>>> the unused symbols somehow (I don't know how) included in each >>>>>>> expression. >>>>>>> >>>>>>> Thank you again for your time in reading my problems and any >>>>>>> potential help you may think of. I can attach code if necessary, I just >>>>>>> didn't want to make my post more confusing. >>>>>>> Have an excellent day. >>>>>>> Sincerely, >>>>>>> Guy Parsey >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "sympy" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> Visit this group at >>>>>>> http://groups.google.com/**group/sympy<http://groups.google.com/group/sympy> >>>>>>> . >>>>>>> For more options, visit >>>>>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>>>>>> . >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "sympy" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> Visit this group at >>>>>> http://groups.google.com/**group/sympy<http://groups.google.com/group/sympy> >>>>>> . >>>>>> For more options, visit >>>>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>>>>> . >>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "sympy" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at >>>>> http://groups.google.com/**group/sympy<http://groups.google.com/group/sympy> >>>>> . >>>>> For more options, visit >>>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>>>> . >>>>> >>>>> >>>>> >>>> >>>> -- >> You received this message because you are subscribed to the Google Groups >> "sympy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/sympy. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/sympy. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sympy. For more options, visit https://groups.google.com/groups/opt_out.
