On Thu, Jun 14, 2012 at 3:38 PM, Nathaniel Smith <n...@pobox.com> wrote: > On Thu, Jun 14, 2012 at 7:53 PM, James Bergstra > <bergs...@iro.umontreal.ca> wrote: >> On Thu, Jun 14, 2012 at 11:01 AM, Nathaniel Smith <n...@pobox.com> wrote: >> >>>> Indeed that would be great as sympy already has already excellent math >>>> expression rendering. >>>> >>>> An alternative would be to output mathml or something similar that >>>> could be understood by the mathjax rendering module of the IPython >>>> notebook. >>> >>> I'd find it quite useful if it could spit out the derivative as Python >>> code that I could check and integrate into my source. I often have a >>> particular function that I need to optimize in many different >>> situations, but would rather not pull in a whole (complex and perhaps >>> fragile) bytecode introspection library just to repeatedly recompute >>> the same function on every run... >>> >>> -N >> >> I was hoping to get by with bytecode-> bytecode interface, are there >> bytecode -> source tools that could help here? > > Not that I know of -- you might try googling "python reverse engineer" > or similar. Mostly people treat bytecode as the internal intermediate > format it is. I'm sort of confused at why people are suddenly excited > about using (some particular CPython release's version of) bytecode as > an input format when both the ast module and Cython are perfectly > capable of parsing real Python source into a nice abstract format, but > you all seem to be having fun so hey. >
Heh, yeah I'm sure the bytecode high will wear off soon enough. For now though, the key advantage over ast manipulation is that bytecode seems easier to *run* than an ast. Actually running a non-trivial Python code fragment is the only reliable way of determining what it will do. Python is so weakly typed that static analysis is bound to be even more fragile (imagine trying to do static analysis of a function that takes variables from globals or a closure!?) In general, the drawback of running the bytecode is that the trace depend on control flow. You obviously don't get to follow both an `if` and `else` branch if you are trying to emulate the real interpreter. Automatic differentiation is a good fit here because as a general rule an epsilon change to parameters does not change the control flow path. I'm hoping that violations of this general rule are both (a) easy for autodiff to detect and (b) easy for programmers to rewrite... but time will tell. As for the CPython-version-specificity of bytecode I haven't found it changes much from 2.5 to 2.7... there are new instructions for conditional branching, but those are trivial to accommodate. Again, maybe there are some surprises on the way, but so far smooth sailing. >> Otherwise it might be possible to appeal to the symbolic intermediate >> representation to produce more legible source. >> >> With regards to "pulling in a whole bytecode introspection library" I >> don't really see what you mean. If the issue is that you want some way >> to verify that the output function is actually computing the right >> thing, then I hear you - that's an issue. If the issue that autodiff >> itself is slow, then I'd like to hear more about the application, >> because in minimization you usually have to call the function many >> times (hundreds) so the autodiff overhead should be relatively small >> (I'm not counting Theano's function compilation time here, which still >> can be significant... but that's a separate concern.) > > For example, I wrote a library routine for doing log-linear > regression. Doing this required computing the derivative of the > likelihood function, which was a huge nitpicky hassle; took me a few > hours to work out and debug. But it's still just 10 lines of Python > code that I needed to figure out once and they're done forever, now. > I'd have been perfectly happy if I could have gotten those ten lines > by asking a random unreleased library I pulled off github, which > depended on heavy libraries like Theano and relied on a mostly > untested emulator for some particular version of the CPython VM. But > I'd be less happy to ask everyone who uses my code to install that > library as well, just so I could avoid having to spend a few hours > doing math. This isn't a criticism or your library or anything, it's > just that I'm always going to be reluctant to rely on an automatic > differentiation tool that takes arbitrary code as input, because it > almost certainly cannot be made fully robust. So it'd be nice to have > the option to stick a human in the loop. > > -N > Thanks, that makes sense. Sounds like that print function that Olivier was asking about would be just the thing. I'm reading this mainly as a plea for e.g. Theano to provide better human-readable output, or for sympy to provide better support for tensor expressions. Let me know if that's not fair. My understanding is that your wish would be textual output as close to runnable numpy code as possible. - James -- http://www-etud.iro.umontreal.ca/~bergstrj _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
