On Thu, Jun 14, 2012 at 3:42 PM, Olivier Grisel <olivier.gri...@ensta.org> wrote: > 2012/6/14 James Bergstra <bergs...@iro.umontreal.ca>: >> On Thu, Jun 14, 2012 at 4:00 AM, Olivier Grisel >> <olivier.gri...@ensta.org> wrote: >>> 2012/6/13 James Bergstra <bergs...@iro.umontreal.ca>: >>>> Further to the recent discussion on lazy evaluation & numba, I moved >>>> what I was doing into a new project: >>>> >>>> PyAutoDiff: >>>> https://github.com/jaberg/pyautodiff >>>> >>>> It currently works by executing CPython bytecode with a numpy-aware >>>> engine that builds a symbolic expression graph with Theano... so you >>>> can do for example: >>>> >>>>>>> import autodiff, numpy as np >>>>>>> autodiff.fmin_l_bfgs_b(lambda x: (x + 1) ** 2, [np.zeros(())]) >>>> >>>> ... and you'll see `[array(-1.0)]` printed out. >>>> >>>> In the future, I think it should be able to export the >>>> gradient-computing function as bytecode, which could then be optimized >>>> by e.g. numba or a theano bytecode front-end. For now it just compiles >>>> and runs the Theano graph that it built. >>>> >>>> It's still pretty rough (you'll see if you look at the code!) but I'm >>>> excited about it. >>> >>> Very interesting. Would it be possible to use bytecode introspection >>> to printout the compute and display a symbolic representation of an >>> arbitrary python + numpy expression? >>> >>> E.g. something along the lines of: >>> >>>>>> g = autodiff.gradient(lambda x: (x + 1) ** 2, [np.zeros(())]) >>>>>> print g >>> f(x) = 2 * x + 2 >>>>>> g(np.arrange(3)) >>> array[2, 4, 6] >>> >>> -- >>> Olivier >>> http://twitter.com/ogrisel - http://github.com/ogrisel >> >> So... almost? >> >> I just hacked this gradient function to see what theano could print >> out, and the first thing that happened (after my own mistakes were >> sorted out) was an error because the lambda expression was defined to >> work on a 0-d array, but then you evaluated g on a vector. Was this >> part of the test? If so, I'm not sure I think it's a good idea, I'm >> assuming it was a cut-and-paste oversight and moving on.... > > Indeed, my bad. I wrote that email in a hurry while waiting in line > for boarding in a plane while still using the airport wifi... > >> I settled on >> (https://github.com/jaberg/pyautodiff/blob/master/autodiff/tests/test_gradient.py) >> ``` >> import numpy as np >> from autodiff import Gradient >> >> def test_basic(): >> g = Gradient(lambda x: ((x + 1) ** 2).sum(), [np.zeros(3)]) >> print g >> print g(np.arange(3)) >> ``` >> >> The output is ... well... ugly but correct: >> Elemwise{Composite{[mul(i0, add(i1, i2))]}}(TensorConstant{(1,) of >> 2.0}, TensorConstant{(1,) of 1.0}, <TensorType(float64, vector)>) >> [array([ 2., 4., 6.])] > > Indeed it's a bit hard to parse by a human :) > >> So with some effort on pretty-printing I'm pretty confident that this >> could work, at least for simple examples. Pretty-printing is always a >> challenge for non-trivial examples. One option might be to convert >> the internal symbolic graph to sympy? > > Indeed that would be great as sympy already has already excellent math > expression rendering. > > An alternative would be to output mathml or something similar that > could be understood by the mathjax rendering module of the IPython > notebook.
I'd find it quite useful if it could spit out the derivative as Python code that I could check and integrate into my source. I often have a particular function that I need to optimize in many different situations, but would rather not pull in a whole (complex and perhaps fragile) bytecode introspection library just to repeatedly recompute the same function on every run... -N _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
