Stefan Behnel wrote: > Hi, > > should we let Cython know certain stdlib modules and consider their import > as something that won't change meaning at runtime? > > This is mainly driven by loop optimisations. Imagine > > from itertools import izip > > always referred to the same thing. Then Cython could translate > > for a,b in izip(seq1, seq2): > ... > > into (the equivalent of) > > while True: > try: > temp_a = seq1.next() > temp_b = seq2.next() > except StopIteration: > break > a,b = temp_a, temp_b > ... > > instead of letting izip create a new iterator and tons of tuples along the > way, plus the obvious optimisations for constant sequences, lists, dicts, > etc. (Note that doing this for Py2's builtin zip() would not be correct as > the sequences are allowed to change during iteration in that case). > > http://wiki.cython.org/enhancements/forin > > Or think of the math module and its constants and functions. We could use > specialised C functions instead if we know the types that we are applying > them to. That way, users wouldn't have to care about two sets of functions > in Python's math module and C's math.h. > > Comments?
This has been on my mind too for a while, and I'm undecided, or rather ambient: I both love and hate the idea :-) I guess it depends on the motivation. I'm sceptical of hit-or-miss optimization in general -- that is, I don't think we can aim for "magically optimizing" very much user code through single optimization points like this. They are more for users who already know that their code will be both interpreted in Python and compiled with Cython -- of course, for such users an optimized izip could still be useful. I think my opinion is that I'd like to see this split this into a couple of orthogonal and generic features: a) Auto-cimport (if a directive is set, which defaults to on?). I.e. if you do a Python import, and a pxd file can be found, a cimport is done automatically. b) Overloading combined with fall-through to Python space. I started a CEP on this [1], Robert discussed it with me, but it never got finished. One way is to allow e.g. (in "math.pxd") double sin(double x): return libc.math.sin(x) object sin(object x) Somehow it must be added here though that sin should be looked up at module initialization using getattr rather than through __pyx_capi/Cython call convention. c) For izip (and, in time, zip in __builtin__ too) I'm more in favour of a slightly magic optimization -- it's easier to be an expert on hacking pxd files than the Cython compiler. Namely: If a generator function contains exactly one "yield", and is declared "inline", and used in a for-loop, then the entire generator will be inlined into the function (putting the for-loop body at the position of the yield). [1] http://wiki.cython.org/enhancements/overlaypythonmodules, but I wouldn't bother with it -- Dag Sverre _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
