On 05/31/2012 08:50 PM, Robert Bradshaw wrote:
On Thu, May 31, 2012 at 7:04 AM, Dag Sverre Seljebotn
<d.s.seljeb...@astro.uio.no> wrote:
[Discussion on numfo...@googlegroups.com please]
I've uploaded a draft-state SEP 201 (previously CEP 1000):
https://github.com/numfocus/sep/blob/master/sep201.rst
"""
Many callable objects are simply wrappers around native code. This holds for
any Cython function, f2py functions, manually written CPython extensions,
Numba, etc.
Obviously, when native code calls other native code, it would be nice to
skip the significant cost of boxing and unboxing all the arguments.
"""
The thread about this on the Cython list is almost endless:
http://thread.gmane.org/gmane.comp.python.cython.devel/13416/focus=13443
There was a long discussion on the key-comparison vs. interned-string
approach. I've written both up in SEP 201 since it was the major point of
contention. There was some benchmarks starting here:
http://thread.gmane.org/gmane.comp.python.cython.devel/13416/focus=13443
And why provide a table and not a get_function_pointer starting here:
http://thread.gmane.org/gmane.comp.python.cython.devel/13416/focus=13443
For those who followed that and don't want to read the entire spec, the
aspect of flags is new. How do we avoid to duplicate entries/check against
two signatures for cases like a GIL-holding caller wanting to call a nogil
function? My take: For key-comparison you can compare under a mask, for
interned-string we should have additional flags field.
The situation is a bit awkward: The Cython list consensus (well, me and
Robert Bradshaw) decided on what is "Approach 1" (key-comparison) in SEP
201. I pushed for that.
Still, now that a month has passed, I just think key-comparison is too ugly,
and that the interning mechanism shouldn't be *that* hard to code up,
probably 500 lines of C code if one just requires the GIL in a first
iteration, and that keeping the spec simpler is more important.
So I'm tentatively proposing Approach 2.
I'm still not convinced that a hybrid approach, where signatures below
some cutoff are compiled down to keys, is not a worthwhile approach.
This gets around variable-length keys (both the complexity and
possible runtime costs for long keys) and allows simple libraries to
produce and consume fast callables without participating in the
interning mechanism.
I still think this gives us the "worst of both worlds", all the
disadvantages and none of the advantages.
How many simple libraries are there really? Cython on one end, the
magnificently complicated NumPy ufuncs on the other? Thinking big,
perhaps PyPy and Julia? Cython, PyPy, Julia would all have to deal with
long signatures anyway. And NumPy ufuncs are already complicated so even
more low-level stuff wouldn't hurt.
It's unclear how to rendezvous on a common interning interface without
the GIL/Python, so perhaps requiring the GIL to use it not to onerous.
An alternative is to acquire the GIL in the first/reference
implementation (which could allow the interning function pointers to
be cached by an external GIL-oblivions JIT for example). Presumably
some other locking mechanism would be required if the GIL is not used,
so the overhead would likely not be that great.
Yes. I guess a goal could be to make sure there's no ABI breakage
if/when the GIL requirement is lifted.
Since modules can already have a reference to the interner by the time
the first module interfacing with a GIL-less world is imported, this is
non-trivial, but "every problem can be solved with another level of
indirection", and particularly this one.
Good idea on separating out interning as a separate spec; that's
definitely useful for interfaces etc. as well down the line. I can get
to work on a string interning spec and implementation as SEP 202 in
spare minutes over the next month or so, but I won't bother unless SEP
201 will uses interning. My role in that depends on Travis' timeline as
well, as my ETA is so unpredictable.
Dag
_______________________________________________
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
