On Wed, 16 May 2012 20:49:18 +0100, mark florisson
<markflorisso...@gmail.com> wrote:
On 16 May 2012 20:15, Stefan Behnel <stefan...@behnel.de> wrote:
"Martin v. Löwis", 16.05.2012 20:33:
Does this use case make sense to everyone?
The reason why we are discussing this on python-dev is that we are
looking
for a general way to expose these C level signatures within the
Python
ecosystem. And Dag's idea was to expose them as part of the type
object,
basically as an addition to the current Python level tp_call()
slot.
The use case makes sense, yet there is also a long-standing
solution
already to expose APIs and function pointers: the capsule objects.
If you want to avoid dictionary lookups on the server side,
implement
tp_getattro, comparing addresses of interned strings.
I think Martin has a point there. Why not just use a custom
attribute on
callables that hold a PyCapsule? Whenever we see inside of a Cython
implemented function that an object variable that was retrieved from
the
outside, either as a function argument or as the result of a
function call,
is being called, we try to unpack a C function pointer from it on
all
assignments to the variable. If that works, we can scan for a
suitable
signature (either right away or lazily on first access) and cache
that. On
each subsequent call through that variable, the cached C function
will be used.
That means we'd replace Python variables that are being called by
multiple
local variables, one that holds the object and one for each C
function with
a different signature that it is being called with. We set the C
function
variables to NULL when the Python function variable is being
assigned to.
When the C function variable is NULL on call, we scan for a matching
signature and assign it to the variable. When no matching signature
can be
found, we set it to (void*)-1.
Additionally, we allow explicit user casts of Python objects to C
function
types, which would then try to unpack the C function, raising a
TypeError
on mismatch.
Assignments to callable variables can be expected to occur much less
frequently than calls to them, so this will give us a good trade-off
in
most cases. I don't see why this kind of caching would be any slower
inside
of loops than what we were discussing so far.
Stefan
_______________________________________________
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
This works really well for local variables, but for globals, def
methods or callbacks as attributes, this won't work so well, as they
may be rebound at any time outside of the module scope. I think in
+1. The python-dev discussion is pretty focused on the world of a
manually written C extension. But code generation is an entirely
different matter. Python puts in place pretty efficient boundaries
against full-program static analysis, so there's really not much we can
do.
Here's some of my actual code I have for wrapping a C++ library:
cdef class CallbackEventReceiver(BasicEventReceiver):
cdef object callback
def __init__(self, callback):
self.callback = callback
cdef dispatch_event(self, ...):
self.callback(...)
The idea is that you can subclass BasicEventReceiver in Cython for
speed, but if you want to use a Python callable then this converter is
used.
This code is very performance critical. And, the *loop* in question
sits deep inside a C++ library.
Good luck pre-acquiring the function pointer of self.callback in any
useful way. Even if it is not exported by the class, that could be
overridden by a subclass. I stress the fact that this is real world code
by yours truly (unfortunately not open source, it wraps a closed source
library).
Yes, you can tell users to be mindful of this and make as much as
possible local variables, introduce final modifiers and __nomonkey__ and
whatnot, but that's a large price to pay to avoid hacking tp_flags.
Dag
_______________________________________________
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
