On Tue, Apr 17, 2012 at 3:34 PM, Dag Sverre Seljebotn <d.s.seljeb...@astro.uio.no> wrote: > On 04/17/2012 04:20 PM, Nathaniel Smith wrote: >> Since you've set this up... I have a suggestion for something that may >> be worth trying, though I've hesitated to propose it seriously. And >> that is, an API where instead of scanning a table, the C-callable >> exposes a pointer-to-function, something like >> int get_funcptr(PyObject * self, PyBytesObject * signature, struct >> c_function_info * out) > > > Hmm. There's many ways to implement that function though. It shifts the > scanning logic from the caller to the callee;
Yes, that's part of the point :-). Or, well, I guess the point is more that it shifts the scanning logic from the ABI docs to the callee. > you would need to call it > multiple times for different signatures... Yes, I'm not sure what I think about this -- there are arguments either way for who should handle promotion. E.g., imagine the following situation: We have a JITable function We have already JITed the int64 version of this function Now we want to call it with an int32 Question: should we promote to int64, or should we JIT? Later you write: > if found in table: > do dispatch > else if object supports get_funcptr: > call get_funcptr > else: > python dispatch If we do promotion during the table scanning, then we'll never call get_funcptr and we'll never JIT an int32 version. OTOH, if we call get_funcptr before doing promotion, then we'll end up calling get_funcptr multiple times for different signatures regardless. OTOOH, there are a *lot* of possible coercions for, say, a 3-argument function with return, so just enumerating them is not necessarily a good strategy. Possibly if get_functpr can't handle the initial signature, it should return a table of signatures that it *is* willing to handle... assuming that most callees will either be able to handle a fixed set of types (cython variants) or else handle pretty much anything (JIT), and only the former will reach this code path. Or we could write down the allowed promotions (stealing from the C99 spec), and require the callee to pick the best promotion if it can't handle the initial request. Or we could put this part off until version 2, once we see how eager callers are to actually implement a real promotion engine. > But if the overhead can be shown to be miniscule then it does perhaps make > the API nicer, even if it feels like paying for nothing at the moment. But > see below. > > Will definitely not get around to this today; anyone else feel free... > > >> >> The rationale is, if we want to support JITed functions where new >> function pointers may be generated on the fly, the array approach has >> a serious problem. You have to decide how many array slots to allocate >> ahead of time, and if you run out, then... too bad. I guess you get to > > > Note that the table is jumped to by a pointer in the PyObject, i.e. the > PyObject I've tested with is > > [object data, &table, table] Oh, I see! I thought you were embedding it in the object, to avoid an extra indirection (and potential cache miss). That's probably necessary, for the reasons you say, but also makes the get_funcptr approach potentially more competitive. > So a JIT could have the table in a separate location on the heap, then it > can allocate a new table, copy over the contents, and when everything is > ready, then do an atomic pointer update (using the assembly instructions/gcc > intrinsics, not pthreads or locking). > > The old table would need to linger for a bit, but could at latest be > deallocated when the PyObject is deallocated. IMHO we should just hold the GIL through lookups, which would simplify tihs, but that's mostly based on the naive intuition that we shouldn't be passing around Python boxes in no-GIL code. Maybe there are good reasons to. - N _______________________________________________ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
