On 14 February 2012 13:21, Maciej Fijalkowski <fij...@gmail.com> wrote:
> On Tue, Feb 14, 2012 at 3:12 PM, Stefan Behnel <stefan...@behnel.de> wrote:
>> Armin Rigo, 14.02.2012 13:13:
>>> the main problem is that the RPython-to-C translation that we
>>> do is not just a one-format traduction. We need to tweak the
>>> intermediate code in various ways depending on various settings. The
>>> way it is tweaked really depends on these settings in a way that
>>> cannot be captured just by C macros. (If it could, we might have
>>> written the whole project in C instead of RPython in the first place.)
>>> You would end up with a version of the Cython C code that only works
>>> on *some* kind of PyPy, like using the minimark GC, without any JIT
>>> support, and without sandboxing support. (Just to add an example, I
>>> could also say "and without stackless support", but this is no longer
>>> true nowadays because we no longer rely on a transformation for
>>> stackless.)
>>
>> Hmm, if that is so, how would you ever want to make PyPy bidirectionally
>> interface with anything at all? How does ctypes even work in PyPy? Is it
>> just that you're lucky that ctypes can be controlled completely from within
>> PyPy and doesn't let any internals leak into the outside world? Then how is
>> rffi supposed to do it better? And how are you planning to export numpypy
>> buffers to non-PyPy code?
>>
>> It's one thing to export low-level data and simple C functions etc. to
>> external code. However, the open C-API of CPython is a serious part of its
>> success story. It's not just legacy code that uses it, it continues to be
>> an important part of the platform. If PyPy can't have such an API, that's a
>> serious drawback of the architecture.
>>
>> Stefan
>
> I think CPython C API is a serious part of it's success because it's
> "good enough for a lot of cases" not because it's necessary for it's
> success. In my opinion a decent FFI (not ctypes, I mean a decent one)
> and better performance would eliminate this need completely. From our
> perspective then, CPython C API is just legacy.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev@python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
Yes, I don't think the full C API is a necessity, although it would be
convenient for Cython. Compiling to pure python (+ctypes) (or python
bytecode, which wouldn't really be much easier or harder) is possible
for the obvious cases, but there are areas where it could be tricky:
1) C++ support
(http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html)
Perhaps the code could be wrapped in an API with external C linkage,
callable through ctypes.
2) cdef (C) functions as C callbacks
For cdef functions called from Cython simply generate a python
function. When you take the address of a cdef function, you get the
address to an actual C function that constructs ctypes objects holding
a pointer to the arguments and the return value that it passes to a
python function. Perhaps for a nogil function original Cython
mechanics could be used if there is no with gil block present, or any
function calls declared with the generic 'except *' clause.
3) managing the GIL
You can't really release the GIL in python code, and if most of your
Cython code isn't actual C code, releasing the GIL won't be feasible.
Through ctypes one could release the GIL, but nogil blocks are much
more coarse grained than that (perhaps nogil blocks should be compiled
to C functions, which are called through a ctypes GIL-releasing
call?). Or maybe explicit releases should just be ignored.
4) buffers (http://docs.cython.org/src/tutorial/numpy.html and
https://sage.math.washington.edu:8091/hudson/job/cython-docs/doclinks/1/src/userguide/memoryviews.html)
Buffers could perhaps be addressed by type-checks only to check
matching dtypes as well as contiguity constraints (and some other
features, like obtaining a memoryview of C data etc
(numpy.ctypeslib.as_array?)), but leaving the rest purely to
(num)pypy.
5) parallelism (http://docs.cython.org/src/userguide/parallelism.html)
This could just be sequential, as it is now in pure-python mode.
Under this model I think the first three points are particularly
though. Could RPython in any way alleviate the countless problems with
this approach (and how?), and would it allow the flexibility to
implement these features, or part of them? Going the python + ctypes
way through to the end, and supporting all the features, will be a lot
of work. Also, I see how much code could be compiled to pure-python
(+ctypes), but I'm pretty sure not all code could be handled this way.
The code that can't be handled will need to call back into pypy
somehow, which would mandate a small (but existing) C API.
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
