Hi all, Fijal and me would like to raise interest among various groups of people about building a better ctypes replacement for Python.
The general background first, at least as far as we know it. People generally agree that CPython extension modules are not extremely hard to write, but also not the ultimate solution in the matter of interfacing with C code. Obviously you need to know and write C code, and learn the large CPython C API. Moreover a CPython extension module is rather specific to CPython or even a particular version of it. Even if you are lucky and your CPython extensions work on all 2.x versions, it won't work unmodified on Python 3.x, and be slow (or not work at all) on PyPy. (I'm leaving Jython and IronPython out of the picture here but it's very far from ideal for them too.) The working alternatives nowadays are Cython and ctypes. For Cython you have to learn a whole new language; we can argue infinitely about preferences, but from my point of view it's mostly just a nicer way to write CPython C extensions using --- instead of C --- a custom language. About ctypes, I would say that the fact that its use remains marginal is enough to show that its mission kind of failed. Its problems are more than one, but most importantly the level at which you describe the information is slightly too low - it's ABI-level instead of API-level. It does not make a difference for some libraries, but for some it does make a huge difference, like when you use macros. We would like something that can be potentially used to wrap almost all C libraries. So we would like to propose something (a bit more publically than the two subprojects of PyPy attempting to do that --- did you know about their existence?). The goal would be to come up with a better-designed project. "Better designed" in my vocabulary implies "simpler". I'm starting in this mailing list but the idea is to try to move it in its own project, and to develop at the same time at least a CPython 2.7 and 3.x and PyPy implementation. The simplest FFI we know of for a high-level language is LuaJIT's FFI. If you are interested, read the four pages starting at http://luajit.org/ext_ffi.html . A crucial design decision was to avoid making use of tons of objects representing the C types; instead, the user mostly specifies C types as plain strings, simplifying the basic API a lot. So we would like to propose some design very similar to that one. The basic public API looks more or less fine to me. There are of course preferences and topics to discuss, but let's leave that for later. Most importantly, we would welcome a system where the C declarations can be parsed either internally or (for more complicated cases) externally, with the help of a real C compiler. This might include cases like macros, where the user would need to write (as C code) a small function calling the macro, and have it compiled when he imports his Python module. Also, because it would parse strings, we can also add support for parsing strings in the format of Cython declarations. This would still lower the entry barrier, and have the nice effect that the user can sometimes copy-and-paste Cython declarations, instead of copy-and-pasting carefully chosen pieces of a potentially messy .h file full of custom macros (see for example /usr/include/zlib.h). The benefit would be that, unlike Cython, he gets to call the declared functions directly from his regular Python code. Opinions? Interests? This mail is deliberately low on details about how we think we can do it. Instead I'm seeking general reactions for now and would like to move this soon to its own project, independent of PyPy. A bientôt, Armin. _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
