Author: Armin Rigo <ar...@tunes.org> Branch: extradoc Changeset: r5576:4e6f2b0b4afb Date: 2016-01-02 19:22 +0100 http://bitbucket.org/pypy/extradoc/changeset/4e6f2b0b4afb/
Log: Draft blog post diff --git a/blog/draft/cffi-embedding.rst b/blog/draft/cffi-embedding.rst new file mode 100644 --- /dev/null +++ b/blog/draft/cffi-embedding.rst @@ -0,0 +1,81 @@ +======================== +Using CFFI for embedding +======================== + +CFFI_ has been a great success so far to call C libraries in your +Python programs, in a way that is both simple and that works across +CPython 2.x and 3.x and PyPy. + +We are now adding support for *embedding* Python inside non-Python +programs. This is traditionally done using the CPython C API: from C +code, you call ``Py_Initialize()`` and then some other functions like +``PyRun_SimpleString()``. In the simple cases it is, indeed, simple +enough; but it can become a more complicated story if you throw in +supporting application-dependent object types, and correctly running +on multiple threads, and so on. + +Moreover, this approach is specific to CPython (2.x or 3.x, which you +can do in a similar way). It does not work on PyPy, which has its own +smaller `embedding API`_. + +CFFI now supports embedding directly---and there is no fixed API at +all. The idea is to write some Python script with a ``cdef()`` which +declares a number of ``extern "Python"`` functions. When running the +script, it creates the C source code and compile it to a +dynamically-linked library (``.so`` on Linux). This is the same as in +the regular API-mode usage, and ``extern "Python"`` was `introduced in +CFFI 1.4`_. What is new is that you also give a bit of +initialization-time Python code directly in the script, which will be +compiled into the ``.so`` too---and the ``extern "Python"`` are now +also "exported" from the dynamically-linked library as regular C +functions. + +In other words, this library can now be used directly from any C +program (and it is still importable in Python). It exposes the C API +of your choice, which you specified with the ``extern "Python"`` +declarations. You can use it to make whatever custom API makes sense +in your particular case. You can even make directly a "plug-in" for +any program that supports them, just by exporting the API expected for +such plugins. + +This is still being finalized, but please try it out. (You can also +see `embedding.py`_ directly online here for a quick glance.) + +* get the branch ``static-callback-embedding`` of CFFI (``hg clone https://bitbucket.org/cffi/cffi && hg up static-callback-embedding``) + +* make the ``_cffi_backend.so`` (``python setup_base.py build_ext -f -i``) + +* run ``embedding`` in the ``demo`` directory (``cd demo; PYTHONPATH=.. python embedding.py``) + +* run ``gcc`` to build the C sources (``gcc -shared -fPIC _embedding_cffi.c -o _embedding_cffi.so -lpython2.7 -I/usr/include/python2.7``) + +* try out the demo C program in ``embedding_test.c`` (``gcc embedding_test.c _embedding_cffi.so && PYTHONPATH=.. LD_LIBRARY_PATH=. a.out``). + +Note that if you get ``ImportError: cffi extension module +'_embedding_cffi' has unknown version 0x2701``, it means that the +``_cffi_backend`` module loaded is a pre-installed one instead of the +more recent one in ``..``. Be sure to use ``PYTHONPATH`` for now. + +Very similar steps can be followed on PyPy, but it requires the +``cffi-static-callback-embedding`` branch of PyPy, which you must +first translate from sources. + +You get a CPython/PyPy that is automatically initialized (using locks +in case of multi-threading) the first time any of the ``extern +"Python"`` functions is called from the C program. The custom +initialization-time Python code is run at that time too. If this code +starts to be big, you may consider moving it to several modules or +packages and importing them from the initialization-time Python code; +in that case you have to be careful about setting up the correct +``sys.path``. + +Note that right now it does not support CPython's notion of multiple +subinterpreters. The logic creates a single global Python interpreter, +and everything is run in that context. Idea about how to support that +cleanly would be welcome ``:-)`` More generally, any feedback is +appreciated. + + +Have fun, + +Armin _______________________________________________ pypy-commit mailing list pypy-commit@python.org https://mail.python.org/mailman/listinfo/pypy-commit
