Author: Armin Rigo <ar...@tunes.org>
Branch: static-callback-embedding
Changeset: r2576:c251068e143e
Date: 2016-01-13 20:56 +0100
http://bitbucket.org/cffi/cffi/changeset/c251068e143e/
Log: Finalize and review embedding.rst.
diff --git a/doc/source/embedding.rst b/doc/source/embedding.rst
--- a/doc/source/embedding.rst
+++ b/doc/source/embedding.rst
@@ -4,72 +4,123 @@
.. contents::
-From *version 1.5,* you can use CFFI to generate a ``.so/.dll`` which
-exports the API of your choice to any C application that wants to link
-with this ``.so/.dll``.
+You can use CFFI to generate a ``.so/.dll`` which exports the API of
+your choice to any C application that wants to link with this
+``.so/.dll``.
+
+This is entirely *new in version 1.5.*
Usage
-----
+.. __: overview.html#embedding
+
See the `paragraph in the overview page`__ for a quick introduction.
-In this section, we explain every step in more details. We call *DLL*
-the dynamically-loaded library that we are producing; it is a file
-with the (default) extension ``.dll`` on Windows or ``.so`` on other
-platforms. As usual, it is produced by generating some intermediate
-``.c`` code and then calling the regular platform-specific C compiler.
+In this section, we explain every step in more details. We will use
+here this slightly expanded example:
-.. __: overview.html#embedding
+.. code-block:: c
+
+ /* file plugin.h */
+ typedef struct { int x, y; } point_t;
+ extern int do_stuff(point_t *);
+
+.. code-block:: python
+
+ # file plugin_build.py
+ import cffi
+ ffi = cffi.FFI()
+
+ with open('plugin.h') as f:
+ ffi.embedding_api(f.read())
+
+ ffi.set_source("my_plugin", '''
+ #include "plugin.h"
+ ''')
+
+ ffi.embedding_init_code("""
+ from my_plugin import ffi
+
+ @ffi.def_extern()
+ def do_stuff(p):
+ print("adding %d and %d" % (p.x, p.y))
+ return p.x + p.y
+ """)
+
+ ffi.compile(target="plugin-1.5.*", verbose=True)
+
+Running the code above produces a *DLL*, i,e, a dynamically-loadable
+library. It is a file with the extension ``.dll`` on Windows or
+``.so`` on other platforms. As usual, it is produced by generating
+some intermediate ``.c`` code and then calling the regular
+platform-specific C compiler.
+
+Here are some details about the methods used above:
* **ffi.embedding_api(source):** parses the given C source, which
declares functions that you want to be exported by the DLL. It can
also declare types, constants and global variables that are part of
the C-level API of your DLL.
- The functions are automatically defined in the ``.c`` file: they
- contain code that initializes the Python interpreter the first time
- any of them is called, followed by code to call the attached
- Python function (with ``@ffi.def_extern()``, see next point).
+ The functions that are found in ``source`` will be automatically
+ defined in the ``.c`` file: they will contain code that initializes
+ the Python interpreter the first time any of them is called,
+ followed by code to call the attached Python function (with
+ ``@ffi.def_extern()``, see next point).
The global variables, on the other hand, are not automatically
- produced; you have to write their definition explicitly in
- ``ffi.set_source()``, as regular C code.
+ produced. You have to write their definition explicitly in
+ ``ffi.set_source()``, as regular C code (see the point after next).
* **ffi.embedding_init_code(python_code):** this gives
initialization-time Python source code. This code is copied inside
the DLL. At runtime, the code is executed when the DLL is first
initialized, just after Python itself is initialized. This newly
- initialized Python interpreter has got the DLL ready to be imported,
- typically with a line like ``from module_name import ffi, lib``
- (where ``module_name`` is the name given in first argument to
- ``ffi.set_source()``).
+ initialized Python interpreter has got an extra module ready to be
+ imported, typically with a line like "``from my_plugin import ffi,
+ lib``". The name ``my_plugin`` comes from the first argument to
+ ``ffi.set_source()``. (This module represents "the caller's C
+ world" from the point of view of Python.)
- This Python code can import other modules or packages as usual (it
- might need to set up ``sys.path`` first). You should use the
- decorator ``@ffi.def_extern()`` to attach a Python function to each
- of the C functions declared within ``ffi.embedding_api()``. (If you
- don't, calling the C function results for now in a message printed
- to stderr and a zero return value.)
+ The initialization-time Python code can import other modules or
+ packages as usual (it might need to set up ``sys.path`` first). For
+ every function declared within ``ffi.embedding_api()``, it should
+ use the decorator ``@ffi.def_extern()`` to attach a corresponding
+ Python function to it. (Of course, the decorator can appear either
+ directly in the initialization-time Python code, or in any other
+ module that it imports. The usual Python rules apply, e.g. you need
+ "``from my_plugin import ffi``" in a module, otherwise you can't say
+ ``@ffi.def_extern()``.)
-* **ffi.set_source(module_name, c_code):** set the name of the module
- from Python's point of view. It also gives more C code which will
- be included in the generated C code. In simple examples it can be
- an empty string. It is where you would ``#include`` some other
- files, define global variables, and so on. The macro
+ If the initialization-time Python code fails with an exception, then
+ you get tracebacks printed to stderr. If some function remains
+ unattached but the C code calls it, an error message is also printed
+ to stderr and the function returns zero/null.
+
+* **ffi.set_source(c_module_name, c_code):** set the name of the
+ module from Python's point of view. It also gives more C code which
+ will be included in the generated C code. In trivial examples it
+ can be an empty string. It is where you would ``#include`` some
+ other files, define global variables, and so on. The macro
``CFFI_DLLEXPORT`` is available to this C code: it expands to the
platform-specific way of saying "the following declaration should be
- exported from the DLL". For example, you would put "``int
+ exported from the DLL". For example, you would put "``extern int
my_glob;``" in ``ffi.embedding_api()`` and "``CFFI_DLLEXPORT int
my_glob = 42;``" in ``ffi.set_source()``.
-
+
+ Currently, any *type* declared in ``ffi.embedding_api()`` must also
+ be present in the ``c_code``. This is automatic if this code
+ contains a line like ``#include "plugin.h"`` in the example above.
+
* **ffi.compile([target=...] [, verbose=True]):** make the C code and
compile it. By default, it produces a file called
- ``module_name.dll`` or ``module_name.so``, but the default can be
- changed with the optional ``target`` keyword argument. You can use
- ``target="foo.*"`` with a literal ``*`` to ask for a file called
- ``foo.dll`` on Windows or ``foo.so`` elsewhere. (The ``target``
- file name can contain characters not usually allowed in Python
- module names.)
+ ``c_module_name.dll`` or ``c_module_name.so``, but the default can
+ be changed with the optional ``target`` keyword argument. You can
+ use ``target="foo.*"`` with a literal ``*`` to ask for a file called
+ ``foo.dll`` on Windows or ``foo.so`` elsewhere. (One point of the
+ separate ``target`` file name is to include characters not usually
+ allowed in Python module names, like "``plugin-1.5.*``".)
For more complicated cases, you can call instead
``ffi.emit_c_code("foo.c")`` and compile the resulting ``foo.c``
@@ -82,10 +133,57 @@
More reading
------------
-(XXX should point to a few places in the rest of the CFFI docs where
-people starting from embedding would like to go next)
+If you're reading this page about embedding and you are not familiar
+with CFFI already, here are a few pointers to what you could read
+next:
-XXX copy the content of ffi.embedding() to a .h
+* For the ``@ffi.def_extern()`` functions, integer C types are passed
+ simply as Python integers; and simple pointers-to-struct and basic
+ arrays are all straightforward enough. However, sooner or later you
+ will need to read about this topic in more details here__.
+
+* ``@ffi.def_extern()``: see `documentation here,`__ notably on what
+ happens if the Python function raises an exception.
+
+* In embedding mode, the major direction is C code that calls Python
+ functions. This is the opposite of the regular extending mode of
+ CFFI, in which the major direction is Python code calling C. That's
+ why the page `Using the ffi/lib objects`_ talks first about the
+ latter, and why the direction "C code that calls Python" is
+ generally referred to as "callbacks" in that page. (If you also
+ need to have your Python code call C code, read more about
+ `Embedding and Extending`_ below.)
+
+* ``ffi.embedding_api(source)``: follows the same syntax as
+ ``ffi.cdef()``, `documented here.`__ You can use the "``...``"
+ syntax as well, although in practice it may be less useful than it
+ is for ``cdef()``. On the other hand, it is expected that often the
+ C sources that you need to give to ``ffi.embedding_api()`` would be
+ exactly the same as the content of some ``.h`` file that you want to
+ give to users of your DLL. That's why the example above does this::
+
+ with open('foo.h') as f:
+ ffi.embedding(f.read())
+
+ Note that a drawback of this approach is that ``ffi.embedding()``
+ doesn't support ``#ifdef`` directives. You may have to use a more
+ convoluted expression like::
+
+ with open('foo.h') as f:
+ lines = [line for line in f if not line.startswith('#')]
+ ffi.embedding(''.join(lines))
+
+ As in the example above, you can also use the same ``foo.h`` from
+ ``ffi.set_source()``::
+
+ ffi.set_source('module_name', '#include "foo.h"')
+
+
+.. __: using.html#working
+.. __: using.html#def-extern
+.. __: cdef.html#cdef
+
+.. _`Using the ffi/lib objects`: using.html
Embedding and Extending
@@ -98,29 +196,19 @@
variables declared in ``ffi.embedding_api()`` (it is how you should
read/write the global variables from Python).
-But you can use ``ffi.cdef()`` *in addition to*
-``ffi.embedding_api()`` to exchange more C functions and global
-variables between C and Python, without also making them exports of
-the DLL. See `here for more about cdef.`__
+You can use *both* ``ffi.embedding_api()`` and ``ffi.cdef()`` in the
+same build script. You put in the former the declarations you want to
+be exported by the DLL; you put in the latter only the C functions and
+types that you want to share between C and Python, but not export from
+the DLL.
-.. __: cdef.html#cdef
-
-``ffi.cdef()`` is used to access functions and variables that you can,
-and should, define as ``static`` in ``set_source()``. On the other
-hand, the C functions declared with ``ffi.embedding_api()`` work
-similarly to ``extern "Python"`` functions from ``ffi.cdef()``.
-See `here for more about extern "Python".`__ See `here for details
-about @ffi.def_extern().`__
-
-.. __: using.html#extern-python
-.. __: using.html#extern-python-ref
-
-In some cases, you want to write a DLL-exported C function in C
-directly, maybe to handle some cases before calling Python functions.
-To do that, you must *not* write the function's signature in
-``ffi.embedding_api()``. You must only write the custom function
-definition in ``ffi.set_source()``, and prefix it with the macro
-CFFI_DLLEXPORT:
+As an example of that, consider the case where you would like to have
+a DLL-exported C function written in C directly, maybe to handle some
+cases before calling Python functions. To do that, you must *not* put
+the function's signature in ``ffi.embedding_api()``. (Note that this
+requires more hacks if you use ``ffi.embedding(f.read())``.) You must
+only write the custom function definition in ``ffi.set_source()``, and
+prefix it with the macro CFFI_DLLEXPORT:
.. code-block:: c
@@ -131,20 +219,25 @@
This function can, if it wants, invoke Python functions using the
general mechanism of "callbacks" (technically a call from C to Python,
-although in this case it is not calling anything back): you need a
-``ffi.cdef()`` with "``extern "Python" int mycb(int);``", and then you
-can write this in ``ffi.set_source()``:
+although in this case it is not calling anything back):
-.. code-block:: c
+.. code-block:: python
- static int mycb(int); /* the callback: forward declaration, to make
- it accessible from the C code that follows */
+ ffi.cdef("""
+ extern "Python" int mycb(int);
+ """)
- CFFI_DLLEXPORT int myfunc(int a, int b)
- {
- int product = a * b; /* some custom C code */
- return mycb(product);
- }
+ ffi.set_source("my_plugin", """
+
+ static int mycb(int); /* the callback: forward declaration, to make
+ it accessible from the C code that follows
*/
+
+ CFFI_DLLEXPORT int myfunc(int a, int b)
+ {
+ int product = a * b; /* some custom C code */
+ return mycb(product);
+ }
+ """)
and then the Python initialization code needs to contain the lines:
@@ -165,5 +258,12 @@
As the above explanation hints, this is how ``ffi.embedding_api()``
actually implements function calls that directly invoke Python code;
-we have merely decomposed it explicitly, in order to add some custom C
-code in the middle.
+here, we have merely decomposed it explicitly, in order to add some
+custom C code in the middle.
+
+In case you need to force, from C code, Python to be initialized
+before the first ``@ffi.def_extern()`` is called, you can do so by
+calling the C function ``cffi_start_python()`` with no argument. It
+returns an integer, 0 or -1, to tell if the initialization succeeded
+or not. Currently there is no way to prevent a failing initialization
+from also dumping a traceback and more information to stderr.
diff --git a/doc/source/using.rst b/doc/source/using.rst
--- a/doc/source/using.rst
+++ b/doc/source/using.rst
@@ -604,8 +604,6 @@
""")
-.. _extern-python-ref:
-
Extern "Python": reference
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -632,6 +630,8 @@
return a default value. This can be controlled with ``error`` and
``onerror``, described below.
+.. _def-extern:
+
The ``@ffi.def_extern()`` decorator takes these optional arguments:
* ``name``: the name of the function as written in the cdef. By default
_______________________________________________
pypy-commit mailing list
pypy-commit@python.org
https://mail.python.org/mailman/listinfo/pypy-commit
