On Sun, 23 Feb 2020, Daniel Alley wrote:
I would like to package this library as a pre-built Python wheel:
https://github.com/fedora-modularity/libmodulemd
This library uses PyGObject, so importing it looks like this:
import gi
gi.require_version("Modulemd", "2.0")
from gi.repository import Modulemd
I can't find any examples of this, nor any documentation, nor discussion about
it. It seems like it would be, at minimum, a bit more complicated than
libraries based on normal bindings. Is this
possible, and are there any special requirements that are needed to do so?
It's been a bit since I've used gobject-introspection, but I _think_ the
way this works for a normal (OS-installed) GObject package is
a) the package ordinarily provides no actual Python library
b) the package provides a girepository-1.0/Modulemd-2.0.typelib file in
the system lib directory
c) when you import it, PyGObject loads the C libmodulemd library and
generates Python bindings based on the typelib file
d) as a special case, a package _can_ provide a Python "override" library,
but that amends the autogenerated bindings, it's not a complete set of
bindings on its own (and it's Python code, not native code). Modulemd
appears to do this.
So your users' code needs to be able to
- import PyGObject (and libgobject) itself, either from the system or from
their virtualenv
- import the C libmodulemd library from your wheel, which you can compile
for manylinux1
- find the typelib file, which you can put in your wheel
- point libgobject at the typelib file and the C library
- point PyGObject at the override file (which should hopefully be
automatic if it's on sys.path)
The GObject docs https://developer.gnome.org/gi/stable/GIRepository.html
say:
GIRepository will typically look for a girepository-1.0 directory under
the library directory used when compiling gobject-introspection.
It is possible to control the search paths programmatically, using
g_irepository_prepend_search_path(). It is also possible to modify the
search paths by using the GI_TYPELIB_PATH environment variable. The
environment variable takes precedence over the default search path and
the g_irepository_prepend_search_path() calls.
You will also need to make sure the C library is importable. For "normal"
Python wheels, users import a compiled Python shared object (so that the
usual Python path is used as the search path), and that shared object has
a normal shared object dependency on the underlying C library which is
also shipped in the wheel.auditwheel sets an $ORIGIN-relative rpath in
that .so file (using patchelf) so that the Python module, having been
found inside the virtualenv, can find its C library in a relative path to
its own location. Since there is no compiled Python module in your case,
because PyGObject is dynamically generating the bindings at runtime, I
don't think there is a straightforward way of informing PyGObject of where
to find the C library.
Personally, I'd approach this by first aiming for an 80% solution where I
expect users to set GI_TYPELIB_PATH and LD_LIBRARY_PATH so that the
typelib file and the C library can both be found, i.e., they use it by
running something to the effect of
os.setenv("GI_TYPELIB_PATH", "myvenv/lib/girepository-1.0")
os.setenv("LD_LIBRARY_PATH", "myvenv/lib")
gi.require_version("Modulemd", "2.0")
from gi.repository import Modulemd
That would let me confirm that I've actually gotten all the libraries
compiling properly inside the wheel and the code actually works. Then
there's a question of how to do this automatically. A 90% solution would
be to just decide that your wheel has a top-level Python module to do
this, e.g., you tell your users that if they're using the wheel they just
do "import Modulemd" and you create a Modulemd.py that does
os.setenv("GI_TYPELIB_PATH", some relative path from __file__)
etc.
(For bonus points, call g_irepository_prepend_search_path() / see if
PyGObject has some binding to it, instead of setting $GI_TYPELIB_PATH, and
use ctypes to load the actual C library using RTLD_GLOBAL so that it's
already loaded when PyGObject goes looking for it, instead of setting
$LD_LIBRARY_PATH.)
In my (naive) opinion, a 100% solution here would be teaching PyGObject
how to find both typelib files and C libraries in paths based on sys.path,
and then your users could use the standard upstream import instructions
unmodified. (Actually, it's possible PyGObject does this already, but I
don't immediately see anything about it in the docs, and my assumption is
if you can't find examples of others doing this, the use case hasn't come
up.)
One other question is whether your users are importing libgobject from the
system or from a wheel. For the average desktop Linux user, it's probably
fine to get libgobject from the system (and probably _preferable_ - you
likely want the same version as the Gtk/GNOME/etc. libraries they might
import, and if they're importing any of those, they almost certainly want
the system version of Gtk etc.) It appears that PyGObject is on PyPI as
sdists only, so if you don't want to assume your users have libgobject
installed, you may have to first fight the battle of packaging up GObject,
GLib, etc. into wheels.
(Relatedly, I'm guessing the reason nobody has done this yet is that most
software that supports GObject introspection is GNOME-related in some
fashion and therefore most people want it from their OS package manager
and not from a wheel.)
Again, it's been a while since I've worked with GObject introspection, so
if I got something wrong, anyone should feel free to correct me :)
--
Geoffrey Thomas
https://ldpreload.com
geo...@ldpreload.com
_______________________________________________
Wheel-builders mailing list
Wheel-builders@python.org
https://mail.python.org/mailman/listinfo/wheel-builders
