Antoine Rougier finished his internship recently so here's a summary of what he did in the "backends" branch. For the record, the goal of his work was to explore how to change our backends into proper plugins so that we can easily avoid hard dependencies between the hwloc core and external libraries such as CUDA, libpci, ...
He defined three types of backends: * "Base" is a normal OS backend such as Linux/Solaris/... This guy manages PU, Cores, Sockets, ... Only one of these can be used at the same time. * "Additional" is what is added to "Base" backends. This is libpci, and things added inside PCI devices (CUDA, ...). Multiple of these may be used at the same time. They are likely invoked sequentially after the Base backend discovery. * "Global" is a special case for XML, Synthetic and Custom, which replaces both Base and Additional. It's not clear that this type will be needed in the end, "Base" with a special flag to disable "Additional" backends might be enough. During init(), all existing backends are placed on a list of "available" backends. And a list of "enabled" backends is initialized to empty. Between init() and load(), calling set_xml(), set_synthetic(), set_fsroot(), ... will append some backends to the "enabled" list (actually two lists, one for Base, one for Additional). During load(), we check whether some Base backends are "enabled". Otherwise we enable the default one of the current OS. Then we actually discover things using the (unique) enabled Base backend followed by all enabled Additional backends. Aside from the main "discover" callback, backends may also define some callbacks to be invoked when new object are created. The main example is Linux creating "OS devices" when a new "PCI device" is added by the PCI backend. CUDA could use that too to fill GPU PCI devices. This is not strictly needed since adding these devices could still be done later, once the PCI backend is done. We'll see. I'll work on putting all this in a branch soon. All this is about making interaction between backends nicer. Once this is done, we'll be able to make actual plugins out of all this. One problem that will come is that some backends are almost directly used from outside the core. For instance exporting a topology to XML is a public API call going down to XML plugin. lstopo and hwloc-ps using Linux-specific tid_get_cpubind() calls causes similar problems. Instead of allowing random API calls into plugin internals, we could keep these backends internal, i.e. not making them plugins. At least for OS backends, it makes sense. "synthetic" and "custom" also have no reason to be pluginified either, they depend on nothing. XML would like to be a plugin so that we stop depending on libxml2, but we have an intermediate level to ease this. The main XML functions do not depend on libxml2, they can remain internal and call either libxml2 or our minimalistic no-libxml2 support underneath. So we can keep the common code and the minimalistic support internal, and only move the libxml2-specific callbacks to a plugin. Summary of what plugins we could have in the end: * for main backends: + internal: synthetic, xml-core, custom, linux, solaris, ... + plugins: pci, cuda, display, ... * for "low-level" xml backends: + internal: minimalistic xml support + plugin: libxml2 * and maybe lstopo backends one day + internal: console, txt, fig, windows + plugin: cairo Brice
