On Aug 7, 2012, at 5:02 AM, Brice Goglin wrote: > Antoine Rougier finished his internship recently so here's a summary of > what he did in the "backends" branch. For the record, the goal of his > work was to explore how to change our backends into proper plugins so > that we can easily avoid hard dependencies between the hwloc core and > external libraries such as CUDA, libpci, ...
In general, this sounds most excellent. Some specific comments, below. > Aside from the main "discover" callback, backends may also define some > callbacks to be invoked when new object are created. The main example is > Linux creating "OS devices" when a new "PCI device" is added by the PCI > backend. CUDA could use that too to fill GPU PCI devices. This is not > strictly needed since adding these devices could still be done later, > once the PCI backend is done. We'll see. This is a nifty idea. Is the idea that callback can be registered to be fired when a specific PCI vendor / device ID are found? > All this is about making interaction between backends nicer. Once this > is done, we'll be able to make actual plugins out of all this. One > problem that will come is that some backends are almost directly used > from outside the core. For instance exporting a topology to XML is a > public API call going down to XML plugin. lstopo and hwloc-ps using > Linux-specific tid_get_cpubind() calls causes similar problems. Another possibility is deprecating the old functions and making new functions, like hwloc_export_topo_to_file(..., HWLOC_FILE_FORMAT_XML), or something like that. I.e., call a dispatch routine before the actual back-end routine -- this would preserve the abstraction/plugin barrier. (you mention another viable possibility below, too -- I mention this one only for completeness) I don't know if that's attractive or not, but I offer the possibility. :-) > Instead of allowing random API calls into plugin internals, we could > keep these backends internal, i.e. not making them plugins. At least for > OS backends, it makes sense. "synthetic" and "custom" also have no > reason to be pluginified either, they depend on nothing. It might be nice to view all plugins as the same -- regardless of whether they are internal (i.e., part of libhwloc) or external (i.e., a standalone DSO). That way, the majority of the core code doesn't have to know/care whether plugins are internal or external. It would also allow slurping external plugins to be internal, which will be fairly important for embedded mode. A specific case which has come up for this multiple times is when higher-level MPI bindings packages (e.g., Python) dlopen libmpi into a private namespace. When OMPI then tries to dlopen its own DSO/external plugins, they can't find the symbols in libmpi that they depend on (because libmpi is in a private namespace). Hence, OMPI has to be built in a slurp-all-plugins-to-be-internal mode to support such configurations. As such, we'll need hwloc to also support this slurp-all-plugins-to-be-internal kind of mode, too. I can help with the build mojo for this, if desired. > XML would like to be a plugin so that we stop depending on libxml2, but > we have an intermediate level to ease this. The main XML functions do > not depend on libxml2, they can remain internal and call either libxml2 > or our minimalistic no-libxml2 support underneath. So we can keep the > common code and the minimalistic support internal, and only move the > libxml2-specific callbacks to a plugin. > > Summary of what plugins we could have in the end: > * for main backends: > + internal: synthetic, xml-core, custom, linux, solaris, ... > + plugins: pci, cuda, display, ... > * for "low-level" xml backends: > + internal: minimalistic xml support > + plugin: libxml2 > * and maybe lstopo backends one day > + internal: console, txt, fig, windows > + plugin: cairo > > Brice > > _______________________________________________ > hwloc-devel mailing list > hwloc-de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
