Now that the project has made it to CMake, I wonder if there is enthusiasm to
break apart GDAL's build system to take advantage of its plugin capability in a
default fashion to cut down on the massiveness of our default
stuff-you-actually-want build situation. People rightly complain that
libgdal.so links too much stuff, and in most cases, people don't need it all.
What if we were to default to having drivers be free-standing CMake projects
that installed versioned plugin shared objects into known locations that could
be activated at library load (or use) time? GDAL has this dynamic plugin
capability right now, but it is not widely used, and build configuration is not
so convenient for its usage. Plugins would allow each of these drivers to link
to their esoteric dependencies without polluting the common library, and the
situation we have now, where multiple drivers have similar sets of
dependencies, could be handled in some kind of hierarchical fashion based on
common dependencies.
For deployments like rasterio's wheels, external GDAL plugins could be loaded
at runtime by the rasterio libgdal.so, and rasterio wouldn't need to be
responsible for building a libgdal.so that provided kitchen sink capabilities.
People could show up with their own driver plugins that included dependencies
as needed.
GDAL's reputation of being hard to build and deploy is well earned and
deserved. IMO, the CMake configuration has increased its build speed, system
flexibility, and commonality on supported platforms, but it hasn't addressed a
core challenge which is GDAL links to the world one DLL at a time. I don't know
if the project wants to prioritize this topic relative to other issues, but we
have some tools and resources available to us to improve the situation if so.
Some thoughts/questions regarding this:
- you definitely don't want to build all drivers (that can be built as
plugins) as plugins. This works (this is actually my dev setup, to
ensure that the core exports all the symbols needed for drivers built as
plugins), but I've found there's a measurable penalty in doing so (300
ms to load the 127 plugins on my system, ie each GDAL command line
invokation will at least take 300 ms, which might be a significant
penalty in some workflows) . So GDAL_ENABLE_PLUGINS_NO_DEPS=YES (cf
https://gdal.org/development/building_from_source.html#cmdoption-arg-GDAL_ENABLE_PLUGINS_NO_DEPS-BOOL)
is typically discouraged. But 10 or so plugins should be fine.
- you can "emulate" building a single driver as plugin, by re-running a
GDAL build with just the dependencies of that driver, and setting
-DGDAL_ENABLE_DRIVER_xxxx_PLUGIN=ON. The only build artifact of interest
in that use case is the [gdal|ogr]_XXX.so/dll corresponding to that
driver. The main inconvenience I can see with this current approach is
that you have to pay the price for the core lib to be rebuilt, whereas
with the free-standing CMake project, you'd point to the installed GDAL
headers & lib
- by free-standing CMake project, what do you mean ? Couldn't that just
the existing frmts/XXXX/CMakeLists.txt file that would detect it is
called as the top level CMakeLists.txt (probably by checking
${CMAKE_CURRENT_SOURCE_DIR} == ${PROJECT_SOURCE_DIR}) and then change
its logic to find libgdal and its dependencies ? And users would "cmake
-S frmts/XXXX -B plugin_XXXX" to configure just for that plugin.
- I don't understand the "where multiple drivers have similar sets of
dependencies, could be handled in some kind of hierarchical fashion
based on common dependencies". why is a special case needed if several
drivers share the same dependencies ? Or maybe you were thinking to
drivers depending on other drivers ? That situation can indeed happen
currently, but when it happens, such drivers aren't allowed to be built
as plugins. They have to be built with libgdal. Both because it makes
the build system simpler, and also to avoid deployment complication
where you'd have to know that gdal_XXXX.so links against gdal_YYYY.so.
- we would need to keep the current monolithic CMake project approach
working, at the very least for people doing static builds, but even for
people doing dynamic builds as it might be still be more practical if
you can install all the dependencies at once. Having the standalone
capability in addition would be an extra complexity, so we need to be
sure it addresses real use cases, and perhaps restrict it to a few
select drivers. Obvious candidates for standalone CMake projects would
likely be for the few drivers that have proprietary dependencies (ECW,
JP2KAK, MrSID, Oracle, ...) and aren't shipped by FOSS binary distributions.
- offering the possibility of building just a driver as a plugin in a
unit way would require people to build against the same GDAL headers as
libgdal. So people would have to download the same version of the GDAL
sources as the one used by the libgdal provided by rasterio + having the
required SDK + installing the libs of the SDK in a path where
Python/rasterio can find them. Is it in the expected cultural background
of rasterio users to do that on their own ?
Even
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
