On Tue, 31 Mar 2020 at 19:03, Mark Olesen <[email protected]> wrote:
> I've been in discussion with Kurt about the openfoam package and have > been looking at adjusting (improving) its structure. > Being very new to debian (and especially debian packaging), there are a > few things that I'd like to get some more ideas about - thus the posting > here. > > With OpenFOAM it is fairly common for users to have multiple versions > installed at any one time. For example, to migrate coding for API > changes between versions, or to validate the results of models using > different versions before making a complete switch, or even if an > element is partly broken in a newer version but works in an older version. > Many big systems share the need for multiple versions. These systems are often constantly evolving so any distro package will quickly be outdated for active research. Distro packages do, however have advantages for student labs or workshops where ease of installation is more important than having the most recent updates. NASA provides OCSSW, an ocean remote sensing package <https://oceandata.sci.gsfc.nasa.gov/ocssw/>, that is independent of any distribution. This package supports legacy sensors and is constantly adding new sensors, so has to support decades old data formats as well as current data. It uses hdf4, hdf5, and netcdf libraries. Past experience has been that distro versions of these libraries could not be trusted. For example, the ncdump program could be configured to support netcdf or hdf4. NASA provided both versions, renaming the hdf4 version to ncdump_hdf. These libraries have a number of compile-time configuration options that may vary from the choices used by other packages. A second example is TeX Live. The "official CTAN" version installed in a top-level directory. It is common to have multiple versions installed and switch between them by adjusting the PATH. Some packages are closely tied to particular compiler and/or library versions, so commonly include build tools and 3rd party libraries in the system's directory. Environment modules, luamod, or ad hoc sourced scripts are used to adjust user's environment. For some of these systems, users routinely modify and rebuild parts of the system so need stable compliers and 3rd party libraries. Needless churn occurs when such big systems are packaged in the usual way. System maintainers and users have to sort out distro-specific organization changes and deal with problems resulting from distro updates to build tools or 3rd party libraries. > > In the RPM world, I've handled this by an admittedly somwehat ugly > approach. > > 1) Version-specific packages such as openfoam1906, openfoam1912 etc with > a "minor" version according to the patch-level (eg, > > https://build.opensuse.org/package/view_file/science/openfoam1912/openfoam1912.spec > ) > > 2) An openfoam meta-package to specify the latest production version by > requiring the version-specific package (eg, > https://build.opensuse.org/package/view_file/science/openfoam/openfoam.spec > ) > > > To allow simultaneous installation of multiple versions, I'm installing > into /opt/openfoam1912, /opt/openfoam1906 etc. This is okay for a > third-party package, but not for a debian-package. What would be the > correct place for these? > IMO, using alternatives is not viable. There are simply too many exes > and libs, names may change, etc. > The NASA OCSSW software uses environment variables for the locations of various types of data. Binaries use relative paths for dynamic libraries, so are selected by changing the PATH variable. Installation is done with a python script that fetches git bundles. The top level directory contains a shell script that is sourced to set the environment. It is common to have a couple versions as well as a version used for testing and debugging. > Had thought about dropping everything into a multi-arch directory, but > not sure how that should be properly realised, and where the sources > would then be located. > > Looking for ideas and discussion. > For TeX Live, many distros package a recent version. This gives a common base for other packages that use TeX/LaTeX. For multi-year projects (books) a CTAN version can provide stability. There are clearly use cases that are not well served by distro packages, resulting in ad-hoc approaches. This creates difficulties for people moving between projects, as they have to learn new conventions and practices for each new big system. I think there are always going to be systems that have to installed outside the distro structure, but it could be helpful to organize packages (as you appear to have done) so it is easy to move to a distro package. This also means the internal organization of the system would follow distro conventions, making it easier for a new user to find things. As you noted, the alternatives system is not appropriate for switching between versions of large systems. Environment modules have been in use since Sun and SGI were popular. Environment Canada has used an in-house system -- not sure if it is still active -- which may have been done to get around some limitation of the environment modules approach. -- George N. White III
