Dear Maarten,
On 15/08/2017 21:12, Maarten Bosmans wrote:
Hi List,
I'm trying to get to know EasyBuild. So far there's been lots of
little surprises and frustrations, but as I'm slowly getting used to
the whole system, it looks like a nice way to manage building
(scientific) software from source.
To echo David: welcome to the wonderful world of EasyBuild!
A couple of things I still do not fully understand. One of them is why
build tools like CMake and M4 are recompiled for each toolchain. In
most easyconfigs where CMake is in the builddependencies it is
specified without toolchain. So it seems that the installation of
toolchain-specific CMakes is mainly because the limited availability
of CMake easyconfigs using the dummy toolchain. Am I missing
something, or would it make more sense to make an easyconfig for each
CMake version using the dummy toolchain and not for other toolchains?
The same thing goes for flex and bison, but these packages actually
have a lib part, so for some use cases it could actually make sense to
predicate their version on the toolchain.
Historically, this has come from laziness, sort of...
The toolchain used to install a particular software package is by
default inherited by the dependencies, unless a specific toolchain is
specified for them (unless --minimal-toolchains is used, see [1]). That
means that tools like CMake & co potentially get installed with several
toolchains.
We (at HPC-UGent) didn't care much about that, since EasyBuild took care
of installing the matching CMake that we knew that worked, and we didn't
mind the additional modules that got installed that way.
Recently other sites have been pointing out that this is rather
stupid/annoying since it leads to a needless explosion of additional
modules, which led to implementing support for --minimal-toolchains,
where the subtoolchains are being considered first without having to
hard-code a subtoolchain for each dependency (which makes things more
complex later when bumping the toolchain).
Especially the last couple of weeks & months there has been a major
shift towards only having easyconfigs for tools like CMake with the
GCCcore toolchain, which makes them compatible with both the most common
'foss' and 'intel' toolchains, see [2].
Since EasyBuild v3.0, the default behavior is to not only first consider
the parent toolchain, but also consider the subtoolchains after that in
the hunt for a matching easyconfig file (using --minimal-toolchains
basically only flips the order from subtoolchains first to parent
toolchain as last resort).
We prefer from staying away from using the dummy toolchain because then
we're at the mercy of the OS w.r.t. which compiler is being used to
build those tools.
For other similar annoyances like having multiple versions of CMake
installed with a particular toolchain we are also thinking about
solutions, for example being able to 'freeze' dependency versions
(probably depending on the toolchain & toolchain version) through some
kind of configuration file, but this hasn't been implemented yet.
I hope this clarifies things a bit...
regards,
Kenneth
[1]
http://easybuild.readthedocs.io/en/latest/Manipulating_dependencies.html#using-minimal-toolchains-for-dependencies
[2]
https://github.com/easybuilders/easybuild-easyconfigs/pulls?utf8=%E2%9C%93&q=is%3Apr%20sort%3Aupdated-desc%20is%3Aclosed%20GCCcore
