Here at TAMU we set:
EASYBUILD_MINIMAL_TOOLCHAINS=1
when loading the EasyBuild-<cluster> module (which
sets all our other EASYBUILD_* variables for the cluster).
So, by default, all builds use --minimal-toolchains
In the case where we want a module from a "higher" toolchain
(e.g. foss vs GCCcore), we use: --use-existing-modules to override
(after building the foss version first).
Jack Perdue
Lead Systems Administrator
High Performance Research Computing
TAMU Division of Research
j-per...@tamu.edu http://hprc.tamu.edu
HPRC Helpdesk: h...@hprc.tamu.edu
On 06/07/2018 05:07 AM, Caspar van Leeuwen wrote:
Ah, to answer my own question, I think I know where the difference
comes from. With eb -S I find:
* $CFGS2/l/LibTIFF/LibTIFF-4.0.9-GCCcore-6.4.0.eb
* $CFGS2/l/LibTIFF/LibTIFF-4.0.9-foss-2017b.eb
but the only version for libpng is
* $CFGS1/l/libpng/libpng-1.6.32-GCCcore-6.4.0.eb
I guess that means that EasyBuild looks at which EasyConfigs are
available (and not which modules are actually installed), and thén
goes for the 'maximal-toolchain' unless you specify --minimal-toolchain.
I'm doubting whether I should simply set --minimal-toolchain (=True)
as default on the system. Any people who do that? Any reasons why I
shouldn't?
Cheers,
Caspar
------------------------------------------------------------------------
*From: *"Caspar van Leeuwen" <caspar.vanleeu...@surfsara.nl>
*To: *"easybuild" <easybuild@lists.ugent.be>
*Sent: *Thursday, 7 June, 2018 11:42:37
*Subject: *[easybuild] Dependency resolution for dependencies build
with subtoolchains
Hi all,
I keep being puzzled by the dependency resolution step in EasyBuild.
I'm trying to install GDAL-2.2.3-foss-2017b-Python-2.7.14.eb. Among
many others, it has dependencies on libpng and LibTIFF:
dependencies = [
....
('libpng', '1.6.32'),
('LibTIFF', '4.0.9'),
....
]
On the system, I have
libpng/1.6.32-GCCcore-6.4.0
LibTIFF/4.0.9-GCCcore-6.4.0
installed. EasyBuild happily finds that the libpng dependency is
resolved, since GCCcore-6.4.0 is a subtoolchain of foss-2017b.
However, for LibTIFF, it doesn't: it explicitely looks for a
foss-2017b build, and fails to find one.
* [x] .../easyconfigs/l/libpng/libpng-1.6.32-GCCcore-6.4.0.eb
(module: libpng/1.6.32-GCCcore-6.4.0)
* [ ] .../easyconfigs/l/LibTIFF/LibTIFF-4.0.9-foss-2017b.eb (module:
LibTIFF/4.0.9-foss-2017b)
When passing the --minimal-toolchain option, it does find the
LibTIFF/4.0.9-GCCcore-6.4.0. However, for reproducibility, I prefer to
avoid command line options. Furthermore, I think (?) the
minimal-toolchain option is intended for when a dependency could be
resolved by both the full, and a subtoolchain (e.g. when
LibTIFF/4.0.9-GCCcore-6.4.0 and LibTIFF/4.0.9-foss-2017b would both
be available) and in those cases I generally don't want to use the
minimal-toolchain.
Moreover, I would really like to understand why libpng and LibTIFF are
being resolved differently here. Could it be that one of the other
dependencies of GDAL in turn also has a dependency on libpng or
LibTIFF, and that this somehow changes the behavior? And is that
difference in behavior intentional, or is this a 'bug'?
Regards,
Caspar van Leeuwen
SURFsara
