Hi all, At JSC we use more or less the same setup as Jack to manage things.
Mikael, there are many people using flat naming schemes, for them it's less complicated (from a user perspective) to upgrade everything to the top level toolchain so that everything is visually coherent. There have been a few suggestions at various points to tackle this, "fat" easyconfigs is still my favourite option (yaml format easyconfigs for tackling all toolchains for a software/version in a single file). There are a couple of open PRs somewhere related to it. It would probably be easier in the short term to have an option to allow them to bump the toolchain to the top of the hierarchy, that would also be of benefit in the yaml format. It's a bit delicate though and you need to be careful. Alan On Fri, 8 Jun 2018, 09:47 Mikael Öhman, <micket...@gmail.com<mailto:micket...@gmail.com>> wrote: This brings up a different question though; Why does LibTIFF-4.0.9-foss-2017b.eb exists? >From what i can see, it should be identical to the GCCcore version. Best regards, Mikael On Thu, Jun 7, 2018 at 12:36 PM, Jack Perdue <j-per...@tamu.edu<mailto:j-per...@tamu.edu>> wrote: Here at TAMU we set: EASYBUILD_MINIMAL_TOOLCHAINS=1 when loading the EasyBuild-<cluster> module (which sets all our other EASYBUILD_* variables for the cluster). So, by default, all builds use --minimal-toolchains In the case where we want a module from a "higher" toolchain (e.g. foss vs GCCcore), we use: --use-existing-modules to override (after building the foss version first). Jack Perdue Lead Systems Administrator High Performance Research Computing TAMU Division of Research j-per...@tamu.edu<mailto:j-per...@tamu.edu> http://hprc.tamu.edu HPRC Helpdesk: h...@hprc.tamu.edu<mailto:h...@hprc.tamu.edu> On 06/07/2018 05:07 AM, Caspar van Leeuwen wrote: Ah, to answer my own question, I think I know where the difference comes from. With eb -S I find: * $CFGS2/l/LibTIFF/LibTIFF-4.0.9-GCCcore-6.4.0.eb * $CFGS2/l/LibTIFF/LibTIFF-4.0.9-foss-2017b.eb but the only version for libpng is * $CFGS1/l/libpng/libpng-1.6.32-GCCcore-6.4.0.eb I guess that means that EasyBuild looks at which EasyConfigs are available (and not which modules are actually installed), and thén goes for the 'maximal-toolchain' unless you specify --minimal-toolchain. I'm doubting whether I should simply set --minimal-toolchain (=True) as default on the system. Any people who do that? Any reasons why I shouldn't? Cheers, Caspar ------------------------------------------------------------------------ *From: *"Caspar van Leeuwen" <caspar.vanleeu...@surfsara.nl<mailto:caspar.vanleeu...@surfsara.nl>> *To: *"easybuild" <easybuild@lists.ugent.be<mailto:easybuild@lists.ugent.be>> *Sent: *Thursday, 7 June, 2018 11:42:37 *Subject: *[easybuild] Dependency resolution for dependencies build with subtoolchains Hi all, I keep being puzzled by the dependency resolution step in EasyBuild. I'm trying to install GDAL-2.2.3-foss-2017b-Python-2.7.14.eb. Among many others, it has dependencies on libpng and LibTIFF: dependencies = [ .... ('libpng', '1.6.32'), ('LibTIFF', '4.0.9'), .... ] On the system, I have libpng/1.6.32-GCCcore-6.4.0 LibTIFF/4.0.9-GCCcore-6.4.0 installed. EasyBuild happily finds that the libpng dependency is resolved, since GCCcore-6.4.0 is a subtoolchain of foss-2017b. However, for LibTIFF, it doesn't: it explicitely looks for a foss-2017b build, and fails to find one. * [x] .../easyconfigs/l/libpng/libpng-1.6.32-GCCcore-6.4.0.eb (module: libpng/1.6.32-GCCcore-6.4.0) * [ ] .../easyconfigs/l/LibTIFF/LibTIFF-4.0.9-foss-2017b.eb (module: LibTIFF/4.0.9-foss-2017b) When passing the --minimal-toolchain option, it does find the LibTIFF/4.0.9-GCCcore-6.4.0. However, for reproducibility, I prefer to avoid command line options. Furthermore, I think (?) the minimal-toolchain option is intended for when a dependency could be resolved by both the full, and a subtoolchain (e.g. when LibTIFF/4.0.9-GCCcore-6.4.0 and LibTIFF/4.0.9-foss-2017b would both be available) and in those cases I generally don't want to use the minimal-toolchain. Moreover, I would really like to understand why libpng and LibTIFF are being resolved differently here. Could it be that one of the other dependencies of GDAL in turn also has a dependency on libpng or LibTIFF, and that this somehow changes the behavior? And is that difference in behavior intentional, or is this a 'bug'? Regards, Caspar van Leeuwen SURFsara ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------
