Åke Sandgren <[email protected]> writes: > On 2/20/20 1:01 PM, Loris Bennett wrote: >> Åke Sandgren <[email protected]> writes: >> >>> On 2/13/20 4:32 PM, Loris Bennett wrote: >>>> PS: Slightly strangely, EB has a patch for PMIx 2.2.1 but no actual EC: >>> >>> Now available in >>> https://github.com/easybuilders/easybuild-easyconfigs/pull/9877 >> >> Am I going to need ECs for all the versions of PMIx I need combined with >> all the toolchains I'm using? Or should using --minimal-toolchains >> allow me to get away with PMIx which are just dependent on GCCcore? > > You will need easyconfigs for PMIx of version x for whatever GCCcore > version is used in the toolchain you build something for. > We use PMIx-3.1.4 for the OpenMPI version used in foss(cuda)/2019b since > that version of OpenMPI understands PMIx 3.1.4 (i.e. OpenMPI 3.1.4 does > understand PMIx 3.1.4, but OpenMPI 3.1.2 and 3.1.3 only knows PMIx 3.0.x > > There is a reason for the somewhat complicated rules for OpenMPI vs PMIx > in our hooks. The ones in PR is slightly old and I could send you our > current setup if you want it.
That would be great if you could send me the current version. >> If I try >> >> [build@g004 ~]$ eb TensorFlow-2.1.0-fosscuda-2019b-Python-3.7.4.eb --robot >> --force --dry-run --minimal-toolchains >> == temporary log file in case of crash /tmp/eb-i7YVXd/easybuild-Gj_6g8.log >> == Running parse hook for >> TensorFlow-2.1.0-fosscuda-2019b-Python-3.7.4.eb... >> == Running parse hook for fosscuda-2019b.eb... >> == Running parse hook for GCC-8.3.0.eb... >> == Running parse hook for CUDA-10.1.243-GCC-8.3.0.eb... >> == Running parse hook for GCC-8.3.0.eb... >> == Running parse hook for OpenMPI-3.1.4-gcccuda-2019b.eb... >> == Running parse hook for gcccuda-2019b.eb... >> == Running parse hook for GCCcore-8.3.0.eb... >> == Running parse hook for binutils-2.32-GCCcore-8.3.0.eb... >> == Running parse hook for M4-1.4.18.eb... >> == Running parse hook for binutils-2.32.eb... >> == Running parse hook for GCCcore-8.3.0.eb... >> >> WARNING: Failed to determine toolchain hierarchy for fosscuda/2019b when >> determining subtoolchain for dependency 'Bazel': 'Failed to process >> easyconfig >> /trinity/shared/easybuild/software/EasyBuild/4.1.1/easybuild/easyconfigs/o/OpenMPI/OpenMPI-3.1.4-gcccuda-2019b.eb: >> Failed to determine minimal toolchain for dep PMIx 2.1.3' >> >> ERROR: Failed to process easyconfig >> /trinity/shared/easybuild/software/EasyBuild/4.1.1/easybuild/easyconfigs/t/TensorFlow/TensorFlow-2.1.0-fosscuda-2019b-Python-3.7.4.eb: >> Failed to determine minimal toolchain for dep Bazel 0.29.1 >> >> However, it seems like the right EC for Bazel is available: >> >> [build@g004 ~]$ eb -S ^Bazel-0.29.1-GCCcore* >> >> CFGS1=/trinity/shared/easybuild/software/EasyBuild/4.1.1/easybuild/easyconfigs/b/Bazel >> * $CFGS1/Bazel-0.29.1-GCCcore-8.2.0.eb >> * $CFGS1/Bazel-0.29.1-GCCcore-8.3.0.eb >> >> Or is --minimal-toolchains not the right approach here? > > --minimal-toolchains only tells EB to start looking at the lowest > toolchain/compiler level, i.e. GCCcore, and search upwards to the > topmost level, foss/intel, for easyconfigs. By default it searches > top-down. So if the EC for TensorFlow has a dependency on GCCcore-8.3.0.eb as the output from the parse hook indicates, why is Bazel-0.29.1-GCCcore-8.3.0.eb not considered appropriate if the search starts at the level of GGCcore? Cheers, Loris -- Dr. Loris Bennett (Mr.) ZEDAT, Freie Universität Berlin Email [email protected]
