I wonder if this is a bug (feature?) with `--module-depends-on`? If you remove that command line arg does it succeed? Did you also have that flag when you used 2019a? I remember in the past that recursive module unloading (which that triggers) can be tricky when it comes to toolchains.
On Wed, 23 Oct 2019 at 11:06, Douglas Scofield <[email protected]> wrote: > Hi Alan, > > > $ eb --dry-run gompi-2018b.eb > == temporary log file in case of crash > /scratch/eb-qUHeQG/easybuild-8PK2Hu.log > Dry run: printing build status of easyconfigs and dependencies > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/m/M4/M4-1.4.18.eb > (module: M4/1.4.18) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/z/zlib/zlib-1.2.11.eb > (module: zlib/1.2.11) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/h/help2man/help2man-1.47.4.eb > (module: help2man/1.47.4) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/m/M4/M4-1.4.17.eb > (module: M4/1.4.17) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/b/Bison/Bison-3.0.4.eb > (module: Bison/3.0.4) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/f/flex/flex-2.6.4.eb > (module: flex/2.6.4) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/b/binutils/binutils-2.30.eb > (module: binutils/2.30) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/g/GCCcore/GCCcore-7.3.0.eb > (module: GCCcore/7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/z/zlib/zlib-1.2.11-GCCcore-7.3.0.eb > (module: zlib/1.2.11-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/h/help2man/help2man-1.47.4-GCCcore-7.3.0.eb > (module: help2man/1.47.4-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/m/M4/M4-1.4.18-GCCcore-7.3.0.eb > (module: M4/1.4.18-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/b/Bison/Bison-3.0.4-GCCcore-7.3.0.eb > (module: Bison/3.0.4-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/b/Bison/Bison-3.0.5-GCCcore-7.3.0.eb > (module: Bison/3.0.5-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/f/flex/flex-2.6.4-GCCcore-7.3.0.eb > (module: flex/2.6.4-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/b/binutils/binutils-2.30-GCCcore-7.3.0.eb > (module: binutils/2.30-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69-GCCcore-7.3.0.eb > (module: Autoconf/2.69-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/g/GCC/GCC-7.3.0-2.30.eb > (module: GCC/7.3.0-2.30) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/a/Automake/Automake-1.16.1-GCCcore-7.3.0.eb > (module: Automake/1.16.1-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/l/libtool/libtool-2.4.6-GCCcore-7.3.0.eb > (module: libtool/2.4.6-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/n/ncurses/ncurses-6.0.eb > (module: ncurses/6.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/a/Autotools/Autotools-20180311-GCCcore-7.3.0.eb > (module: Autotools/20180311-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/x/xorg-macros/xorg-macros-1.19.2-GCCcore-7.3.0.eb > (module: xorg-macros/1.19.2-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/g/gettext/gettext-0.19.8.1.eb > (module: gettext/0.19.8.1) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/n/numactl/numactl-2.0.11-GCCcore-7.3.0.eb > (module: numactl/2.0.11-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/l/libpciaccess/libpciaccess-0.14-GCCcore-7.3.0.eb > (module: libpciaccess/0.14-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/x/XZ/XZ-5.2.4-GCCcore-7.3.0.eb > (module: XZ/5.2.4-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/l/libxml2/libxml2-2.9.8-GCCcore-7.3.0.eb > (module: libxml2/2.9.8-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/h/hwloc/hwloc-1.11.10-GCCcore-7.3.0.eb > (module: hwloc/1.11.10-GCCcore-7.3.0) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/o/OpenMPI/OpenMPI-3.1.1-GCC-7.3.0-2.30.eb > (module: OpenMPI/3.1.1-GCC-7.3.0-2.30) > * [x] > /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/g/gompi/gompi-2018b.eb > (module: gompi/2018b) > == Temporary log file(s) /scratch/eb-qUHeQG/easybuild-8PK2Hu.log* have > been removed. > == Temporary directory /scratch/eb-qUHeQG has been removed. > > > When I look for that eb file in our tree I get multiple sizes, which > puzzles me but I see some seem to be text-substituted versions. > > $ find . -name gompi-2018b.eb -ls > 9246547087760406794 4 -rw-rw-r-- 1 iusan sw 470 Mar 28 > 2019 > ./software/EasyBuild/3.8.1/lib/python2.7/site-packages/easybuild_framework-3.8.1-py2.7.egg/test/framework/easyconfigs/test_ecs/g/gompi/gompi-2018b.eb > 9246547087757627753 4 -rw-rw-r-- 1 iusan sw 448 Mar 28 > 2019 > ./software/EasyBuild/3.8.1/lib/python2.7/site-packages/easybuild_easyconfigs-3.8.1-py2.7.egg/easybuild/easyconfigs/g/gompi/gompi-2018b.eb > 9246547087721524313 4 -rw-r--r-- 1 iusan sw 386 Sep 20 > 15:22 > ./software/EasyBuild/4.0.1/lib/python2.7/site-packages/test/framework/easyconfigs/test_ecs/g/gompi/gompi-2018b.eb > 9246547087736405835 4 -rw-r--r-- 1 iusan sw 435 Sep 20 > 16:17 > ./software/EasyBuild/4.0.1/easybuild/easyconfigs/g/gompi/gompi-2018b.eb > 9246547087747523760 4 -rw-rw-r-- 1 douglas sw 406 Oct 21 > 12:14 ./software/gompi/2018b/easybuild/reprod/gompi-2018b.eb > 9246547087747523765 4 -rw-rw-r-- 1 douglas sw 435 Sep 20 > 16:17 ./software/gompi/2018b/easybuild/gompi-2018b.eb > 9246547087769446482 4 -rw-rw-r-- 1 douglas sw 2520 Oct 21 > 12:14 ./ebfiles_repo/gompi/gompi-2018b.eb > > > $ diff > ./software/EasyBuild/4.0.1/easybuild/easyconfigs/g/gompi/gompi-2018b.eb > ./software/gompi/2018b/easybuild/reprod/gompi-2018b.eb > 1c1 > < easyblock = "Toolchain" > --- > > easyblock = 'Toolchain' > 10,12c10 > < toolchain = SYSTEM > < > < local_gccver = '7.3.0-2.30' > --- > > toolchain = {'name': 'system', 'version': 'system'} > 16,17c14,15 > < ('GCC', local_gccver), # includes both GCC and binutils > < ('OpenMPI', '3.1.1', '', ('GCC', local_gccver)), > --- > > ('GCC', '7.3.0-2.30'), > > ('OpenMPI', '3.1.1', '', ('GCC', '7.3.0-2.30')), > 18a17 > > > > > -Douglas > > PS: Thanks Åke we're getting there :-) > > -- > Douglas G. Scofield > Evolutionary Biology Centre, Uppsala University > [email protected] > [email protected] > > > > On 23 Oct 2019, at 10:51, Alan O'Cais <[email protected]> wrote: > > Hi, > > Did you by any chance modify the definition of gompi/2018b toolchain > (i.e., create a modified version the gompi-2018b.eb file)? What does the > output of > ``` > eb --dry-run gompi-2018b.eb > ``` > look like? > > Alan > > On Wed, 23 Oct 2019 at 10:38, <[email protected]> wrote: > >> Hi, new to EasyBuild as is our center (UPPMAX at Uppsala University). We >> install a lot of bioinformatics software so are looking forward to >> contributing easyconfigs as we learn EasyBuild. >> >> We're trying to install the foss-2018b toolchain but it keeps failing >> attempting to install FFTW-3.3.8-gompi-2018b with what seems to be a >> straightforward config mismatch, but with our *very* limited experience we >> can't see anything obvious. It doesn't install with EasyBuild 3.8.1 >> either, >> but FFTW-3.3.8-foss-2019a installs cleanly. >> >> >> $ eb --module-depends-on --robot --module-syntax=Tcl foss-2018b.eb >> == temporary log file in case of crash >> /scratch/eb-dFaRLw/easybuild-eFncBe.log >> == resolving dependencies ... >> == processing EasyBuild easyconfig /sw/EasyBuild/software/EasyBuild/4.0.1/ >> easybuild/easyconfigs/f/FFTW/FFTW-3.3.8-gompi-2018b.eb >> == building and installing FFTW/3.3.8-gompi-2018b... >> == fetching files... >> == creating build dir, resetting environment... >> == starting iteration #0 ... >> == unpacking... >> == patching... >> == preparing... >> == FAILED: Installation ended unsuccessfully (build directory: >> /sw/EasyBuild/ >> build/FFTW/3.3.8/gompi-2018b): build failed (first 300 chars): List of >> toolchain dependency modules and toolchain definition do not match (found >> [] >> vs expected set(['GCC', 'OpenMPI'])) (took 17 sec) >> == Results of the build can be found in the log file(s) >> /scratch/eb-dFaRLw/ >> easybuild-FFTW-3.3.8-20191023.101730.GDYpv.log >> ERROR: Build of >> /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/easyconfigs/ >> f/FFTW/FFTW-3.3.8-gompi-2018b.eb failed (err: "build failed (first 300 >> chars): >> List of toolchain dependency modules and toolchain definition do not match >> (found [] vs expected set(['GCC', 'OpenMPI']))") >> >> >> $ eb --show-config >> # >> # Current EasyBuild configuration >> # (C: command line argument, D: default value, E: environment variable, F: >> configuration file) >> # >> buildpath (E) = /sw/EasyBuild/build >> containerpath (E) = /sw/EasyBuild/containers >> installpath (E) = /sw/EasyBuild >> packagepath (E) = /sw/EasyBuild/packages >> prefix (E) = /sw/EasyBuild >> repositorypath (E) = /sw/EasyBuild/ebfiles_repo >> robot-paths (D) = /sw/EasyBuild/software/EasyBuild/4.0.1/easybuild/ >> easyconfigs >> sourcepath (E) = /sw/EasyBuild/sources >> > > > -- > Dr. Alan O'Cais > E-CAM Software Manager > Juelich Supercomputing Centre > Forschungszentrum Juelich GmbH > 52425 Juelich, Germany > > Phone: +49 2461 61 5213 > Fax: +49 2461 61 6656 > E-mail: [email protected] > WWW: http://www.fz-juelich.de/ias/jsc/EN > > > > ------------------------------------------------------------------------------------------------ > > ------------------------------------------------------------------------------------------------ > Forschungszentrum Juelich GmbH > 52425 Juelich > Sitz der Gesellschaft: Juelich > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 > Vorsitzender des Aufsichtsrats: MinDir Volker Rieke > Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), > Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, > Prof. Dr. Sebastian M. Schmidt > > ------------------------------------------------------------------------------------------------ > > ------------------------------------------------------------------------------------------------ > > > > > > > > > > > När du har kontakt med oss på Uppsala universitet med e-post så innebär > det att vi behandlar dina personuppgifter. För att läsa mer om hur vi gör > det kan du läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/ > > E-mailing Uppsala University means that we will process your personal > data. For more information on how this is performed, please read here: > http://www.uu.se/en/about-uu/data-protection-policy > -- Dr. Alan O'Cais E-CAM Software Manager Juelich Supercomputing Centre Forschungszentrum Juelich GmbH 52425 Juelich, Germany Phone: +49 2461 61 5213 Fax: +49 2461 61 6656 E-mail: [email protected] WWW: http://www.fz-juelich.de/ias/jsc/EN
