I get $ eb M4-1.4.17-foss-2018b.eb -D == temporary log file in case of crash /tmp/eb-ZCIA8v/easybuild-JS80fY.log
<removed a bunch of warnings> ERROR: Failed to process easyconfig /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018b-OpenBLAS-0.3.1.eb: Failed to determine minimal toolchain for dep OpenBLAS 0.3.1-GCC-7.3.0-2.30 and a bunch of warnings (full log below). eb -S OpenBLAS-0.3.1-GCC-7.3.0-2.30 gives CFGS1=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/o/OpenBLAS * $CFGS1/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb and eb OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb -D == temporary log file in case of crash /tmp/eb-VAv5nH/easybuild-9UafJT.log Dry run: printing build status of easyconfigs and dependencies CFGS=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs * [x] $CFGS/m/M4/M4-1.4.18.eb (module: M4/1.4.18) * [x] $CFGS/z/zlib/zlib-1.2.11.eb (module: zlib/1.2.11) * [x] $CFGS/h/help2man/help2man-1.47.4.eb (module: help2man/1.47.4) * [x] $CFGS/m/M4/M4-1.4.17.eb (module: M4/1.4.17) * [x] $CFGS/b/Bison/Bison-3.0.4.eb (module: Bison/3.0.4) * [x] $CFGS/f/flex/flex-2.6.4.eb (module: flex/2.6.4) * [x] $CFGS/b/binutils/binutils-2.30.eb (module: binutils/2.30) * [x] $CFGS/g/GCCcore/GCCcore-7.3.0.eb (module: GCCcore/7.3.0) * [x] $CFGS/z/zlib/zlib-1.2.11-GCCcore-7.3.0.eb (module: zlib/1.2.11-GCCcore-7.3.0) * [x] $CFGS/h/help2man/help2man-1.47.4-GCCcore-7.3.0.eb (module: help2man/1.47.4-GCCcore-7.3.0) * [x] $CFGS/m/M4/M4-1.4.18-GCCcore-7.3.0.eb (module: M4/1.4.18-GCCcore-7.3.0) * [x] $CFGS/b/Bison/Bison-3.0.4-GCCcore-7.3.0.eb (module: Bison/3.0.4-GCCcore-7.3.0) * [x] $CFGS/b/Bison/Bison-3.0.5-GCCcore-7.3.0.eb (module: Bison/3.0.5-GCCcore-7.3.0) * [x] $CFGS/f/flex/flex-2.6.4-GCCcore-7.3.0.eb (module: flex/2.6.4-GCCcore-7.3.0) * [x] $CFGS/b/binutils/binutils-2.30-GCCcore-7.3.0.eb (module: binutils/2.30-GCCcore-7.3.0) * [x] $CFGS/g/GCC/GCC-7.3.0-2.30.eb (module: GCC/7.3.0-2.30) * [x] $CFGS/o/OpenBLAS/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb (module: OpenBLAS/0.3.1-GCC-7.3.0-2.30) == Temporary log file(s) /tmp/eb-VAv5nH/easybuild-9UafJT.log* have been removed. == Temporary directory /tmp/eb-VAv5nH has been removed. == temporary log file in case of crash /tmp/eb-ZCIA8v/easybuild-JS80fY.log WARNING: Use of 3 unknown easyconfig parameters detected in gompi-2018b.eb: binutilsver, gccver, tcver If these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically. For more information, see https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html . WARNING: Deprecated functionality, will no longer work in v5.0: Use of 'dummy' toolchain is deprecated, use 'system' toolchain instead; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information WARNING: Use of 3 unknown easyconfig parameters detected in gompi-2018b.eb: binutilsver, gccver, tcver If these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically. For more information, see https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html . WARNING: Use of 2 unknown easyconfig parameters detected in GCC-4.9.3-binutils-2.25.eb: binutilsver, mpfr_version If these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically. For more information, see https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html . WARNING: Deprecated functionality, will no longer work in v5.0: Use of 'dummy' toolchain is deprecated, use 'system' toolchain instead; see http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html for more information WARNING: Use of 2 unknown easyconfig parameters detected in GCC-4.9.3-binutils-2.25.eb: binutilsver, mpfr_version If these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically. For more information, see https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html . WARNING: Use of 2 unknown easyconfig parameters detected in GNU-4.9.3-2.25.eb: binutilsver, gccver If these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically. For more information, see https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html . WARNING: Use of 3 unknown easyconfig parameters detected in OpenMPI-1.8.8-GNU-4.9.3-2.25.eb: binfile, libfile, libs If these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically. For more information, see https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html . WARNING: Use of 2 unknown easyconfig parameters detected in GNU-4.9.3-2.25.eb: binutilsver, gccver If these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically. For more information, see https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html . ERROR: Failed to process easyconfig /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018b-OpenBLAS-0.3.1.eb: Failed to determine minimal toolchain for dep OpenBLAS 0.3.1-GCC-7.3.0-2.30 On Mon, Nov 18, 2019 at 4:28 PM Kenneth Hoste <[email protected]> wrote: > On 18/11/2019 21:50, Niek de Klein wrote: > > Hi Kenneth, > > > > I spoke too soon, some of my installations work but not all. When I try > > to isntall Autoconf-2.69-foss-2018.eb I get > > > > ERROR: Failed to process easyconfig > > > /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69-foss-2018b.eb: > > > Failed to determine minimal toolchain for dep M4 1.4.17 > > > > with eb --search I get: > > > > eb --search M4-1.4.17-foss-2018b > > * > > > /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/m/M4/M4-1.4.17-foss-2018b.eb > > > > eb --search ^foss-2018b.eb > > * > > > /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/f/foss/foss-2018b.eb > > > > So it can find both the M4-1.4.17-foss-2018b.eb file and the > > foss-2018.eb file. > > Can you try "eb M4-1.4.17-foss-2018b.eb -D"? > > > > > > Kind regards, > > Niek > > > > > > > > > > > > On Mon, Nov 18, 2019 at 3:21 PM Niek de Klein <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi Kenneth, > > > > Thanks for your help! > > > > 1. is solved now with your suggestion of > > EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 > > > > Now that the bootstrap.py ran correctly it can also find the > > easybuild files in the location that I have configured, so this > > solves 1 and 2 as well. > > > > Thanks, > > Niek > > > > On Mon, Nov 18, 2019 at 3:01 PM Kenneth Hoste > > <[email protected] <mailto:[email protected]>> wrote: > > > > Dear Niek, > > > > Sorry that you're hitting these issues, let's try and get them > > sorted... > > > > On 18/11/2019 20:00, Niek de Klein wrote: > > > Hi all, > > > > > > I am trying to have my own installation of EasyBuild, but I > > have 3 problems: > > > > > > 1. When I try to run bootstrap.py it gives an error that > > md5sums are > > > missing, but I'm not sure where to add them as I thought that > > should > > > then have been included in the EasyBuild-4.0.1.eb file: > > > > > > easybuild.tools.build_log.EasyBuildError: "Build of > > > /tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed > > (first 300 > > > chars): Missing checksum for > easybuild-framework-4.0.1.tar.gz')" > > > > Sorry for not responding to the GitHub issue [1] you opened on > this > > (will do so now with the answer below). > > > > We still need to clean up the bootstrap script a bit after the > > EasyBuild > > v4.0 release, it's still doing a bunch of stuff it shouldn't > > anymore, > > and it's not compatible yet with Python 3 (see also > > https://github.com/easybuilders/easybuild-framework/pull/3028). > > > > > > Your active EasyBuild configuration has --enforce-checksums > > enabled (via > > $EASYBUILD_ENFORCE_CHECKSUMS), which explains the problem you're > > seeing. > > > > The bootstrap script auto-generates an easyconfig file, but it > > doesn't > > include checksums (it could I guess, with a bit more work). > > > > You can dance around this though, by temporarily disabling the > > checksums > > requirement, only for the bootstrap: > > > > EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python bootstrap <path> > > > > > > If you want to, you can bootstrap EasyBuild to a temporary > > location (for > > example /tmp/$USER), and then use that installation to pull in an > > easyconfig file for EasyBuild v4.0.1 that does have checksums, > > using "eb > > --from-pr 9149 --installpath <final_path>" (which pulls in the > > easyconfig from > > > https://github.com/easybuilders/easybuild-easyconfigs/pull/9149/files) > > > > for the actual installation you intend to use. > > > > > > > > > > > 2. I changed the robo and installation path to be different, > > but I keep > > > getting "Failed to determine minimal toolchain for dep ..." > > errors. The > > > eb --show-config output is at the end of this e-mail, and > > shows that my > > > robot-paths is > > > > > > /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/, > > > > > which is where I cloned the easyconfigs repository. > > > > What happens if you use "eb --search" with this setup, can 'eb' > > find > > easyconfig files? > > > > Try something like "eb --search tensorflow-1.14" for example. > > > > > > > > > > 3. Even if the robo path wasn't working, tools that are > > already in my > > > module avail are not being picked up. For example, EasyBuild > > that is not > > > working uses "toolchain = {'name': 'foss', 'version': > > '2018b'}" and has > > > as one of the dependencies "('ncurses', '6.0'),". When I try > > to install > > > it I get > > > > > > ERROR: Failed to process easyconfig > > > > > > > /groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb: > > > > > Failed to determine minimal toolchain for dep ncurses 6.0 > > > > > > but when I do ml avail ncur I get: > > > > > > ------------------------------------------------------ > > > /groups/umcg-biogen/tmp01/apps/modules/devel > > > ------------------------------------------------------- > > > ncurses/6.0-foss-2018b > > > > > > > > > ------------------------------------------------------------------- > > > /apps/modules/devel > > > > > > ------------------------------------------------------------------- > > > ncurses/6.0 ncurses/6.1-foss-2018b > > ncurses/6.1-GCCcore-7.3.0 (D) > > > > > > So the ncurses module is already available > > > > > > > > > Am I missing something in my configuration? > > > > This is a consequence of having --minimal-toolchains enabled, > which > > requires that EasyBuild is able to find the easyconfig file for > the > > toolchain (foss-2018b.eb in this case). > > > > It needs to figure out what the (versions of) the subtoolchains > > are for > > foss/2018b, so it can check for ncurses installed with GCCcore > > first, > > before gompi, before foss. > > > > So this is basically the same issue as with problem 2, I bet "eb > > --search ^foss-2018b.eb" doesn't give you any result? > > > > > > Are you 100% sure the path you specified for --robot-paths is > > correct? > > > > Note that you can also prepend or append to the robot search path > > (rather than override the default), see > > > https://easybuild.readthedocs.io/en/latest/Using_the_EasyBuild_command_line.html#controlling-robot-search-path > > > > . > > > > I hope this helps... > > > > > > regards, > > > > Kenneth > > > > > > [1] > https://github.com/easybuilders/easybuild-framework/issues/3079 > > > > > > > > Thanks for your help! > > > Niek de Klein > > > > > > > > > # > > > # Current EasyBuild configuration > > > # (C: command line argument, D: default value, E: environment > > variable, > > > F: configuration file) > > > # > > > buildpath (E) = > > > /groups/umcg-biogen/tmp01/apps//.tmp/easybuild/builds/ > > > containerpath (D) = > > > /home/umcg-ndeklein/.local/easybuild/containers > > > enforce-checksums (E) = True > > > installpath (E) = > /groups/umcg-biogen/tmp01/apps/ > > > minimal-toolchains (E) = True > > > repositorypath (D) = > > > /home/umcg-ndeklein/.local/easybuild/ebfiles_repo > > > robot-paths (E) = > > > > > > /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/ > > > set-gid-bit (E) = True > > > sourcepath (E) = > > /groups/umcg-biogen/tmp01/apps//sources/ > > > umask (E) = 002 > > > verify-easyconfig-filenames (E) = True > > > > > > > > >
