On Tue, Nov 19, 2019 at 4:00 PM Kenneth Hoste <kenneth.ho...@ugent.be
<mailto:kenneth.ho...@ugent.be>> wrote:
Dear Niek,
On 19/11/2019 20:42, Niek de Klein wrote:
> I get
>
> $ eb M4-1.4.17-foss-2018b.eb -D
Please try running this with debugging enabled, and then zip up the log
file from /tmp (see path printed at the line like shown below):
eb --debug M4-1.4.17-foss-2018b.eb -D
> == temporary log file in case of crash
/tmp/eb-ZCIA8v/easybuild-JS80fY.log
>
> <removed a bunch of warnings>
>
> ERROR: Failed to process easyconfig
>
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018b-OpenBLAS-0.3.1.eb:
> Failed to determine minimal toolchain for dep OpenBLAS
0.3.1-GCC-7.3.0-2.30
>
> and a bunch of warnings (full log below).
>
> eb -S OpenBLAS-0.3.1-GCC-7.3.0-2.30
>
> gives
>
>
CFGS1=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/o/OpenBLAS
> * $CFGS1/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb
>
> and
>
> eb OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb -D
> == temporary log file in case of crash
/tmp/eb-VAv5nH/easybuild-9UafJT.log
> Dry run: printing build status of easyconfigs and dependencies
>
CFGS=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs
> * [x] $CFGS/m/M4/M4-1.4.18.eb (module: M4/1.4.18)
> * [x] $CFGS/z/zlib/zlib-1.2.11.eb (module: zlib/1.2.11)
> * [x] $CFGS/h/help2man/help2man-1.47.4.eb (module: help2man/1.47.4)
> * [x] $CFGS/m/M4/M4-1.4.17.eb (module: M4/1.4.17)
> * [x] $CFGS/b/Bison/Bison-3.0.4.eb (module: Bison/3.0.4)
> * [x] $CFGS/f/flex/flex-2.6.4.eb (module: flex/2.6.4)
> * [x] $CFGS/b/binutils/binutils-2.30.eb (module: binutils/2.30)
> * [x] $CFGS/g/GCCcore/GCCcore-7.3.0.eb (module: GCCcore/7.3.0)
> * [x] $CFGS/z/zlib/zlib-1.2.11-GCCcore-7.3.0.eb (module:
> zlib/1.2.11-GCCcore-7.3.0)
> * [x] $CFGS/h/help2man/help2man-1.47.4-GCCcore-7.3.0.eb (module:
> help2man/1.47.4-GCCcore-7.3.0)
> * [x] $CFGS/m/M4/M4-1.4.18-GCCcore-7.3.0.eb (module:
> M4/1.4.18-GCCcore-7.3.0)
> * [x] $CFGS/b/Bison/Bison-3.0.4-GCCcore-7.3.0.eb (module:
> Bison/3.0.4-GCCcore-7.3.0)
> * [x] $CFGS/b/Bison/Bison-3.0.5-GCCcore-7.3.0.eb (module:
> Bison/3.0.5-GCCcore-7.3.0)
> * [x] $CFGS/f/flex/flex-2.6.4-GCCcore-7.3.0.eb (module:
> flex/2.6.4-GCCcore-7.3.0)
> * [x] $CFGS/b/binutils/binutils-2.30-GCCcore-7.3.0.eb (module:
> binutils/2.30-GCCcore-7.3.0)
> * [x] $CFGS/g/GCC/GCC-7.3.0-2.30.eb (module: GCC/7.3.0-2.30)
> * [x] $CFGS/o/OpenBLAS/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb (module:
> OpenBLAS/0.3.1-GCC-7.3.0-2.30)
> == Temporary log file(s) /tmp/eb-VAv5nH/easybuild-9UafJT.log*
have been
> removed.
> == Temporary directory /tmp/eb-VAv5nH has been removed.
>
> == temporary log file in case of crash
/tmp/eb-ZCIA8v/easybuild-JS80fY.log
>
> WARNING: Use of 3 unknown easyconfig parameters detected in
> gompi-2018b.eb: binutilsver, gccver, tcver
> If these are just local variables please rename them to start with
> 'local_', or try using --fix-deprecated-easyconfigs to do this
> automatically.
> For more information, see
>
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
> .
>
>
> WARNING: Deprecated functionality, will no longer work in v5.0:
Use of
> 'dummy' toolchain is deprecated, use 'system' toolchain instead; see
>
http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html
> for more information
>
>
> WARNING: Use of 3 unknown easyconfig parameters detected in
> gompi-2018b.eb: binutilsver, gccver, tcver
> If these are just local variables please rename them to start with
> 'local_', or try using --fix-deprecated-easyconfigs to do this
> automatically.
> For more information, see
>
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
> .
>
>
> WARNING: Use of 2 unknown easyconfig parameters detected in
> GCC-4.9.3-binutils-2.25.eb: binutilsver, mpfr_version
> If these are just local variables please rename them to start with
> 'local_', or try using --fix-deprecated-easyconfigs to do this
> automatically.
> For more information, see
>
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
> .
>
>
> WARNING: Deprecated functionality, will no longer work in v5.0:
Use of
> 'dummy' toolchain is deprecated, use 'system' toolchain instead; see
>
http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html
> for more information
>
>
> WARNING: Use of 2 unknown easyconfig parameters detected in
> GCC-4.9.3-binutils-2.25.eb: binutilsver, mpfr_version
> If these are just local variables please rename them to start with
> 'local_', or try using --fix-deprecated-easyconfigs to do this
> automatically.
> For more information, see
>
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
> .
>
>
> WARNING: Use of 2 unknown easyconfig parameters detected in
> GNU-4.9.3-2.25.eb: binutilsver, gccver
> If these are just local variables please rename them to start with
> 'local_', or try using --fix-deprecated-easyconfigs to do this
> automatically.
> For more information, see
>
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
> .
>
>
> WARNING: Use of 3 unknown easyconfig parameters detected in
> OpenMPI-1.8.8-GNU-4.9.3-2.25.eb: binfile, libfile, libs
> If these are just local variables please rename them to start with
> 'local_', or try using --fix-deprecated-easyconfigs to do this
> automatically.
> For more information, see
>
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
> .
>
>
> WARNING: Use of 2 unknown easyconfig parameters detected in
> GNU-4.9.3-2.25.eb: binutilsver, gccver
> If these are just local variables please rename them to start with
> 'local_', or try using --fix-deprecated-easyconfigs to do this
> automatically.
> For more information, see
>
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
> .
>
> ERROR: Failed to process easyconfig
>
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018b-OpenBLAS-0.3.1.eb:
> Failed to determine minimal toolchain for dep OpenBLAS
0.3.1-GCC-7.3.0-2.30
>
> On Mon, Nov 18, 2019 at 4:28 PM Kenneth Hoste
<kenneth.ho...@ugent.be <mailto:kenneth.ho...@ugent.be>
> <mailto:kenneth.ho...@ugent.be <mailto:kenneth.ho...@ugent.be>>>
wrote:
>
> On 18/11/2019 21:50, Niek de Klein wrote:
> > Hi Kenneth,
> >
> > I spoke too soon, some of my installations work but not
all. When
> I try
> > to isntall Autoconf-2.69-foss-2018.eb I get
> >
> > ERROR: Failed to process easyconfig
> >
>
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69-foss-2018b.eb:
>
> > Failed to determine minimal toolchain for dep M4 1.4.17
> >
> > with eb --search I get:
> >
> > eb --search M4-1.4.17-foss-2018b
> > *
> >
>
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/m/M4/M4-1.4.17-foss-2018b.eb
> >
> > eb --search ^foss-2018b.eb
> > *
> >
>
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/f/foss/foss-2018b.eb
> >
> > So it can find both the M4-1.4.17-foss-2018b.eb file and the
> > foss-2018.eb file.
>
> Can you try "eb M4-1.4.17-foss-2018b.eb -D"?
>
>
> >
> > Kind regards,
> > Niek
> >
> >
> >
> >
> >
> > On Mon, Nov 18, 2019 at 3:21 PM Niek de Klein
> <niekdekl...@gmail.com <mailto:niekdekl...@gmail.com>
<mailto:niekdekl...@gmail.com <mailto:niekdekl...@gmail.com>>
> > <mailto:niekdekl...@gmail.com
<mailto:niekdekl...@gmail.com> <mailto:niekdekl...@gmail.com
<mailto:niekdekl...@gmail.com>>>> wrote:
> >
> > Hi Kenneth,
> >
> > Thanks for your help!
> >
> > 1. is solved now with your suggestion of
> > EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1
> >
> > Now that the bootstrap.py ran correctly it can also
find the
> > easybuild files in the location that I have
configured, so this
> > solves 1 and 2 as well.
> >
> > Thanks,
> > Niek
> >
> > On Mon, Nov 18, 2019 at 3:01 PM Kenneth Hoste
> > <kenneth.ho...@ugent.be
<mailto:kenneth.ho...@ugent.be> <mailto:kenneth.ho...@ugent.be
<mailto:kenneth.ho...@ugent.be>>
> <mailto:kenneth.ho...@ugent.be
<mailto:kenneth.ho...@ugent.be> <mailto:kenneth.ho...@ugent.be
<mailto:kenneth.ho...@ugent.be>>>> wrote:
> >
> > Dear Niek,
> >
> > Sorry that you're hitting these issues, let's try
and get
> them
> > sorted...
> >
> > On 18/11/2019 20:00, Niek de Klein wrote:
> > > Hi all,
> > >
> > > I am trying to have my own installation of
EasyBuild,
> but I
> > have 3 problems:
> > >
> > > 1. When I try to run bootstrap.py it gives an
error that
> > md5sums are
> > > missing, but I'm not sure where to add them as I
> thought that
> > should
> > > then have been included in the
EasyBuild-4.0.1.eb file:
> > >
> > > easybuild.tools.build_log.EasyBuildError:
"Build of
> > > /tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err:
'build
> failed
> > (first 300
> > > chars): Missing checksum for
> easybuild-framework-4.0.1.tar.gz')"
> >
> > Sorry for not responding to the GitHub issue [1] you
> opened on this
> > (will do so now with the answer below).
> >
> > We still need to clean up the bootstrap script a bit
> after the
> > EasyBuild
> > v4.0 release, it's still doing a bunch of stuff it
shouldn't
> > anymore,
> > and it's not compatible yet with Python 3 (see also
> >
https://github.com/easybuilders/easybuild-framework/pull/3028).
> >
> >
> > Your active EasyBuild configuration has
--enforce-checksums
> > enabled (via
> > $EASYBUILD_ENFORCE_CHECKSUMS), which explains the
problem
> you're
> > seeing.
> >
> > The bootstrap script auto-generates an easyconfig
file,
> but it
> > doesn't
> > include checksums (it could I guess, with a bit
more work).
> >
> > You can dance around this though, by temporarily
> disabling the
> > checksums
> > requirement, only for the bootstrap:
> >
> > EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python
bootstrap <path>
> >
> >
> > If you want to, you can bootstrap EasyBuild to a
temporary
> > location (for
> > example /tmp/$USER), and then use that installation to
> pull in an
> > easyconfig file for EasyBuild v4.0.1 that does have
> checksums,
> > using "eb
> > --from-pr 9149 --installpath <final_path>" (which
pulls
> in the
> > easyconfig from
> >
>
https://github.com/easybuilders/easybuild-easyconfigs/pull/9149/files)
> >
> > for the actual installation you intend to use.
> >
> > >
> > >
> > > 2. I changed the robo and installation path to be
> different,
> > but I keep
> > > getting "Failed to determine minimal toolchain
for dep
> ..."
> > errors. The
> > > eb --show-config output is at the end of this
e-mail, and
> > shows that my
> > > robot-paths is
> > >
> >
>
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/,
> >
> > > which is where I cloned the easyconfigs repository.
> >
> > What happens if you use "eb --search" with this setup,
> can 'eb'
> > find
> > easyconfig files?
> >
> > Try something like "eb --search tensorflow-1.14"
for example.
> >
> >
> > >
> > > 3. Even if the robo path wasn't working, tools
that are
> > already in my
> > > module avail are not being picked up. For example,
> EasyBuild
> > that is not
> > > working uses "toolchain = {'name': 'foss',
'version':
> > '2018b'}" and has
> > > as one of the dependencies "('ncurses',
'6.0'),". When
> I try
> > to install
> > > it I get
> > >
> > > ERROR: Failed to process easyconfig
> > >
> >
>
/groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb:
> >
> > > Failed to determine minimal toolchain for dep
ncurses 6.0
> > >
> > > but when I do ml avail ncur I get:
> > >
> > >
------------------------------------------------------
> > > /groups/umcg-biogen/tmp01/apps/modules/devel
> > >
-------------------------------------------------------
> > > ncurses/6.0-foss-2018b
> > >
> > >
> >
>
-------------------------------------------------------------------
> > > /apps/modules/devel
> > >
> >
>
-------------------------------------------------------------------
> > > ncurses/6.0 ncurses/6.1-foss-2018b
> > ncurses/6.1-GCCcore-7.3.0 (D)
> > >
> > > So the ncurses module is already available
> > >
> > >
> > > Am I missing something in my configuration?
> >
> > This is a consequence of having --minimal-toolchains
> enabled, which
> > requires that EasyBuild is able to find the easyconfig
> file for the
> > toolchain (foss-2018b.eb in this case).
> >
> > It needs to figure out what the (versions of) the
> subtoolchains
> > are for
> > foss/2018b, so it can check for ncurses installed with
> GCCcore
> > first,
> > before gompi, before foss.
> >
> > So this is basically the same issue as with
problem 2, I
> bet "eb
> > --search ^foss-2018b.eb" doesn't give you any result?
> >
> >
> > Are you 100% sure the path you specified for
--robot-paths is
> > correct?
> >
> > Note that you can also prepend or append to the robot
> search path
> > (rather than override the default), see
> >
>
https://easybuild.readthedocs.io/en/latest/Using_the_EasyBuild_command_line.html#controlling-robot-search-path
> >
> > .
> >
> > I hope this helps...
> >
> >
> > regards,
> >
> > Kenneth
> >
> >
> > [1]
> https://github.com/easybuilders/easybuild-framework/issues/3079
> >
> > >
> > > Thanks for your help!
> > > Niek de Klein
> > >
> > >
> > > #
> > > # Current EasyBuild configuration
> > > # (C: command line argument, D: default value, E:
> environment
> > variable,
> > > F: configuration file)
> > > #
> > > buildpath (E) =
> > >
/groups/umcg-biogen/tmp01/apps//.tmp/easybuild/builds/
> > > containerpath (D) =
> > > /home/umcg-ndeklein/.local/easybuild/containers
> > > enforce-checksums (E) = True
> > > installpath (E) =
> /groups/umcg-biogen/tmp01/apps/
> > > minimal-toolchains (E) = True
> > > repositorypath (D) =
> > > /home/umcg-ndeklein/.local/easybuild/ebfiles_repo
> > > robot-paths (E) =
> > >
> >
>
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/
> > > set-gid-bit (E) = True
> > > sourcepath (E) =
> > /groups/umcg-biogen/tmp01/apps//sources/
> > > umask (E) = 002
> > > verify-easyconfig-filenames (E) = True
> > >
> > >
> >
>
