Re: [easybuild] Can't get local install to work

[Niek de Klein]

Dear Kenneth,

I had easyconfigs that I used on an old cluster mixed into the new
easyconfig files which I was installing on a new cluster, which is why the
files were still from EasyBuild 3.x. I have removed all the old easyconfigs
that I had included and am now remaking them using the new easyconfig info.
For the easyconfig that I had the previous problem this fixes it.

Thanks a lot for your help,
Niek

On Tue, Nov 19, 2019 at 4:40 PM Kenneth Hoste 
wrote:

> Dear Niek,
>
> On 19/11/2019 22:28, Niek de Klein wrote:
> > Dear Kenneth,
> >
> > Attached the log.
>
> Thank you, that's helpful to figure this out.
>
> So, here's what's going on...
>
> When resolving dependencies for M4-1.4.17-foss-2018b.eb, EasyBuild goes
> looking for the easyconfigs for the subtoolchains of foss/2018b,
> starting with gompi/2018b.
>
> It finds an easyconfig for gompi/2018b at
>
> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/g/gompi/gompi-2018b.eb
>
> This easyconfig file specifies that gompi/2018b consists of GCC 4.9.3,
> binutils 2.25 and OpenMPI 1.8.8, which is very incorrect...
>
> This makes things go horribly wrong when it tries to find the easyconfig
> for OpenBLAS, eventually.
>
>
> So, question is why that gompi-2018b.eb easyconfig file isn't what it's
> supposed to be (see
>
> https://github.com/easybuilders/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/g/gompi/gompi-2018b.eb
> ).
>
> The warnings you're hitting also show that you're using a copy of the
> easyconfig files from an EasyBuild 3.x version.
>
> The easyconfigs included in EasyBuild 4.0.x have been cleaned up to
> avoid those warnings...
>
>
>
> regards,
>
> Kenneth
> >
> > On Tue, Nov 19, 2019 at 4:00 PM Kenneth Hoste  > > wrote:
> >
> > Dear Niek,
> >
> > On 19/11/2019 20:42, Niek de Klein wrote:
> >  > I get
> >  >
> >  > $ eb M4-1.4.17-foss-2018b.eb -D
> >
> >
> > Please try running this with debugging enabled, and then zip up the
> log
> > file from /tmp (see path printed at the line like shown below):
> >
> > eb --debug M4-1.4.17-foss-2018b.eb -D
> >
> >
> >  > == temporary log file in case of crash
> > /tmp/eb-ZCIA8v/easybuild-JS80fY.log
> >
> >  >
> >  > 
> >  >
> >  > ERROR: Failed to process easyconfig
> >  >
> >
>  
> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018b-OpenBLAS-0.3.1.eb:
> >
> >  > Failed to determine minimal toolchain for dep OpenBLAS
> > 0.3.1-GCC-7.3.0-2.30
> >  >
> >  > and a bunch of warnings (full log below).
> >  >
> >  > eb -S OpenBLAS-0.3.1-GCC-7.3.0-2.30
> >  >
> >  > gives
> >  >
> >  >
> >
>  
> CFGS1=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/o/OpenBLAS
> >  >   * $CFGS1/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb
> >  >
> >  > and
> >  >
> >  > eb OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb -D
> >  > == temporary log file in case of crash
> > /tmp/eb-VAv5nH/easybuild-9UafJT.log
> >  > Dry run: printing build status of easyconfigs and dependencies
> >  >
> >
>  
> CFGS=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs
> >  >   * [x] $CFGS/m/M4/M4-1.4.18.eb (module: M4/1.4.18)
> >  >   * [x] $CFGS/z/zlib/zlib-1.2.11.eb (module: zlib/1.2.11)
> >  >   * [x] $CFGS/h/help2man/help2man-1.47.4.eb (module:
> help2man/1.47.4)
> >  >   * [x] $CFGS/m/M4/M4-1.4.17.eb (module: M4/1.4.17)
> >  >   * [x] $CFGS/b/Bison/Bison-3.0.4.eb (module: Bison/3.0.4)
> >  >   * [x] $CFGS/f/flex/flex-2.6.4.eb (module: flex/2.6.4)
> >  >   * [x] $CFGS/b/binutils/binutils-2.30.eb (module: binutils/2.30)
> >  >   * [x] $CFGS/g/GCCcore/GCCcore-7.3.0.eb (module: GCCcore/7.3.0)
> >  >   * [x] $CFGS/z/zlib/zlib-1.2.11-GCCcore-7.3.0.eb (module:
> >  > zlib/1.2.11-GCCcore-7.3.0)
> >  >   * [x] $CFGS/h/help2man/help2man-1.47.4-GCCcore-7.3.0.eb (module:
> >  > help2man/1.47.4-GCCcore-7.3.0)
> >  >   * [x] $CFGS/m/M4/M4-1.4.18-GCCcore-7.3.0.eb (module:
> >  > M4/1.4.18-GCCcore-7.3.0)
> >  >   * [x] $CFGS/b/Bison/Bison-3.0.4-GCCcore-7.3.0.eb (module:
> >  > Bison/3.0.4-GCCcore-7.3.0)
> >  >   * [x] $CFGS/b/Bison/Bison-3.0.5-GCCcore-7.3.0.eb (module:
> >  > Bison/3.0.5-GCCcore-7.3.0)
> >  >   * [x] $CFGS/f/flex/flex-2.6.4-GCCcore-7.3.0.eb (module:
> >  > flex/2.6.4-GCCcore-7.3.0)
> >  >   * [x] $CFGS/b/binutils/binutils-2.30-GCCcore-7.3.0.eb (module:
> >  > binutils/2.30-GCCcore-7.3.0)
> >  >   * [x] $CFGS/g/GCC/GCC-7.3.0-2.30.eb (module: GCC/7.3.0-2.30)
> >  >   * [x] $CFGS/o/OpenBLAS/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb (module:
> >  > OpenBLAS/0.3.1-GCC-7.3.0-2.30)
> >  > == Temporary log file(s) /tmp/eb-VAv5nH/easybuild-9UafJT.log*
> > have been
> >  > removed.
> >  > == Temporary directory /tmp/eb-VAv5nH has been removed.
> >  >
> >  

Re: [easybuild] Can't get local install to work

[Kenneth Hoste]

Dear Niek,

On 19/11/2019 22:28, Niek de Klein wrote:

Dear Kenneth,

Attached the log.


Thank you, that's helpful to figure this out.

So, here's what's going on...

When resolving dependencies for M4-1.4.17-foss-2018b.eb, EasyBuild goes 
looking for the easyconfigs for the subtoolchains of foss/2018b, 
starting with gompi/2018b.


It finds an easyconfig for gompi/2018b at 
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/g/gompi/gompi-2018b.eb


This easyconfig file specifies that gompi/2018b consists of GCC 4.9.3, 
binutils 2.25 and OpenMPI 1.8.8, which is very incorrect...


This makes things go horribly wrong when it tries to find the easyconfig 
for OpenBLAS, eventually.



So, question is why that gompi-2018b.eb easyconfig file isn't what it's 
supposed to be (see 
https://github.com/easybuilders/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/g/gompi/gompi-2018b.eb).


The warnings you're hitting also show that you're using a copy of the 
easyconfig files from an EasyBuild 3.x version.


The easyconfigs included in EasyBuild 4.0.x have been cleaned up to 
avoid those warnings...




regards,

Kenneth


On Tue, Nov 19, 2019 at 4:00 PM Kenneth Hoste > wrote:


Dear Niek,

On 19/11/2019 20:42, Niek de Klein wrote:
 > I get
 >
 > $ eb M4-1.4.17-foss-2018b.eb -D


Please try running this with debugging enabled, and then zip up the log
file from /tmp (see path printed at the line like shown below):

eb --debug M4-1.4.17-foss-2018b.eb -D


 > == temporary log file in case of crash
/tmp/eb-ZCIA8v/easybuild-JS80fY.log

 >
 > 
 >
 > ERROR: Failed to process easyconfig
 >

/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018b-OpenBLAS-0.3.1.eb:

 > Failed to determine minimal toolchain for dep OpenBLAS
0.3.1-GCC-7.3.0-2.30
 >
 > and a bunch of warnings (full log below).
 >
 > eb -S OpenBLAS-0.3.1-GCC-7.3.0-2.30
 >
 > gives
 >
 >

CFGS1=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/o/OpenBLAS
 >   * $CFGS1/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb
 >
 > and
 >
 > eb OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb -D
 > == temporary log file in case of crash
/tmp/eb-VAv5nH/easybuild-9UafJT.log
 > Dry run: printing build status of easyconfigs and dependencies
 >

CFGS=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs
 >   * [x] $CFGS/m/M4/M4-1.4.18.eb (module: M4/1.4.18)
 >   * [x] $CFGS/z/zlib/zlib-1.2.11.eb (module: zlib/1.2.11)
 >   * [x] $CFGS/h/help2man/help2man-1.47.4.eb (module: help2man/1.47.4)
 >   * [x] $CFGS/m/M4/M4-1.4.17.eb (module: M4/1.4.17)
 >   * [x] $CFGS/b/Bison/Bison-3.0.4.eb (module: Bison/3.0.4)
 >   * [x] $CFGS/f/flex/flex-2.6.4.eb (module: flex/2.6.4)
 >   * [x] $CFGS/b/binutils/binutils-2.30.eb (module: binutils/2.30)
 >   * [x] $CFGS/g/GCCcore/GCCcore-7.3.0.eb (module: GCCcore/7.3.0)
 >   * [x] $CFGS/z/zlib/zlib-1.2.11-GCCcore-7.3.0.eb (module:
 > zlib/1.2.11-GCCcore-7.3.0)
 >   * [x] $CFGS/h/help2man/help2man-1.47.4-GCCcore-7.3.0.eb (module:
 > help2man/1.47.4-GCCcore-7.3.0)
 >   * [x] $CFGS/m/M4/M4-1.4.18-GCCcore-7.3.0.eb (module:
 > M4/1.4.18-GCCcore-7.3.0)
 >   * [x] $CFGS/b/Bison/Bison-3.0.4-GCCcore-7.3.0.eb (module:
 > Bison/3.0.4-GCCcore-7.3.0)
 >   * [x] $CFGS/b/Bison/Bison-3.0.5-GCCcore-7.3.0.eb (module:
 > Bison/3.0.5-GCCcore-7.3.0)
 >   * [x] $CFGS/f/flex/flex-2.6.4-GCCcore-7.3.0.eb (module:
 > flex/2.6.4-GCCcore-7.3.0)
 >   * [x] $CFGS/b/binutils/binutils-2.30-GCCcore-7.3.0.eb (module:
 > binutils/2.30-GCCcore-7.3.0)
 >   * [x] $CFGS/g/GCC/GCC-7.3.0-2.30.eb (module: GCC/7.3.0-2.30)
 >   * [x] $CFGS/o/OpenBLAS/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb (module:
 > OpenBLAS/0.3.1-GCC-7.3.0-2.30)
 > == Temporary log file(s) /tmp/eb-VAv5nH/easybuild-9UafJT.log*
have been
 > removed.
 > == Temporary directory /tmp/eb-VAv5nH has been removed.
 >
 > == temporary log file in case of crash
/tmp/eb-ZCIA8v/easybuild-JS80fY.log
 >
 > WARNING: Use of 3 unknown easyconfig parameters detected in
 > gompi-2018b.eb: binutilsver, gccver, tcver
 > If these are just local variables please rename them to start with
 > 'local_', or try using --fix-deprecated-easyconfigs to do this
 > automatically.
 > For more information, see
 >

https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html

 > .
 >
 >
 > WARNING: Deprecated functionality, will no longer work in v5.0:
Use of
 > 'dummy' toolchain is deprecated, use 'system' toolchain instead; see
 >
http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html

 > for more information
 >
 >
 > WARNING: Use of 3 

Re: [easybuild] Can't get local install to work

[Kenneth Hoste]

Dear Niek,

On 19/11/2019 20:42, Niek de Klein wrote:

I get

$ eb M4-1.4.17-foss-2018b.eb -D



Please try running this with debugging enabled, and then zip up the log 
file from /tmp (see path printed at the line like shown below):


eb --debug M4-1.4.17-foss-2018b.eb -D



== temporary log file in case of crash /tmp/eb-ZCIA8v/easybuild-JS80fY.log






ERROR: Failed to process easyconfig 
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018b-OpenBLAS-0.3.1.eb: 
Failed to determine minimal toolchain for dep OpenBLAS 0.3.1-GCC-7.3.0-2.30


and a bunch of warnings (full log below).

eb -S OpenBLAS-0.3.1-GCC-7.3.0-2.30

gives

CFGS1=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/o/OpenBLAS
  * $CFGS1/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb

and

eb OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb -D
== temporary log file in case of crash /tmp/eb-VAv5nH/easybuild-9UafJT.log
Dry run: printing build status of easyconfigs and dependencies
CFGS=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs
  * [x] $CFGS/m/M4/M4-1.4.18.eb (module: M4/1.4.18)
  * [x] $CFGS/z/zlib/zlib-1.2.11.eb (module: zlib/1.2.11)
  * [x] $CFGS/h/help2man/help2man-1.47.4.eb (module: help2man/1.47.4)
  * [x] $CFGS/m/M4/M4-1.4.17.eb (module: M4/1.4.17)
  * [x] $CFGS/b/Bison/Bison-3.0.4.eb (module: Bison/3.0.4)
  * [x] $CFGS/f/flex/flex-2.6.4.eb (module: flex/2.6.4)
  * [x] $CFGS/b/binutils/binutils-2.30.eb (module: binutils/2.30)
  * [x] $CFGS/g/GCCcore/GCCcore-7.3.0.eb (module: GCCcore/7.3.0)
  * [x] $CFGS/z/zlib/zlib-1.2.11-GCCcore-7.3.0.eb (module: 
zlib/1.2.11-GCCcore-7.3.0)
  * [x] $CFGS/h/help2man/help2man-1.47.4-GCCcore-7.3.0.eb (module: 
help2man/1.47.4-GCCcore-7.3.0)
  * [x] $CFGS/m/M4/M4-1.4.18-GCCcore-7.3.0.eb (module: 
M4/1.4.18-GCCcore-7.3.0)
  * [x] $CFGS/b/Bison/Bison-3.0.4-GCCcore-7.3.0.eb (module: 
Bison/3.0.4-GCCcore-7.3.0)
  * [x] $CFGS/b/Bison/Bison-3.0.5-GCCcore-7.3.0.eb (module: 
Bison/3.0.5-GCCcore-7.3.0)
  * [x] $CFGS/f/flex/flex-2.6.4-GCCcore-7.3.0.eb (module: 
flex/2.6.4-GCCcore-7.3.0)
  * [x] $CFGS/b/binutils/binutils-2.30-GCCcore-7.3.0.eb (module: 
binutils/2.30-GCCcore-7.3.0)

  * [x] $CFGS/g/GCC/GCC-7.3.0-2.30.eb (module: GCC/7.3.0-2.30)
  * [x] $CFGS/o/OpenBLAS/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb (module: 
OpenBLAS/0.3.1-GCC-7.3.0-2.30)
== Temporary log file(s) /tmp/eb-VAv5nH/easybuild-9UafJT.log* have been 
removed.

== Temporary directory /tmp/eb-VAv5nH has been removed.

== temporary log file in case of crash /tmp/eb-ZCIA8v/easybuild-JS80fY.log

WARNING: Use of 3 unknown easyconfig parameters detected in 
gompi-2018b.eb: binutilsver, gccver, tcver
If these are just local variables please rename them to start with 
'local_', or try using --fix-deprecated-easyconfigs to do this 
automatically.
For more information, see 
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html 
.



WARNING: Deprecated functionality, will no longer work in v5.0: Use of 
'dummy' toolchain is deprecated, use 'system' toolchain instead; see 
http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html 
for more information



WARNING: Use of 3 unknown easyconfig parameters detected in 
gompi-2018b.eb: binutilsver, gccver, tcver
If these are just local variables please rename them to start with 
'local_', or try using --fix-deprecated-easyconfigs to do this 
automatically.
For more information, see 
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html 
.



WARNING: Use of 2 unknown easyconfig parameters detected in 
GCC-4.9.3-binutils-2.25.eb: binutilsver, mpfr_version
If these are just local variables please rename them to start with 
'local_', or try using --fix-deprecated-easyconfigs to do this 
automatically.
For more information, see 
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html 
.



WARNING: Deprecated functionality, will no longer work in v5.0: Use of 
'dummy' toolchain is deprecated, use 'system' toolchain instead; see 
http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html 
for more information



WARNING: Use of 2 unknown easyconfig parameters detected in 
GCC-4.9.3-binutils-2.25.eb: binutilsver, mpfr_version
If these are just local variables please rename them to start with 
'local_', or try using --fix-deprecated-easyconfigs to do this 
automatically.
For more information, see 
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html 
.



WARNING: Use of 2 unknown easyconfig parameters detected in 
GNU-4.9.3-2.25.eb: binutilsver, gccver
If these are just local variables please rename them to start with 
'local_', or try using --fix-deprecated-easyconfigs to do this 
automatically.
For more information, see 
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html 
.



WARNING: Use of 3 unknown easyconfig parameters detected in 
OpenMPI-1.8.8-GNU-4.9.3-2.25.eb: 

Re: [easybuild] Can't get local install to work

[Niek de Klein]

I get

$ eb M4-1.4.17-foss-2018b.eb -D
== temporary log file in case of crash /tmp/eb-ZCIA8v/easybuild-JS80fY.log



ERROR: Failed to process easyconfig
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/s/ScaLAPACK/ScaLAPACK-2.0.2-gompi-2018b-OpenBLAS-0.3.1.eb:
Failed to determine minimal toolchain for dep OpenBLAS 0.3.1-GCC-7.3.0-2.30

and a bunch of warnings (full log below).

eb -S OpenBLAS-0.3.1-GCC-7.3.0-2.30

gives

CFGS1=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/o/OpenBLAS
 * $CFGS1/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb

and

eb OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb -D
== temporary log file in case of crash /tmp/eb-VAv5nH/easybuild-9UafJT.log
Dry run: printing build status of easyconfigs and dependencies
CFGS=/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs
 * [x] $CFGS/m/M4/M4-1.4.18.eb (module: M4/1.4.18)
 * [x] $CFGS/z/zlib/zlib-1.2.11.eb (module: zlib/1.2.11)
 * [x] $CFGS/h/help2man/help2man-1.47.4.eb (module: help2man/1.47.4)
 * [x] $CFGS/m/M4/M4-1.4.17.eb (module: M4/1.4.17)
 * [x] $CFGS/b/Bison/Bison-3.0.4.eb (module: Bison/3.0.4)
 * [x] $CFGS/f/flex/flex-2.6.4.eb (module: flex/2.6.4)
 * [x] $CFGS/b/binutils/binutils-2.30.eb (module: binutils/2.30)
 * [x] $CFGS/g/GCCcore/GCCcore-7.3.0.eb (module: GCCcore/7.3.0)
 * [x] $CFGS/z/zlib/zlib-1.2.11-GCCcore-7.3.0.eb (module:
zlib/1.2.11-GCCcore-7.3.0)
 * [x] $CFGS/h/help2man/help2man-1.47.4-GCCcore-7.3.0.eb (module:
help2man/1.47.4-GCCcore-7.3.0)
 * [x] $CFGS/m/M4/M4-1.4.18-GCCcore-7.3.0.eb (module:
M4/1.4.18-GCCcore-7.3.0)
 * [x] $CFGS/b/Bison/Bison-3.0.4-GCCcore-7.3.0.eb (module:
Bison/3.0.4-GCCcore-7.3.0)
 * [x] $CFGS/b/Bison/Bison-3.0.5-GCCcore-7.3.0.eb (module:
Bison/3.0.5-GCCcore-7.3.0)
 * [x] $CFGS/f/flex/flex-2.6.4-GCCcore-7.3.0.eb (module:
flex/2.6.4-GCCcore-7.3.0)
 * [x] $CFGS/b/binutils/binutils-2.30-GCCcore-7.3.0.eb (module:
binutils/2.30-GCCcore-7.3.0)
 * [x] $CFGS/g/GCC/GCC-7.3.0-2.30.eb (module: GCC/7.3.0-2.30)
 * [x] $CFGS/o/OpenBLAS/OpenBLAS-0.3.1-GCC-7.3.0-2.30.eb (module:
OpenBLAS/0.3.1-GCC-7.3.0-2.30)
== Temporary log file(s) /tmp/eb-VAv5nH/easybuild-9UafJT.log* have been
removed.
== Temporary directory /tmp/eb-VAv5nH has been removed.

== temporary log file in case of crash /tmp/eb-ZCIA8v/easybuild-JS80fY.log

WARNING: Use of 3 unknown easyconfig parameters detected in gompi-2018b.eb:
binutilsver, gccver, tcver
If these are just local variables please rename them to start with
'local_', or try using --fix-deprecated-easyconfigs to do this
automatically.
For more information, see
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
.


WARNING: Deprecated functionality, will no longer work in v5.0: Use of
'dummy' toolchain is deprecated, use 'system' toolchain instead; see
http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html
for more information


WARNING: Use of 3 unknown easyconfig parameters detected in gompi-2018b.eb:
binutilsver, gccver, tcver
If these are just local variables please rename them to start with
'local_', or try using --fix-deprecated-easyconfigs to do this
automatically.
For more information, see
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
.


WARNING: Use of 2 unknown easyconfig parameters detected in
GCC-4.9.3-binutils-2.25.eb: binutilsver, mpfr_version
If these are just local variables please rename them to start with
'local_', or try using --fix-deprecated-easyconfigs to do this
automatically.
For more information, see
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
.


WARNING: Deprecated functionality, will no longer work in v5.0: Use of
'dummy' toolchain is deprecated, use 'system' toolchain instead; see
http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html
for more information


WARNING: Use of 2 unknown easyconfig parameters detected in
GCC-4.9.3-binutils-2.25.eb: binutilsver, mpfr_version
If these are just local variables please rename them to start with
'local_', or try using --fix-deprecated-easyconfigs to do this
automatically.
For more information, see
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
.


WARNING: Use of 2 unknown easyconfig parameters detected in
GNU-4.9.3-2.25.eb: binutilsver, gccver
If these are just local variables please rename them to start with
'local_', or try using --fix-deprecated-easyconfigs to do this
automatically.
For more information, see
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
.


WARNING: Use of 3 unknown easyconfig parameters detected in
OpenMPI-1.8.8-GNU-4.9.3-2.25.eb: binfile, libfile, libs
If these are just local variables please rename them to start with
'local_', or try using --fix-deprecated-easyconfigs to do this
automatically.
For more information, see
https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
.


WARNING: Use of 2 unknown easyconfig 

Re: [easybuild] Can't get local install to work

[Kenneth Hoste]

On 18/11/2019 21:50, Niek de Klein wrote:

Hi Kenneth,

I spoke too soon, some of my installations work but not all. When I try 
to isntall Autoconf-2.69-foss-2018.eb I get


ERROR: Failed to process easyconfig 
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69-foss-2018b.eb: 
Failed to determine minimal toolchain for dep M4 1.4.17


with eb --search I get:

eb --search M4-1.4.17-foss-2018b
  * 
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/m/M4/M4-1.4.17-foss-2018b.eb


eb --search ^foss-2018b.eb
  * 
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/f/foss/foss-2018b.eb


So it can find both the M4-1.4.17-foss-2018b.eb file and the 
foss-2018.eb file.


Can you try "eb M4-1.4.17-foss-2018b.eb -D"?




Kind regards,
Niek





On Mon, Nov 18, 2019 at 3:21 PM Niek de Klein > wrote:


Hi Kenneth,

Thanks for your help!

1. is solved now with your suggestion of
EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1

Now that the bootstrap.py ran correctly it can also find the
easybuild files in the location that I have configured, so this
solves 1 and 2 as well.

Thanks,
Niek

On Mon, Nov 18, 2019 at 3:01 PM Kenneth Hoste
mailto:kenneth.ho...@ugent.be>> wrote:

Dear Niek,

Sorry that you're hitting these issues, let's try and get them
sorted...

On 18/11/2019 20:00, Niek de Klein wrote:
 > Hi all,
 >
 > I am trying to have my own installation of EasyBuild, but I
have 3 problems:
 >
 > 1. When I try to run bootstrap.py it gives an error that
md5sums are
 > missing, but I'm not sure where to add them as I thought that
should
 > then have been included in the EasyBuild-4.0.1.eb file:
 >
 >      easybuild.tools.build_log.EasyBuildError: "Build of
 > /tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed
(first 300
 > chars): Missing checksum for easybuild-framework-4.0.1.tar.gz')"

Sorry for not responding to the GitHub issue [1] you opened on this
(will do so now with the answer below).

We still need to clean up the bootstrap script a bit after the
EasyBuild
v4.0 release, it's still doing a bunch of stuff it shouldn't
anymore,
and it's not compatible yet with Python 3 (see also
https://github.com/easybuilders/easybuild-framework/pull/3028).


Your active EasyBuild configuration has --enforce-checksums
enabled (via
$EASYBUILD_ENFORCE_CHECKSUMS), which explains the problem you're
seeing.

The bootstrap script auto-generates an easyconfig file, but it
doesn't
include checksums (it could I guess, with a bit more work).

You can dance around this though, by temporarily disabling the
checksums
requirement, only for the bootstrap:

EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python bootstrap 


If you want to, you can bootstrap EasyBuild to a temporary
location (for
example /tmp/$USER), and then use that installation to pull in an
easyconfig file for EasyBuild v4.0.1 that does have checksums,
using "eb
--from-pr 9149 --installpath " (which pulls in the
easyconfig from
https://github.com/easybuilders/easybuild-easyconfigs/pull/9149/files)

for the actual installation you intend to use.

 >
 >
 > 2. I changed the robo and installation path to be different,
but I keep
 > getting "Failed to determine minimal toolchain for dep ..."
errors. The
 > eb --show-config output is at the end of this e-mail, and
shows that my
 > robot-paths is
 >

/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/,

 > which is where I cloned the easyconfigs repository.

What happens if you use "eb --search" with this setup, can 'eb'
find
easyconfig files?

Try something like "eb --search tensorflow-1.14" for example.


 >
 > 3. Even if the robo path wasn't working, tools that are
already in my
 > module avail are not being picked up. For example, EasyBuild
that is not
 > working uses "toolchain = {'name': 'foss', 'version':
'2018b'}" and has
 > as one of the dependencies "('ncurses', '6.0'),". When I try
to install
 > it I get
 >
 > ERROR: Failed to process easyconfig
 >

/groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb:

 > Failed to determine minimal toolchain for dep ncurses 6.0
 >
 > but when I do ml avail ncur I get:
 >
 > 

Re: [easybuild] Can't get local install to work

[Niek de Klein]

Hi Kenneth,

I spoke too soon, some of my installations work but not all. When I try to
isntall Autoconf-2.69-foss-2018.eb I get

ERROR: Failed to process easyconfig
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69-foss-2018b.eb:
Failed to determine minimal toolchain for dep M4 1.4.17

with eb --search I get:

eb --search M4-1.4.17-foss-2018b
 *
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/m/M4/M4-1.4.17-foss-2018b.eb

eb --search ^foss-2018b.eb
 *
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/f/foss/foss-2018b.eb

So it can find both the M4-1.4.17-foss-2018b.eb file and the foss-2018.eb
file.

Kind regards,
Niek





On Mon, Nov 18, 2019 at 3:21 PM Niek de Klein  wrote:

> Hi Kenneth,
>
> Thanks for your help!
>
> 1. is solved now with your suggestion of
> EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1
>
> Now that the bootstrap.py ran correctly it can also find the easybuild
> files in the location that I have configured, so this solves 1 and 2 as
> well.
>
> Thanks,
> Niek
>
> On Mon, Nov 18, 2019 at 3:01 PM Kenneth Hoste 
> wrote:
>
>> Dear Niek,
>>
>> Sorry that you're hitting these issues, let's try and get them sorted...
>>
>> On 18/11/2019 20:00, Niek de Klein wrote:
>> > Hi all,
>> >
>> > I am trying to have my own installation of EasyBuild, but I have 3
>> problems:
>> >
>> > 1. When I try to run bootstrap.py it gives an error that md5sums are
>> > missing, but I'm not sure where to add them as I thought that should
>> > then have been included in the EasyBuild-4.0.1.eb file:
>> >
>> >  easybuild.tools.build_log.EasyBuildError: "Build of
>> > /tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed (first 300
>> > chars): Missing checksum for easybuild-framework-4.0.1.tar.gz')"
>>
>> Sorry for not responding to the GitHub issue [1] you opened on this
>> (will do so now with the answer below).
>>
>> We still need to clean up the bootstrap script a bit after the EasyBuild
>> v4.0 release, it's still doing a bunch of stuff it shouldn't anymore,
>> and it's not compatible yet with Python 3 (see also
>> https://github.com/easybuilders/easybuild-framework/pull/3028).
>>
>>
>> Your active EasyBuild configuration has --enforce-checksums enabled (via
>> $EASYBUILD_ENFORCE_CHECKSUMS), which explains the problem you're seeing.
>>
>> The bootstrap script auto-generates an easyconfig file, but it doesn't
>> include checksums (it could I guess, with a bit more work).
>>
>> You can dance around this though, by temporarily disabling the checksums
>> requirement, only for the bootstrap:
>>
>> EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python bootstrap 
>>
>>
>> If you want to, you can bootstrap EasyBuild to a temporary location (for
>> example /tmp/$USER), and then use that installation to pull in an
>> easyconfig file for EasyBuild v4.0.1 that does have checksums, using "eb
>> --from-pr 9149 --installpath " (which pulls in the
>> easyconfig from
>> https://github.com/easybuilders/easybuild-easyconfigs/pull/9149/files)
>> for the actual installation you intend to use.
>>
>> >
>> >
>> > 2. I changed the robo and installation path to be different, but I keep
>> > getting "Failed to determine minimal toolchain for dep ..." errors. The
>> > eb --show-config output is at the end of this e-mail, and shows that my
>> > robot-paths is
>> >
>> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/,
>>
>> > which is where I cloned the easyconfigs repository.
>>
>> What happens if you use "eb --search" with this setup, can 'eb' find
>> easyconfig files?
>>
>> Try something like "eb --search tensorflow-1.14" for example.
>>
>>
>> >
>> > 3. Even if the robo path wasn't working, tools that are already in my
>> > module avail are not being picked up. For example, EasyBuild that is
>> not
>> > working uses "toolchain = {'name': 'foss', 'version': '2018b'}" and has
>> > as one of the dependencies "('ncurses', '6.0'),". When I try to install
>> > it I get
>> >
>> > ERROR: Failed to process easyconfig
>> >
>> /groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb:
>>
>> > Failed to determine minimal toolchain for dep ncurses 6.0
>> >
>> > but when I do ml avail ncur I get:
>> >
>> > --
>> > /groups/umcg-biogen/tmp01/apps/modules/devel
>> > ---
>> > ncurses/6.0-foss-2018b
>> >
>> > ---
>> > /apps/modules/devel
>> > ---
>> > ncurses/6.0ncurses/6.1-foss-2018bncurses/6.1-GCCcore-7.3.0
>> (D)
>> >
>> > So the ncurses module is already available
>> >
>> >
>> > Am I missing something in my configuration?
>>
>> This is a consequence of having --minimal-toolchains enabled, which
>> requires that EasyBuild is able to find the easyconfig 

Re: [easybuild] Can't get local install to work

[Niek de Klein]

Hi Kenneth,

Thanks for your help!

1. is solved now with your suggestion of
EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1

Now that the bootstrap.py ran correctly it can also find the easybuild
files in the location that I have configured, so this solves 1 and 2 as
well.

Thanks,
Niek

On Mon, Nov 18, 2019 at 3:01 PM Kenneth Hoste 
wrote:

> Dear Niek,
>
> Sorry that you're hitting these issues, let's try and get them sorted...
>
> On 18/11/2019 20:00, Niek de Klein wrote:
> > Hi all,
> >
> > I am trying to have my own installation of EasyBuild, but I have 3
> problems:
> >
> > 1. When I try to run bootstrap.py it gives an error that md5sums are
> > missing, but I'm not sure where to add them as I thought that should
> > then have been included in the EasyBuild-4.0.1.eb file:
> >
> >  easybuild.tools.build_log.EasyBuildError: "Build of
> > /tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed (first 300
> > chars): Missing checksum for easybuild-framework-4.0.1.tar.gz')"
>
> Sorry for not responding to the GitHub issue [1] you opened on this
> (will do so now with the answer below).
>
> We still need to clean up the bootstrap script a bit after the EasyBuild
> v4.0 release, it's still doing a bunch of stuff it shouldn't anymore,
> and it's not compatible yet with Python 3 (see also
> https://github.com/easybuilders/easybuild-framework/pull/3028).
>
>
> Your active EasyBuild configuration has --enforce-checksums enabled (via
> $EASYBUILD_ENFORCE_CHECKSUMS), which explains the problem you're seeing.
>
> The bootstrap script auto-generates an easyconfig file, but it doesn't
> include checksums (it could I guess, with a bit more work).
>
> You can dance around this though, by temporarily disabling the checksums
> requirement, only for the bootstrap:
>
> EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python bootstrap 
>
>
> If you want to, you can bootstrap EasyBuild to a temporary location (for
> example /tmp/$USER), and then use that installation to pull in an
> easyconfig file for EasyBuild v4.0.1 that does have checksums, using "eb
> --from-pr 9149 --installpath " (which pulls in the
> easyconfig from
> https://github.com/easybuilders/easybuild-easyconfigs/pull/9149/files)
> for the actual installation you intend to use.
>
> >
> >
> > 2. I changed the robo and installation path to be different, but I keep
> > getting "Failed to determine minimal toolchain for dep ..." errors. The
> > eb --show-config output is at the end of this e-mail, and shows that my
> > robot-paths is
> >
> /groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/,
>
> > which is where I cloned the easyconfigs repository.
>
> What happens if you use "eb --search" with this setup, can 'eb' find
> easyconfig files?
>
> Try something like "eb --search tensorflow-1.14" for example.
>
>
> >
> > 3. Even if the robo path wasn't working, tools that are already in my
> > module avail are not being picked up. For example, EasyBuild that is not
> > working uses "toolchain = {'name': 'foss', 'version': '2018b'}" and has
> > as one of the dependencies "('ncurses', '6.0'),". When I try to install
> > it I get
> >
> > ERROR: Failed to process easyconfig
> >
> /groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb:
>
> > Failed to determine minimal toolchain for dep ncurses 6.0
> >
> > but when I do ml avail ncur I get:
> >
> > --
> > /groups/umcg-biogen/tmp01/apps/modules/devel
> > ---
> > ncurses/6.0-foss-2018b
> >
> > ---
> > /apps/modules/devel
> > ---
> > ncurses/6.0ncurses/6.1-foss-2018bncurses/6.1-GCCcore-7.3.0
> (D)
> >
> > So the ncurses module is already available
> >
> >
> > Am I missing something in my configuration?
>
> This is a consequence of having --minimal-toolchains enabled, which
> requires that EasyBuild is able to find the easyconfig file for the
> toolchain (foss-2018b.eb in this case).
>
> It needs to figure out what the (versions of) the subtoolchains are for
> foss/2018b, so it can check for ncurses installed with GCCcore first,
> before gompi, before foss.
>
> So this is basically the same issue as with problem 2, I bet "eb
> --search ^foss-2018b.eb" doesn't give you any result?
>
>
> Are you 100% sure the path you specified for --robot-paths is correct?
>
> Note that you can also prepend or append to the robot search path
> (rather than override the default), see
>
> https://easybuild.readthedocs.io/en/latest/Using_the_EasyBuild_command_line.html#controlling-robot-search-path
> .
>
> I hope this helps...
>
>
> regards,
>
> Kenneth
>
>
> [1] https://github.com/easybuilders/easybuild-framework/issues/3079
>
> >
> > Thanks for your help!
> > Niek de Klein
> >
> >
> > #
> > # Current EasyBuild configuration
> > # (C: command line 

Re: [easybuild] Can't get local install to work

[Kenneth Hoste]

Dear Niek,

Sorry that you're hitting these issues, let's try and get them sorted...

On 18/11/2019 20:00, Niek de Klein wrote:

Hi all,

I am trying to have my own installation of EasyBuild, but I have 3 problems:

1. When I try to run bootstrap.py it gives an error that md5sums are 
missing, but I'm not sure where to add them as I thought that should 
then have been included in the EasyBuild-4.0.1.eb file:


     easybuild.tools.build_log.EasyBuildError: "Build of 
/tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed (first 300 
chars): Missing checksum for easybuild-framework-4.0.1.tar.gz')"


Sorry for not responding to the GitHub issue [1] you opened on this 
(will do so now with the answer below).


We still need to clean up the bootstrap script a bit after the EasyBuild 
v4.0 release, it's still doing a bunch of stuff it shouldn't anymore, 
and it's not compatible yet with Python 3 (see also 
https://github.com/easybuilders/easybuild-framework/pull/3028).



Your active EasyBuild configuration has --enforce-checksums enabled (via 
$EASYBUILD_ENFORCE_CHECKSUMS), which explains the problem you're seeing.


The bootstrap script auto-generates an easyconfig file, but it doesn't 
include checksums (it could I guess, with a bit more work).


You can dance around this though, by temporarily disabling the checksums 
requirement, only for the bootstrap:


EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python bootstrap 


If you want to, you can bootstrap EasyBuild to a temporary location (for 
example /tmp/$USER), and then use that installation to pull in an 
easyconfig file for EasyBuild v4.0.1 that does have checksums, using "eb 
--from-pr 9149 --installpath " (which pulls in the 
easyconfig from 
https://github.com/easybuilders/easybuild-easyconfigs/pull/9149/files) 
for the actual installation you intend to use.





2. I changed the robo and installation path to be different, but I keep 
getting "Failed to determine minimal toolchain for dep ..." errors. The 
eb --show-config output is at the end of this e-mail, and shows that my 
robot-paths is 
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/, 
which is where I cloned the easyconfigs repository.


What happens if you use "eb --search" with this setup, can 'eb' find 
easyconfig files?


Try something like "eb --search tensorflow-1.14" for example.




3. Even if the robo path wasn't working, tools that are already in my 
module avail are not being picked up. For example, EasyBuild that is not 
working uses "toolchain = {'name': 'foss', 'version': '2018b'}" and has 
as one of the dependencies "('ncurses', '6.0'),". When I try to install 
it I get


ERROR: Failed to process easyconfig 
/groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb: 
Failed to determine minimal toolchain for dep ncurses 6.0


but when I do ml avail ncur I get:

-- 
/groups/umcg-biogen/tmp01/apps/modules/devel 
---

    ncurses/6.0-foss-2018b

--- 
/apps/modules/devel 
---

    ncurses/6.0    ncurses/6.1-foss-2018b    ncurses/6.1-GCCcore-7.3.0 (D)

So the ncurses module is already available


Am I missing something in my configuration?


This is a consequence of having --minimal-toolchains enabled, which 
requires that EasyBuild is able to find the easyconfig file for the 
toolchain (foss-2018b.eb in this case).


It needs to figure out what the (versions of) the subtoolchains are for 
foss/2018b, so it can check for ncurses installed with GCCcore first, 
before gompi, before foss.


So this is basically the same issue as with problem 2, I bet "eb 
--search ^foss-2018b.eb" doesn't give you any result?



Are you 100% sure the path you specified for --robot-paths is correct?

Note that you can also prepend or append to the robot search path 
(rather than override the default), see 
https://easybuild.readthedocs.io/en/latest/Using_the_EasyBuild_command_line.html#controlling-robot-search-path 
.


I hope this helps...


regards,

Kenneth


[1] https://github.com/easybuilders/easybuild-framework/issues/3079



Thanks for your help!
Niek de Klein


#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, 
F: configuration file)

#
buildpath                   (E) = 
/groups/umcg-biogen/tmp01/apps//.tmp/easybuild/builds/
containerpath               (D) = 
/home/umcg-ndeklein/.local/easybuild/containers

enforce-checksums           (E) = True
installpath                 (E) = /groups/umcg-biogen/tmp01/apps/
minimal-toolchains          (E) = True
repositorypath              (D) = 
/home/umcg-ndeklein/.local/easybuild/ebfiles_repo
robot-paths                 (E) = 
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/