On 18/11/2019 21:50, Niek de Klein wrote:
Hi Kenneth,
I spoke too soon, some of my installations work but not all. When I try
to isntall Autoconf-2.69-foss-2018.eb I get
ERROR: Failed to process easyconfig
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/a/Autoconf/Autoconf-2.69-foss-2018b.eb:
Failed to determine minimal toolchain for dep M4 1.4.17
with eb --search I get:
eb --search M4-1.4.17-foss-2018b
*
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/m/M4/M4-1.4.17-foss-2018b.eb
eb --search ^foss-2018b.eb
*
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/f/foss/foss-2018b.eb
So it can find both the M4-1.4.17-foss-2018b.eb file and the
foss-2018.eb file.
Can you try "eb M4-1.4.17-foss-2018b.eb -D"?
Kind regards,
Niek
On Mon, Nov 18, 2019 at 3:21 PM Niek de Klein <niekdekl...@gmail.com
<mailto:niekdekl...@gmail.com>> wrote:
Hi Kenneth,
Thanks for your help!
1. is solved now with your suggestion of
EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1
Now that the bootstrap.py ran correctly it can also find the
easybuild files in the location that I have configured, so this
solves 1 and 2 as well.
Thanks,
Niek
On Mon, Nov 18, 2019 at 3:01 PM Kenneth Hoste
<kenneth.ho...@ugent.be <mailto:kenneth.ho...@ugent.be>> wrote:
Dear Niek,
Sorry that you're hitting these issues, let's try and get them
sorted...
On 18/11/2019 20:00, Niek de Klein wrote:
> Hi all,
>
> I am trying to have my own installation of EasyBuild, but I
have 3 problems:
>
> 1. When I try to run bootstrap.py it gives an error that
md5sums are
> missing, but I'm not sure where to add them as I thought that
should
> then have been included in the EasyBuild-4.0.1.eb file:
>
> easybuild.tools.build_log.EasyBuildError: "Build of
> /tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed
(first 300
> chars): Missing checksum for easybuild-framework-4.0.1.tar.gz')"
Sorry for not responding to the GitHub issue [1] you opened on this
(will do so now with the answer below).
We still need to clean up the bootstrap script a bit after the
EasyBuild
v4.0 release, it's still doing a bunch of stuff it shouldn't
anymore,
and it's not compatible yet with Python 3 (see also
https://github.com/easybuilders/easybuild-framework/pull/3028).
Your active EasyBuild configuration has --enforce-checksums
enabled (via
$EASYBUILD_ENFORCE_CHECKSUMS), which explains the problem you're
seeing.
The bootstrap script auto-generates an easyconfig file, but it
doesn't
include checksums (it could I guess, with a bit more work).
You can dance around this though, by temporarily disabling the
checksums
requirement, only for the bootstrap:
EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python bootstrap <path>
If you want to, you can bootstrap EasyBuild to a temporary
location (for
example /tmp/$USER), and then use that installation to pull in an
easyconfig file for EasyBuild v4.0.1 that does have checksums,
using "eb
--from-pr 9149 --installpath <final_path>" (which pulls in the
easyconfig from
https://github.com/easybuilders/easybuild-easyconfigs/pull/9149/files)
for the actual installation you intend to use.
>
>
> 2. I changed the robo and installation path to be different,
but I keep
> getting "Failed to determine minimal toolchain for dep ..."
errors. The
> eb --show-config output is at the end of this e-mail, and
shows that my
> robot-paths is
>
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/,
> which is where I cloned the easyconfigs repository.
What happens if you use "eb --search" with this setup, can 'eb'
find
easyconfig files?
Try something like "eb --search tensorflow-1.14" for example.
>
> 3. Even if the robo path wasn't working, tools that are
already in my
> module avail are not being picked up. For example, EasyBuild
that is not
> working uses "toolchain = {'name': 'foss', 'version':
'2018b'}" and has
> as one of the dependencies "('ncurses', '6.0'),". When I try
to install
> it I get
>
> ERROR: Failed to process easyconfig
>
/groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb:
> Failed to determine minimal toolchain for dep ncurses 6.0
>
> but when I do ml avail ncur I get:
>
> ------------------------------------------------------
> /groups/umcg-biogen/tmp01/apps/modules/devel
> -------------------------------------------------------
> ncurses/6.0-foss-2018b
>
>
-------------------------------------------------------------------
> /apps/modules/devel
>
-------------------------------------------------------------------
> ncurses/6.0 ncurses/6.1-foss-2018b
ncurses/6.1-GCCcore-7.3.0 (D)
>
> So the ncurses module is already available
>
>
> Am I missing something in my configuration?
This is a consequence of having --minimal-toolchains enabled, which
requires that EasyBuild is able to find the easyconfig file for the
toolchain (foss-2018b.eb in this case).
It needs to figure out what the (versions of) the subtoolchains
are for
foss/2018b, so it can check for ncurses installed with GCCcore
first,
before gompi, before foss.
So this is basically the same issue as with problem 2, I bet "eb
--search ^foss-2018b.eb" doesn't give you any result?
Are you 100% sure the path you specified for --robot-paths is
correct?
Note that you can also prepend or append to the robot search path
(rather than override the default), see
https://easybuild.readthedocs.io/en/latest/Using_the_EasyBuild_command_line.html#controlling-robot-search-path
.
I hope this helps...
regards,
Kenneth
[1] https://github.com/easybuilders/easybuild-framework/issues/3079
>
> Thanks for your help!
> Niek de Klein
>
>
> #
> # Current EasyBuild configuration
> # (C: command line argument, D: default value, E: environment
variable,
> F: configuration file)
> #
> buildpath (E) =
> /groups/umcg-biogen/tmp01/apps//.tmp/easybuild/builds/
> containerpath (D) =
> /home/umcg-ndeklein/.local/easybuild/containers
> enforce-checksums (E) = True
> installpath (E) = /groups/umcg-biogen/tmp01/apps/
> minimal-toolchains (E) = True
> repositorypath (D) =
> /home/umcg-ndeklein/.local/easybuild/ebfiles_repo
> robot-paths (E) =
>
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/
> set-gid-bit (E) = True
> sourcepath (E) =
/groups/umcg-biogen/tmp01/apps//sources/
> umask (E) = 002
> verify-easyconfig-filenames (E) = True
>
>
