Hi Marcelo,
On 25/11/2019 19:44, Pablo Escobar Lopez wrote:
On Mon, Nov 25, 2019 at 7:08 PM Marcelo Carignano <[email protected]
<mailto:[email protected]>> wrote:
Hello there,
today I installed EasyBuild on a Cray XC50, installation went ok.
If you're using EasyBuild on a Cray system, you may be interested in how
CSCS uses EasyBuild on their Cray systems (incl. Piz Daint).
They leverage the support for using external modules (see
https://easybuild.readthedocs.io/en/latest/Using_external_modules.html)
to leverage the modules provided in the Cray Programming Environment.
See for example
https://github.com/eth-cscs/production/blob/master/easybuild/cray_external_modules_metadata.cfg
.
See https://easybuild.readthedocs.io/en/latest/Cray-support.html for
more information (probably a bit dated w.r.t. versions and what's
supported).
Then I tried to install CP2K by doing:
> eb CP2K-6.1-intel-2018a.eb --modules-tool EnvironmentModulesC
--module-syntax Tcl --robot
and eb started fetching and installing all the pieces.
I understand this is a complex process and I am not trying to
understand everything in a couple of days, but there are things I
cannot figure out and it will make my day to just have an idea of
what is going on.
First thing is related to --modules-tool EnvironmentModulesC
--module-syntax Tcl
I couldn't figure out how to define the corresponding env variables
to avoid the need of using --modules-tool and --module-syntax. Is
there a way to define these var in .bashrc?
My .bashrc has the following lines:
you can configure easybuild using cli arguments, env vars or a config file
https://easybuild.readthedocs.io/en/latest/Configuration.html#supported-configuration-types
a possible approach is to create an example config file and adapt it to
your needs
https://easybuild.readthedocs.io/en/latest/Configuration.html#generating-a-template-configuration-file
no matter if you use cli arguments, env var or config file it's always
good practice to review what config is being applied
https://easybuild.readthedocs.io/en/latest/Configuration.html#overview-of-current-configuration-show-config-show-full-config
To add to this: for *every* configuration option (see 'eb --help') there
is a corresponding environment variable that EasyBuild picks up on.
In short, --foo-bar translates to $EASYBUILD_FOO_BAR .
So, you can use
export EASYBUILD_MODULES_TOOL=EnvironmentModulesC
export EASYBUILD_MODULE_SYNTAX=Tcl
EASYBUILD_PREFIX=$HOME/.local/easybuild
MODULEPATH=$MODULEPATH:$EASYBUILD_PREFIX/modules/all
The second point is about icc/2018.1.163-GCC-6.4.0-2.28, which is in
the dependency list of the CP2K eb file I started with. How is it
that eb can download the sources from intel? Are they really open? I
have the feeling that the process has stopped since it has been
trying to fetch the files for a long time.. Please let me know if I
am missing something here. When I try to access the intel address
from the icc.eb file, I get the request for an email and serial
number... which I can't believe eb knows to be able to pass it on...
Again, I am a bit confused. Any help is welcome.
You are right, easybuild cannot download the intel tarballs. If you want
to install intel you should download the required files yourself (check
the easyconfigs or the failed build logs to find out the proper tarball
names that easybuild expects) and you have to manually copy those
tarballs to the folder where easybuild will look for them.
https://easybuild.readthedocs.io/en/latest/Configuration.html#sourcepath
Actually, that's not true...
For recent versions of the Intel tools, EasyBuild does know how to
download the files, but it'll take a looong time (because the files are
huge, and the Intel servers are slow).
The download URLs for the Intel tools are public, so anyone can download
the files (so EasyBuild can too).
If EasyBuild auto-downloads them (using source URL + filename), it does
it in memory, so you won't see anything happening on disk until the
download is complete...
So, eb hasn't stalled on you, it just takes forever to download.
If you use "eb -x icc.eb", you'll see the exact download location
EasyBuild uses.
As Pablo suggested, you can also download the necessary files manually
(through a browser, or using wget/curl), and drop them in the right
location in the EasyBuild source path.
If EasyBuild finds the required files there, and the SHA256 checksum
matches, it'll be happy.
If you are starting to learn how easybuild works I would recommend you
to start using a foss toolchain instead of intel toolchain. foss
toolchains are easier for the first try because all the tarballs can be
automatically donwloaded. Once you get the feeling on how easybuild
works with foss toolchains you can move forward to intel toolchains.
e.g. I would start by trying "eb CP2K-6.1-foss-2019a.eb --modules-tool
EnvironmentModulesC --module-syntax Tcl --robot"
regards,
Pablo.
Thank you very much,
Marcelo Carignano.
--
Pablo Escobar López
Linux/HPC systems engineer
sciCORE, University of Basel
SIB Swiss Institute of Bioinformatics
