hi Kenneth,
> (and this is not EasyBuild's fault btw)
+1 indeed!
> Why are you taking this road exactly, what's the added benefit
compared to installing directly into /sw
It's fair to mention non-technical reasons too. I somehow like the idea
of building software in containers since I was sysop on HPC system with
Lustre FS. There the containerization was the easiest way to build
kernel modules with/without OFED and with any kernel for any OS. (we had
mix of centos, rhel, v6+v7). Since then I consider building in
containers as convenient and safe approach and I was not thinking too
much how to do it with Easybuild..
technical reasons:
* reproducibility - On some systems there are devel RPMs just on login
nodes, and on computes there are just the binary packages. Some software
(X, VNC..) can be completely missing. It can lead to surprising results.
(especially with not-so-well-tested, site-specific easyconfigs)
Containerization also completely removes the possible human-factor when
someone somewhere installs some nice RPM from $reasons.. Container has
own recipe which is in git, so tracking of the build env is trivial.
* independent build and deployment actions - yes, I could take one node
and keep it drained in slurm till I'll check the build is fine.. One
node is not too much, but let's do the things correctly.. Containerized
build also mean that I can submit a job with the build task, and next
day/week/month I have software tree ready somewhere in my $scratch and
there is no need to take care about draining and later undraining the
nodes.. (again, human factor elimination).
btw. could you, please, share your workflow too? I'm quite curious,
especially as I understood you do it in more simple way.
cheers
Josef
Josef Dvoracek
fzu.cz/~jose
On 17. 01. 19 16:37, Kenneth Hoste wrote:
Dear Josef,
On 17/01/2019 11:40, Josef Dvoracek wrote:
hi,
I am actually also looking forward for answers from more experienced
users in this thread as the procedures around EB are something what I
could definitely improve at my system.
My findings after ~half year of using Easybuild (EB):
* moving EB directories is bad as it always breaks something
Indeed, don't assume you can do this (and this is not EasyBuild's
fault btw).
* you usually want the possibility of rebuilding/updating software
tree and switching to the new one in some atomic operation, having
possibility to rollback, especially if you're building own
easyconfigs where you can expect new and fancy bugs
* Update and rollback of software tree should be easy and fast
operation.
How I do it:
As I want software build to be in /sw (EASYBUILD_PREFIX=/sw) I build
in Singularity container with bind option "-B /scratch:/sw" - so the
directory "/sw" inside container is mapped to "/scratch" outside. As
I have also enough of RAM, I also do "-B /dev/shm:/builddir" - builds
in tmpfs are IMO faster than on global cluster filesystem..
That means when build in container is done, and I like it, I have
everything ready in /scratch and I can (move|rsync) it into /sw - for
instance as a root job on idle nodes or something similar.. (yes,
then you need to ensure your changes are just incremental).
This is an interesting approach, but it seems overkill to me...
Why are you taking this road exactly, what's the added benefit
compared to installing directly into /sw ?
regards,
Kenneth
cheers
josef
On 17. 01. 19 11:07, Loris Bennett wrote:
Hi,
I set up EasyBuild in my own home directory and managed to build a
bunch
of software. I then copied the whole EasyBuild tree to a different
directory owned by a system user 'build'.
Unfortunately the path to my home directory has been written to various
scripts and, I presume, may have been compiled into some binaries.
Thus
I just want to rebuild the software.
However doing say
eb GCCcore-7.3.0.eb --robot --rebuild
fails with the following message to stdout:
make: *** [all] Error 2
(at
easybuild/software/EasyBuild/3.8.0/lib/python2.7/site-packages/easybuild_framework-3.8.0-py2.7.egg/easybuild/tools/run.py:501
in parse_cmd_output)
== 2019-01-17 10:22:21,683 easyblock.py:2864 WARNING build failed
(first 300 chars): cmd " make -j 40 " exited with exit code 2 and
output:
make[1]: Entering directory
`/trinity/shared/easybuild/build/GCCcore/7.3.0/dummy-/gcc-7.3.0/stage1_obj'
and the following in the log file:
== 2019-01-17 10:22:21,683 easyblock.py:2864 WARNING build failed
(first 300 chars): cmd " make -j 40 " exited with exit code 2 and
output:
make[1]: Entering directory
`/trinity/shared/easybuild/build/GCCcore/7.3.0/dummy-/gcc-7.3.0/stage1_obj'
mkdir -p -- ./fixincludes
mkdir -p -- ./libiberty
mkdir -p -- ./lto-plugin
mkdir -p -- ./intl
mkdir -p -- ./gmp
mkdir -p -- ./libbacktrace
Co
== 2019-01-17 10:22:21,684 easyblock.py:286 INFO Closing log for
application name GCCcore version 7.3.0
As I am slightly pushed for time and probably have to recompile most
stuff anyway, I was wondering what is the most pragmatic way to
proceed.
Of the directories
apps/
build/
ebfiles_repo/
moduleData/
modules/
software/
sources/
could I just delete
build/
modules/
software/
?
Cheers,
Loris
