Hi Malcolm,
First of all: let me apologize for not getting back to you in some of
the issues you've replied to on the EasyBuild GitHub repository.
Your messages have been burning a hole in my inbox, but I didn't get to
responding to them yet...
On 09/02/15 22:49, Cook, Malcolm wrote:
Hi,
I am currently learning EasyBuild/Lmod and strategizing for deployment
to a common network shared mountpoint, say $HPCBC, for use by about
half-dozen HPC linux running CentOS7.
I wonder if there is any information that would help me with pros/cons
on waiting for EB 2.x before deployment, such as:
intended scope of changes
intended backward compatibility (should all existing modules 'just work')
ETA for first release intended for general availability (non-beta)
EasyBuild v2.0 is going to include a couple of backward-incompatible
changes (hence the major version bump), but most of these changes
shouldn't affect users new to the tool.
All the easyblocks and easyconfigs files included in EasyBuild v1.16.1
will still work with EasyBuild v2.0, since non of them are relying on
deprecated functionality for which support will be dropped in EasyBuild
v2.0.
For all details w.r.t. backwards-incompatible changes, see
http://easybuild.readthedocs.org/en/latest/Deprecated-functionality.html .
There are a couple of other major changes we're planning on including in
EasyBuild v2.0, i.e., to stop including a copy of the vsc-base Python
package that serves as a base for the EasyBuild framework and use it as
a proper dependency instead, and dropping support for Python 2.4.x in
favor of Python 2.6.x.
In practice, these changes should not affect end users (assuming your
system Python is 2.6 or a newer Python 2 version).
A detailed overview of the (major) changes being made in EasyBuild v2.0
is available at
https://github.com/hpcugent/easybuild-framework/issues/1000, which also
provides an up-to-date status of the progress being made.
In terms of planning: EasyBuild was planned to be the first release in
2015, and we're sticking to that (i.e. there will be no further releases
on the 1.x branch).
I was hoping to push out EasyBuild v2.0 last week, in time for the 8th
EasyBuild hackathon which is running in Basel this week (see
https://github.com/hpcugent/easybuild/wiki/8th-EasyBuild-hackathon), but
I didn't manage to get it ready (and I didn't want to rush it out either).
So, it'll probably be shifted more towards the end of this month, since
I'll be busy with the hackathon this week, and will have other stuff to
attend to once I'm back at the office.
Also, I am somewhat struggling with my EB 1.x evaluation/testing (also
on CentOS7) and wonder about relative value of using the various
installation methods, being
1) python bootstrap_eb.py $HPCBC/easybuild # currently yields version
1.16.1
2) pip install --install-option "--prefix=${HPCBC}/EasyBuild" easybuild
3)
> pip install --install-option "--prefix=1.16.1" easybuild
b) pip install --install-option "--prefix=1.16.1" easybuild
My impression is that the advantage of using the bootstrap procedure
is that it builds a 'module' file.Is there more that I am missing?
If the bootstrap procedure works for you, and you're happy with using
EasyBuild by loading it as a module, I would recommend that approach.
If you want EasyBuild installed on the system directly so you can use it
without having to load a module, you can resort to the standard Python
installation tools like easy_install or pip.
The main reason we introduced the bootstrap procedure is because the
standard Python installation tools tend to be quite quirky, depending on
how they were installed or configured... Pro tip: stay away from using
the "--user" install option...
Finally, I have the following observations about the install that I
hope you find worthy additions to your installation documentation:
xmlrunner is python module optionally used by eb boostrap unit test
which allows them to run faster! NB - in my hands the test results are
DIFFERENT if present - perhaps a "bad thing". Can be installed with:
`pip install xmlrunner`-- add to
http://easybuild.readthedocs.org/en/latest/Installation.html#optional-dependencies
Thanks for bringing this up. xmlrunner should be entirely optional. I
wasn't aware that it can make the test results run faster... Do you have
any more details there?
There are some known issues w.r.t. robustness of the unit tests, which
we hope to fix in EasyBuild v2.0 (see also the references in
https://github.com/hpcugent/easybuild-framework/issues/1000).
Do you have more details to share on which unit tests are failing for
you? Maybe include the output you're getting in a gist
(https://gist.github.com)
tcl need be installed (in my experience) (regardless of whether Tcl/C
environment modules are to be used - i.e. even if only Lmod is to be
used) - add to
http://easybuild.readthedocs.org/en/latest/Installation.html#requirements
Yes, tclsh is required even if you're using Lmod, because (for now)
EasyBuild only supports generating module files in Tcl syntax, and Lmod
needs tclsh to be able to translate Tcl to Lua on the fly.
Lmod will now also complain if tclsh is missing and it was configured to
be able to consume Tcl module files, since we reported this issue to
Robert McLay, the Lmod maintainer.
We will also mention this in the documentation, as you suggest.
if Lua is built and installed in a non standard location, LUA_ROOT
should be redefined accordingly, for example:
>sed -i '/#define LUA_ROOT/s:/usr/local/:${INSTALL_TOP}/:' src/luaconf.h
This is an Lmod issue, but: i) lua should be available in $PATH when
building/installing Lmod, ii) if Lmod also requires lua.h (it depends on
whether you have lua-term available on your system or not), you can
point to its location using the --with-lua-include configure option.
There is a known issue in Lmod 5.8.6 w.r.t. finding lua.h in the 'make
install' step, which is already fixed in the HEAD version of Lmod in the
GitHub repository (see
https://github.com/TACC/Lmod/commit/ba889b1887d5b7484e420ec1f0dfce372d36e11e).
Don't hesitate to let us know if you have any further questions or
remarks/suggestions.
regards,
Kenneth
