Hi Stuart,
On 26/02/15 20:11, Stuart Barkley wrote:
After a little hiatus I'm back working using easybuild to build
software for our system.
One issue which was confusing me for a while was some unexpected
choices for the *.eb files under different conditions.
Thanks for bringing this up!
I was making modifications to some of the standard .eb files by
putting modified copies in my working directory. Changes included
adding the --with-sge option in the OpenMPI .eb file or adjusting
software versions (some older distfiles are not longer available for
download). I also was creating a meta .eb file to reference other
packages.
I was getting inconsistencies between doing explicit eb commands and
using the --robot option. Eventually I found out that my local
changes were not being seen with --robot and that I needed to add a
--robot-paths option in my configuration file.
Later I had enough .eb files that I decided to move them into a
easyconfig subdirectory (adjusting the --robot-paths option). At that
time I found that just doing 'eb xxx.eb' was not finding the files in
my private easyconfig directory but instead again using the eb-1.6.1
distributed versions. This is still a little awkward in that I need
to include the directory when building a package with local
modifications.
My suggestion is that --robot-path (perhaps renamed) should apply to
.eb files given on the command line in addition to those searched for
with --robot.
OK, I think I understand the problem here, but it seems like you don't
have a good view on what's actually happening (which doesn't mean you're
not pointing out an important issue).
Both --robot-paths and --robot accept paths to easyconfig files to use
when resolving dependencies, not just --robot-paths.
That's mostly historic, but it's also useful: paths specified to --robot
are considered *first*, and you can avoid that the readily provided
easyconfigs that are part of the installation are ever used by defining
--robot-paths (and overwrite the default).
The details of how --robot-paths and --robot work together are
(hopefully) clearly documented at
http://easybuild.readthedocs.org/en/latest/Using_the_EasyBuild_command_line.html#controlling-robot-search-path
.
What you are stumbling over is that --robot-paths (in combination with
any paths that may be specified via --robot) are not being considered
for easyconfig files that are specified on the command line (by
name-only/relative path), while the readily provided easyconfig files
*are* being consider.
To clarify, with "eb foo.eb", if there's no file named 'foo.eb' in the
current working directory, eb will go and see whether there's an
easyconfig file with that name in the collection of readily avaiable
easyconfigs that are part of the EasyBuild installation, but it will not
look for 'foo.eb' in the directories listed in --robot-paths (or --robot).
Did I get that right?
If so: I consider this a bug. I see no reason why we are not considering
the 'robot' paths for easyconfigs that are only specified by name (or
relative path)...
Frankly: I'm quite surprised this hasn't been brought up before...
(maybe someone will correct me and point out an ancient issue in the
GitHub repo now)
How do others feel about this?
Although the window is closing fast, we can get this fixed (fairly
easily) in the upcoming EasyBuild v2.0...
I propose we discuss this during the EasyBuild conf call tomorrow (but
feel free to share your thought in this thread too).
regards,
Kenneth
