On 12 Dec 2013, at 23:18, Bart Verleye wrote:
> Hi,
>
> I have a couple of questions on how to use easybuild (EB) with different
> admins and how to contribute in that case.
>
> I installed EB on a shared drive, and provided a file like the one attached.
> If this file is sourced by the admin, the admin is ready to install software
> with EB.
> So far so good.
Remarks w.r.t. the script:
PYLIB=`python -c "import distutils.sysconfig; print
distutils.sysconfig.get_python_lib(prefix='/share/easybuild/installation'); "`
export
PYTHONPATH=/share/easybuild/installation/lib/python2.6/site-packages/:$PYLIB
Doesn't this yield two exact same entries in $PYTHONPATH? Also, you should
probably consider prepending to an existing $PYTHONPATH, rather than hard
resetting it.
To make it easier to change the /share/easybuild prefix, maybe introduce a
PREFIX variable on top, and use it in other definitions.
You should be using $EASYBUILD_PREFIX rather than $EASYBUILDPREFIX, even though
both work (for now). Same goes for build, install, source path.
See
https://github.com/hpcugent/easybuild/wiki/Configuration#legacy-configuration-deprecated
for more details.
With your current settings, just having the PREFIX setting and SOURCE setting
is sufficient, since you have the build/install path "following" prefix (EB
will do that for you).
> Now an admin wants to create an easyblock and easyconfig for a new software.
> (S)he can do that wherever, but than it becomes difficult to contribute. So
> every admin clones the github in the home directory, and follows
> https://github.com/hpcugent/easybuild/wiki/Setting-up-your-own-easyblocks-repository
> with MYEBDIR the github-clone one. Is that correct?
Yes, every admin can host his own easyblock repo, to avoid annoying others. The
key is not $MYEBDIR, EasyBuild doesn't pick that up or something, it's just a
convenient way in the docs. That variable might as well be called $HERE,
$THISISTHELOCATION or whatever.
The key is setting $PYTHONPATH correctly (step 3 in the docs), and making sure
the easybuild/easyblocks namespace is there (mainly step 2).
I'm assuming admins are not sharing an installation account here?
We have a dedicated install user on our site, and just share that account among
people performing installations.
To be clear: I'm not saying that's a better option, at all. It's probably
better to have separate accounts with "EB rights", and a dedicated easybuild
group, and have the install users be a member of that. That would make it
clearer who did which install (which is a lot harder for us right now).
> But then the added files are not available for the installed system until the
> new release of EB, right?
No, if you set $PYTHONPATH like in step 3, EasyBuild will find your local
easyblocks, and will use them when needed.
You can check this using for example "eb --list-easyblocks" .
No reinstallation is required, that's the whole intention of the procedure
explained at
https://github.com/hpcugent/easybuild/wiki/Setting-up-your-own-easyblocks-repository.
Same goes for toolchain support BTW, see
https://github.com/hpcugent/easybuild/wiki/Compiler-toolchains#create-a-new-compiler-toolchain
.
> The same for the easy-configs?
For easyconfig files, you can set up a directory $EASYCONFIGS_HERE, with a
subdirectory easybuild/easyconfigs in there, and add that path to $PYTHONPATH:
export PYTHONPATH=$EASYCONFIGS_HERE:$PYTHONPATH
When you do that, --robot will also find easyconfigs files in there (it
searches in the Python search path, controlled by $PYTHONPATH, for
subdirectories easybuild/easyconfigs, and then walks through them looking for
easyconfig files).
In EasyBuild v1.10 however, you'll be able to pass multiple paths to --robot,
which is almost as convenient (see
https://github.com/hpcugent/easybuild-framework/pull/786, just merged into
develop).
> What if I want to make a small local change in eb file. For example the path
> to the source of licences from intel software. I don't think I want to commit
> that to the git, but I do want to keep track of the change in our
> installation of eb, so that a reinstall has the correct information.
Making site customizations easier are the next big thing we'll be working on,
so hopefully you won't need to manually edit easyconfig files anymore.
For Intel license files specifically, you can just set $INTEL_LICENSE_FILE
outside of EB. The easyblock for Intel tools will pick up $INTEL_LICENSE_FILE,
and it's preferred over the license_file spec in the easyconfig file if it's
set.
For now, you can keep all customized easyconfigs in a separate directory, and
point to it with --robot. Especially with EB v1.10 this will become useful
since you can supply multiple robot paths.
EasyBuild will always use the installed easyconfigs as a last resort (in EB
v1.10).
> I'm not a PythonPATH or Git expert, but I'm sure your answer will help me to
> become one!
I hope the above helps, let us know if you have any problems getting it to work.
regards,
Kenneth
