Hi John-Paul,
On 09/10/15 21:43, John-Paul Robinson wrote:
Hi,
I'm using EasyBuild 2.3.0 and are wanting to build Cufflinks. We tried
the Cufflinks-2.2.1-goolf-1.4.10.eb on CentOS and the build fails.
How does it fail exactly? It works fine on our Scientific Linux 6 system.
Fixing the the build falls into one of several categories:
* patch GCC 4.7.2 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52903)
* update goolf-1.4.10 to use GCC 4.7.3 (fixes the bug in 4.7.2)
* create a new Cufflinks easyconfig based on goolf-1.7.20
As a learning exercise, I'm following each path. (BTW, I do plan to
submit pull requests once my builds are verified.) In doing so, I'm
wondering if there are there any recommended practices for updating and
easyconfig that has a bug, changing a compiler version dependency or
introducing new easyconfigs.
I wouldn't start with backporting patches to older an compiler version
and rebuilding the compiler, especially since there's a new minor
release (4.7.3) for the same major compiler version (4.7). Just moving
on to the newer minor (or even major) release of the compiler makes more
sense, imho.
We never update toolchain components without bumping the toolchain
version in the easyconfigs we ship via the easybuild-easyconfigs
repository, and I wouldn't recommend it either. Bumping a compiler (or
any other component) should imply also bumping the toolchain version
(that's the whole point of having a toolchain version imho).
Rebuilding Cufflinks with a newer toolchain version makes most sense to me.
And of course, a pull request for it makes sense. :-)
I assume the git-hooks are designed to allow sites to build against
pending upstream commits using the "build against pull request feature".
You're referring to eb --from-pr here, right?
It is mostly intended to easily pull requests to check whether they're
good to merge.
For new easyconfigs, it can also be used to build & install things not
included in the latest release.
But beware that later pull requests may modify the easyconfig before it
gets included in a release (for whatever reason)...
I haven't seen much in the way of recommendations for handling the local
environment to make contributing easier. For example, how to install
the easyconfig or blocks so that I can work against a git clone'd tree
(I'm currently using a symlink to the repo easyconfig dir inside my
personal easybuid env).
There are a couple of ways to sit on top of the current 'develop'
branches, but it also a risk (just so you're aware).
Easyblocks and easyconfigs in the 'develop' branch may require features
in the EasyBuild framework that are not included in the latest EasyBuild
yet, and for all intents and purposes you should always consider
'develop' to be unstable.
Although we are quite (very, even) careful when merging changes to the
develop branch, it's always possible that bugs are introduced that only
pop up during a thorough regression test (which is typically only done
shortly before a new release).
One way is to simply clone the different repositories (framework,
easyblocks, easyconfigs), set $PYTHONPATH and $PATH, and use EasyBuild
develop directly.
Another option is to install the develop branches using
easy_install/pip/whatever (for example, easy_install --prefix <prefix>
https://github.com/hpcugent/easybuild-framework/archive/develop.tar.gz,
after setting $PYTHONPATH and pre-creating the 'site-packages' dir).
If you only care about easyconfigs, you can clone/download the develop
branch, and use the --robot-paths configuration option (see
http://easybuild.readthedocs.org/en/latest/Using_the_EasyBuild_command_line.html#controlling-the-robot-search-path)
to make sure EasyBuild will pick up these easyconfigs.
A simple "git pull" can make sure you're getting the latest easyconfigs
that were reviewed/tested thoroughly before merging.
To use 'develop' easyblocks by occasion, the --include-easyblocks
configuration option can be very useful (see
http://easybuild.readthedocs.org/en/latest/Including_additional_Python_modules.html#include-easyblocks).
There's also a script available to set up the 'develop' clones, see
http://easybuild.readthedocs.org/en/latest/Installation_Alternative.html#installation-of-latest-development-version-using-provided-script
.
Given that the easyconfigs are all distributed
from upstream in one big chunk vs. individual repos, I may have a need
to maintain local variations that don't make it into the upstream or
aren't appropriate for it.
What approaches do you recommend for handling these situations?
To maintain local variations, you can throw your easyconfigs in a
separate directory, and list it first in --robot-paths.
That way, your customized easyconfigs will be considered first, before
other easyconfig files with the same name.
Currently, we don't have a good way of keeping locally maintained copies
of easyconfigs in sync with their equivalent 'upstream'.
One idea has been to add support for 'easyconfig patch files' (.ebp),
where you can just specify the customizations, rather than requiring you
to copy the whole easyconfig file. But nobody has stepped up to
implement this yet, so it remains just an idea.
I hope this helps, do let us know if you have any more questions.
Unless I'm mistaken, this is your first post to the list, so: welcome to
the wonderful world of EasyBuild!
regards,
Kenneth
