Thanks for all the helpful information and for the welcome aboard!
On 10/09/2015 03:26 PM, Kenneth Hoste wrote:
> Hi John-Paul,
>
> On 09/10/15 21:43, John-Paul Robinson wrote:
>> Hi,
>>
>> I'm using EasyBuild 2.3.0 and are wanting to build Cufflinks. We tried
>> the Cufflinks-2.2.1-goolf-1.4.10.eb on CentOS and the build fails.
> How does it fail exactly? It works fine on our Scientific Linux 6 system.
>
The make is failing due to a syntax error in the documentation files.
../../gcc/doc/cppopts.texi:806: @itemx must follow @item
make[3]: *** [doc/cpp.info] Error 1
make[3]: *** Waiting for unfinished jobs....
<snip>
make[3]: *** [doc/gcc.info] Error 1
make[3]: Leaving directory
`/home/jpr/.local/easybuild/build/GCC/4.7.2/dummy-dummy/gcc-4.7.2/obj/gcc'
make[2]: *** [all-stage1-gcc] Error 2
make[2]: Leaving directory
`/home/jpr/.local/easybuild/build/GCC/4.7.2/dummy-dummy/gcc-4.7.2/obj'
make[1]: *** [stage1-bubble] Error 2
make[1]: Leaving directory
`/home/jpr/.local/easybuild/build/GCC/4.7.2/dummy-dummy/gcc-4.7.2/obj'
make: *** [bootstrap] Error 2
(at
easybuild/2.3.0/software/EasyBuild/2.3.0/lib/python2.7/site-packages/easybuild_framework-2.3.0-py2.7.egg/easybuild/tools/run.py:360
in parse_cmd_output)
== 2015-10-08 11:04:18,816 main.easyblock WARNING build failed
(first 300 chars): cmd " make -j 4 bootstrap " exited with exitcode
2 and output:
echo stage3 > stage_final
According to the GCC bug, this is the result of stricter parsing of the
texi files which catch an error that was previously ignored.
>> Fixing the the build falls into one of several categories:
>> * patch GCC 4.7.2 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52903)
>> * update goolf-1.4.10 to use GCC 4.7.3 (fixes the bug in 4.7.2)
>> * create a new Cufflinks easyconfig based on goolf-1.7.20
>>
>> As a learning exercise, I'm following each path. (BTW, I do plan to
>> submit pull requests once my builds are verified.) In doing so, I'm
>> wondering if there are there any recommended practices for updating and
>> easyconfig that has a bug, changing a compiler version dependency or
>> introducing new easyconfigs.
>
> I wouldn't start with backporting patches to older an compiler version
> and rebuilding the compiler, especially since there's a new minor
> release (4.7.3) for the same major compiler version (4.7). Just moving
> on to the newer minor (or even major) release of the compiler makes
> more sense, imho.
>
> We never update toolchain components without bumping the toolchain
> version in the easyconfigs we ship via the easybuild-easyconfigs
> repository, and I wouldn't recommend it either. Bumping a compiler (or
> any other component) should imply also bumping the toolchain version
> (that's the whole point of having a toolchain version imho).
>
> Rebuilding Cufflinks with a newer toolchain version makes most sense
> to me.
> And of course, a pull request for it makes sense. :-)
>
Good to know. I do plan on adding the updated Cufflinks. Currently
working through an error finding the bam library. Trying to understand
how the EBROOT<tag> env vars get set. Will post a separate thread if I
can't figure it out shortly.
I realize the bug in GCC 4.7.2 at this point is pretty old and it's
arguably better to move forward than fix the past. I was mainly
interested in understanding how to approach such scenarios if they are
current.
BTW, I really like that are straight forward mechanisms within Easybuild
to address each of these issues, with a patch automatically applied or
updated easyconfig.
>
>> I assume the git-hooks are designed to allow sites to build against
>> pending upstream commits using the "build against pull request feature".
> You're referring to eb --from-pr here, right?
>
> It is mostly intended to easily pull requests to check whether they're
> good to merge.
>
> For new easyconfigs, it can also be used to build & install things not
> included in the latest release.
> But beware that later pull requests may modify the easyconfig before
> it gets included in a release (for whatever reason)...
>
>> I haven't seen much in the way of recommendations for handling the local
>> environment to make contributing easier. For example, how to install
>> the easyconfig or blocks so that I can work against a git clone'd tree
>> (I'm currently using a symlink to the repo easyconfig dir inside my
>> personal easybuid env).
>
> There are a couple of ways to sit on top of the current 'develop'
> branches, but it also a risk (just so you're aware).
>
> Easyblocks and easyconfigs in the 'develop' branch may require
> features in the EasyBuild framework that are not included in the
> latest EasyBuild yet, and for all intents and purposes you should
> always consider 'develop' to be unstable.
> Although we are quite (very, even) careful when merging changes to the
> develop branch, it's always possible that bugs are introduced that
> only pop up during a thorough regression test (which is typically only
> done shortly before a new release).
>
> One way is to simply clone the different repositories (framework,
> easyblocks, easyconfigs), set $PYTHONPATH and $PATH, and use EasyBuild
> develop directly.
>
> Another option is to install the develop branches using
> easy_install/pip/whatever (for example, easy_install --prefix <prefix>
> https://github.com/hpcugent/easybuild-framework/archive/develop.tar.gz, after
> setting $PYTHONPATH and pre-creating the 'site-packages' dir).
>
> If you only care about easyconfigs, you can clone/download the develop
> branch, and use the --robot-paths configuration option (see
> http://easybuild.readthedocs.org/en/latest/Using_the_EasyBuild_command_line.html#controlling-the-robot-search-path)
> to make sure EasyBuild will pick up these easyconfigs.
> A simple "git pull" can make sure you're getting the latest
> easyconfigs that were reviewed/tested thoroughly before merging.
>
> To use 'develop' easyblocks by occasion, the --include-easyblocks
> configuration option can be very useful (see
> http://easybuild.readthedocs.org/en/latest/Including_additional_Python_modules.html#include-easyblocks).
>
> There's also a script available to set up the 'develop' clones, see
> http://easybuild.readthedocs.org/en/latest/Installation_Alternative.html#installation-of-latest-development-version-using-provided-script
> .
>
Thanks this helps.
I'm less interested in sitting atop the develop branch than in a local
mods branch. Thanks for the reminder to actually pull the from a
release in the repo.
>> Given that the easyconfigs are all distributed
>> from upstream in one big chunk vs. individual repos, I may have a need
>> to maintain local variations that don't make it into the upstream or
>> aren't appropriate for it.
>>
>> What approaches do you recommend for handling these situations?
>
> To maintain local variations, you can throw your easyconfigs in a
> separate directory, and list it first in --robot-paths.
> That way, your customized easyconfigs will be considered first, before
> other easyconfig files with the same name.
>
> Currently, we don't have a good way of keeping locally maintained
> copies of easyconfigs in sync with their equivalent 'upstream'.
> One idea has been to add support for 'easyconfig patch files' (.ebp),
> where you can just specify the customizations, rather than requiring
> you to copy the whole easyconfig file. But nobody has stepped up to
> implement this yet, so it remains just an idea.
>
> I hope this helps, do let us know if you have any more questions.
>
> Unless I'm mistaken, this is your first post to the list, so: welcome
> to the wonderful world of EasyBuild!
>
>
Thanks again for the welcome and helpful feedback.
> regards,
>
> Kenneth
