Hi Stuart,
On 10/03/15 18:18, Stuart Barkley wrote:
Is there a good way to see a dependency tree with easybuild (I'm now
running 2.0.0)? I'm trying to reconcile different dependency
requirements for my large build and am finding it difficult to
determine what packages are requiring other packages.
Have you considered trying eb --dep-graph?
That can be used to generate a .dot file describing the dependency graph
(eb will tell you what's missing).
With the right optional Python packages installed, you can also make it
dump a rendered image (.png, .pdf).
For example:
eb --dep-graph=goolf.dot goolf-1.4.10.eb -r
Currently I am using 'eb --dry-run --robot' to get a complete list of
packages for a potential build, but I cant seem to figure out how to
determine which packages are causing certain other things to be
required.
Yes, it's just a flat list now. We might be able to make it smarter, but
it's certainly non-trivial since a particular package may be a
dependency for several others.
Up until now, I've just been doing builds and when things die, fix
that problem and restart the build. This works after a fashion, but
takes a long time when there are multiple issues.
What kind of problem are you running into, specifically?
Also, once I've
completed a build I need to go back are rebuild everything again in
case one of the later changes impacted an earlier build.
How could changes made later impact an earlier build?
After a build is complete, I can do some grepping of the .eb files in
ebfiles_repo, but even this gets awkward.
Grepping to do what exactly? Figure out what depends on what? If so, use
--dep-graph, it's what you want.
Part of the cause of this issue is that I'm creating a couple of .eb
files with the set of software I want to install (similar to the
HPCBIOS files). Currently, it appears that everything in a single .eb
file needs to use the same library versions and some of the packages
use different library versions (zlib, libpng, boost, cmake).
Yes, since all dependency specifications need to include strict versions
specification, and by installing things via a single easyconfig file,
you're basically saying all of those modules need to be loaded at the
same time, so you can only have one version of each software package in
there.
Alternatively, you can just supply a directory of easyconfig files,
rather than grouping them together in a single easyconfig file?
Another useful option would be a --keep-going option to have easybuild
continue on with other packages even if one fails. I would like to
start a build before going home and have easybuild do as much as
possible.
You can trigger this kind of behavior by using --dump-test-report, since
EasyBuild will always try to put together a full test report for
whatever if was specified to do.
There should be a dedicated runtime option for this though, that's true.
Should be fairly trivial (see the definition of 'exit_on_failure' in
easybuild/main.py).
This is only useful in case of building and installing multiple
independent groups of software packages, of course, since as soon as
building one dependency fails everything that required it will/should fail.
regards,
Kenneth
