Hi Kenneth,
That makes a lot of sense. It also led me to more RTFM. I wanted to utilize our
existing module system on the HPC system as much as possible (to save on compile
time and space), so I opted for your third suggestion. However, it looks like
the direct dependencies of CP2K all need to be compiled, which means I would
have to follow the chain and modify the dependencies of the direct dependencies
to use external modules? This doesn't seem a clean and sustainable approach. I
then tried using
`--filter-deps=Autotools,binutils,flex,GCCcore,GMP,icc,iccifort,ifort,imkl,impi,intel,libtool,M4,pkg-config,zlib`
to exclude all the things I can `module add`. The dry-run seems to indicate this
is what I want, with only one exception: intel/2016b. The compiler toolchain
isn't listed as dependency, so it wasn't filtered by `--filter-dep`, and while
building it complained that `intel/2016b` conflicted with the module I loaded:
~~~
== FAILED: Installation ended unsuccessfully (build directory:
/home/user/tools/easybuild/build/intel/2016b/dummy-dummy): build failed (first
300 chars): intel/2016b(2
3):ERROR:150: Module 'intel/2016b' conflicts with the currently loaded
module(s) 'intel/2016/update3'
== Results of the build can be found in the log file(s)
/tmp/eb-e83z38/easybuild-intel-2016b-20170929.162613.JEywY.log
ERROR: Build of
/home/user/tools/easybuild/software/EasyBuild/3.4.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.4.0-py2.7.egg/easybuild/easyconfigs/i/intel/int
el-2016b.eb failed (err: "build failed (first 300 chars): intel/2016b(23):ERROR:150:
Module 'intel/2016b' conflicts with the currently loaded module(s)
'intel/2016/update3'")
~~~
Is there a way to use external compiler module to satisfy the compiler toolchain
as well?
Thanks,
Roc
On 9/29/2017 3:12 PM, Kenneth Hoste wrote:
Dear Roc,
I think your problem is that you have a binutils/2.26 already available on your
system, and 'eb' is assuming it was installed with EasyBuild...
All modules generated with EasyBuild define an environment variable like
$EBROOT<NAME>, where <NAME> is the software name in capitals.
EasyBuild relies on this later when the modules for dependencies are being
loaded, the 'get_software_root' function picks up $EBROOT*.
I think you have three options:
* don't have modules not built with EasyBuild "in view" when installing
software with EasyBuild, i.e. clean up $MODULEPATH first
* reinstalling the binutils/2.26 module with EasyBuild so it is happy, using "eb
binutils-2.26.eb --force"; whether this is an option depends on whether or not that
binutils module is in use already
* inform EasyBuild about this "external module" so it can pick it up and use
it, see http://easybuild.readthedocs.io/en/latest/Using_external_modules.html
regards,
Kenneth
On 29/09/2017 20:55, rocwh...@list.ru wrote:
Hi Jack,
I didn't set the robot path before, but `eb` did seem to have built at least
some dependencies successfully. Setting the robot path still gave the same error
though. Below is the output of the dry run.
Thank you!
Roc
~~~
$ export EASYBUILD_ROBOT=.
$ eb binutils-2.26-GCCcore-5.4.0.eb -r -D == temporary log file in case of
crash /tmp/eb-ETUuSS/easybuild-WynaPz.log
Dry run: printing build status of easyconfigs and dependencies
CFGS=/home/user/tools/easybuild/software/EasyBuild/3.4.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.4.0-py2.7.egg/easybuild/easyconfigs
* [x] $CFGS/m/M4/M4-1.4.17.eb (module: M4/1.4.17)
* [ ] $CFGS/b/Bison/Bison-3.0.4.eb (module: Bison/3.0.4)
* [ ] $CFGS/f/flex/flex-2.6.0.eb (module: flex/2.6.0)
* [x] $CFGS/z/zlib/zlib-1.2.8.eb (module: zlib/1.2.8)
* [x] $CFGS/b/binutils/binutils-2.26.eb (module: binutils/2.26)
* [x] $CFGS/g/GCCcore/GCCcore-5.4.0.eb (module: GCCcore/5.4.0)
* [ ] $CFGS/m/M4/M4-1.4.17-GCCcore-5.4.0.eb (module: M4/1.4.17-GCCcore-5.4.0)
* [ ] $CFGS/z/zlib/zlib-1.2.8-GCCcore-5.4.0.eb (module:
zlib/1.2.8-GCCcore-5.4.0)
* [ ] $CFGS/b/Bison/Bison-3.0.4-GCCcore-5.4.0.eb (module:
Bison/3.0.4-GCCcore-5.4.0)
* [ ] $CFGS/f/flex/flex-2.6.0-GCCcore-5.4.0.eb (module:
flex/2.6.0-GCCcore-5.4.0)
* [ ] $CFGS/b/binutils/binutils-2.26-GCCcore-5.4.0.eb (module:
binutils/2.26-GCCcore-5.4.0)
~~~
On 9/29/2017 12:11 PM, Jack Perdue wrote:
Howdy Roc,
Do you have a robot path set? e.g.
export EASYBUILD_ROBOT=. # dot/period == current directory.... EB will search
its path as well
As long as you set ROBOT to anything it should find the default
dependencies to build automatically (AFAIK).
The '-D' option to 'eb' is also very helpful.
Jack Perdue
Lead Systems Administrator
High Performance Research Computing
TAMU Division of Research
j-per...@tamu.edu http://hprc.tamu.edu
HPRC Helpdesk: h...@hprc.tamu.edu
On 09/29/2017 11:48 AM, rocwh...@list.ru wrote:
Thanks for the quick reply, Umit!
I didn't see "ready modules" or similar in the binutils eb file. There is a
toolchain `GCCcore-5.4`, which was built prior to binutils and appears to have
succeeded.
More generally, how do I find out which packages I need to compile separately
beforehand? I thought that dependencies would be built automatically? The
command that triggered the binutils error was `eb CP2K-4.1-intel-2016b.eb -r` on
a freshly installed EasyBuild.
Thanks,
Roc
On 9/29/2017 11:30 AM, Sami, Umit D. wrote:
Open up binutils .eb file and look for dependencies and make sure you have
compiled “ready modules” for such dependencies (i.e. GCCcore-5.4)
-----------------------------------
Umit D. Sami
HPC Apps & Systems (ERIS)
w: http://www.partners.org
On 9/29/17, 12:27 PM, "easybuild-requ...@lists.ugent.be on behalf of
rocwh...@list.ru" <easybuild-requ...@lists.ugent.be on behalf of rocwh...@list.ru>
wrote:
Hello,
I was trying to build a package but encountered an error while
building
`binutils-2.26-GCCcore-5.4.0.eb`:
~~~
== FAILED: Installation ended unsuccessfully (build directory:
/home/user/tools/easybuild/build/M4/1.4.17/GCCcore-5.4.0): build failed (first
300 chars): get_software_root software root for binutils was not found in
environment
== Results of the build can be found in the log file(s)
/tmp/eb-oIb4DL/easybuild-M4-1.4.17-20170929.103852.tDgnB.log
ERROR: Build of
/home/user/tools/easybuild/software/EasyBuild/3.4.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.4.0-py2.7.egg/easybuild/easyconfigs/m/M4/M4-1.4.17-GCCcore-5.4.0.eb
failed (err: 'build failed (first 300 chars): get_software_root software root
for binutils was not found in environment')
~~~
The relevant lines in the log file are:
~~~
== 2017-09-29 10:38:58,701 compiler.py:321 INFO _set_optimal_architecture:
using march=native as optarch for x86_64.
== 2017-09-29 10:38:58,968 build_log.py:157 ERROR EasyBuild crashed with
an error (at
easybuild/software/EasyBuild/3.4.0/lib/python2.7/site-packages/vsc_base-2.5.8-py2.7.egg/vsc/utils/exceptions.py:124
in __init__): get_software_root software root for binutils was not found in
environment (at
easybuild/software/EasyBuild/3.4.0/lib/python2.7/site-packages/easybuild_framework-3.4.0-py2.7.egg/easybuild/tools/toolchain/toolchain.py:278
in _get_software_root)
== 2017-09-29 10:38:58,968 easyblock.py:2650 WARNING build failed (first
300 chars): get_software_root software root for binutils was not found in
environment
== 2017-09-29 10:38:58,968 easyblock.py:287 INFO Closing log for
application name M4 version 1.4.17
~~~
Could someone please provide some ideas what might have gone wrong?
Thanks very
much!
Regards,
Roc
The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.
