RE: [easybuild] next EasyBuild conf call: Wed Sept 4th (*today*) 2019, 5pm CEST

[Vanzo, Davide]

Notes of today's conf call are available at:
https://github.com/easybuilders/easybuild/wiki/Conference-call-notes-20190904

The next conf call is scheduled for Wed Sep 18 2019, 5pm CEST.

--
Davide Vanzo, PhD
Sr. HPC Research Scientist
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre


On 2019-09-04 04:57:01-05:00 easybuild-requ...@lists.ugent.be wrote:

Dear EasyBuilders,

The next EasyBuild conf call is planned for Wed Sept 4th 2019 (today),
at 5pm CEST.

This conf call will be hosted by Davide Vanzo, I won't be able to join
the call myself, unexpectedly.

You can join the conf call via 
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftiny.cc%2Feb_conf_callamp;data=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C6ed7e1e0778d43ab815808d7311e3998%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C637031878205931083amp;sdata=dFRGotIOv%2FJtKdVjRdRbiE3ta8dCN6qHIcAnJspzhVQ%3Damp;reserved=0
 .

Current agenda:

* update on progress towards EasyBuild 4.0

* update on 2019b common toolchains

* Qamp;A

Suggestions for additional topics are welcome (please let me know if you
have any).


More information on the EasyBuild conf calls is available at
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feasybuilders%2Feasybuild%2Fwiki%2FConference-callsamp;data=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C6ed7e1e0778d43ab815808d7311e3998%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C637031878205941076amp;sdata=81uYaS3AezH93%2F7cXh0NHcaCTA5qYl9qhzukhZWpc70%3Damp;reserved=0
 .


regards,

Kenneth

[easybuild] RE: Next EasyBuild conf call: Wed July 24th 2019, 5om CEST

[Vanzo, Davide]

Notes of today's conf call are available at:
https://github.com/easybuilders/easybuild/wiki/Conference-call-notes-20190724


The n​​​ext conf call is planned for Wed August 7th.

Best regards
--
Davide Vanzo, PhD
Sr. HPC Research Scientist
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre

[easybuild] Next EasyBuild conf call: Wed July 24th 2019, 5om CEST

[Vanzo, Davide]

Dear EasyBuilders,
The next EasyBuild conf call is planned for Wed July 24th 2019, at 5pm CEST.

You can join the conf call via https://umu.zoom.us/j/561463243
Current agenda:
* Removing R and Perl version suffixes from 2019b
* Q
Suggestions for additional topics are welcome (please let me know if you have 
any).

More information on the EasyBuild conf calls are available at 
https://github.com/easybuilders/easybuild/wiki/Conference-calls

Best regards

--
Davide Vanzo, PhD
Sr. HPC Research Scientist
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre

[easybuild] Next EasyBuild conf call: Wed June 12th 2019, 5pm CET

[Vanzo, Davide]

Dear EasyBuilders,

The next EasyBuild conf call is planned for Wed June 12th 2019, at 5pm CET.

You can join the conf call via https://umu.zoom.us/j/561463243 .

Current agenda:
* Changes in container support for v3.9.2
* Updates on v4.0
* Q

Suggestions for additional topics are welcome (please let me know if you have 
any).

More information on the EasyBuild conf calls is available at
https://github.com/easybuilders/easybuild/wiki/Conference-calls .

Best regards,
--
Davide Vanzo, PhD
Sr. HPC Research Scientist
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre

[easybuild] Next EasyBuild conf call: Wed May 1st 2019, 5pm CET

[Vanzo, Davide]

Dear EasyBuilders,
The next EasyBuild conf call is planned for Wed May 1st 2019, at 5pm CET.
You can join the conf call via https://umu.zoom.us/j/561463243 .
Current agenda:
* Q
Suggestions for additional topics are welcome (please let me know if you have 
any).
More information on the EasyBuild conf calls is available at
https://github.com/easybuilders/easybuild/wiki/Conference-calls .
Best regards,
--
Davide Vanzo, PhD
Sr. HPC Research Scientist
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre

RE: [easybuild] Assertion error for PyTorch-0.3.1 with intelcuda-2017b

[Vanzo, Davide]

Lars,

Thank you for the feedback.
So, using gnu++11 solves the problem but creates a new one. With that standard 
nvcc+icc goes bananas and throws undefined __float128 identifier errors.
I'll try to go down the route of fixing the object.

--
Davide Vanzo, PhD
Sr. HPC Research Scientist
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre


On 2019-02-23 14:49:31-06:00 easybuild-requ...@lists.ugent.be wrote:

In this case, it's kind of rightfully picky - designated initializers like that 
is not standard C++ and normally only exist in C. GNU may happen to expose them 
as part of their GNU flavour of C++.

If you're using `-std=c++11` or such, try `-std=gnu++11`.

You could also try constructing a `pollfd` object before the statement and 
pushing that, or possibly passing `pollfd{.fd = _listen_socket, .events = 
POLLIN}`.

You're probably looking at some fun interaction of GNU extensions with the 
advent of initializer_list:s in C++11.

// Lars

From: easybuild-requ...@lists.ugent.be easybuild-requ...@lists.ugent.be 
on behalf of Kenneth Hoste kenneth.ho...@ugent.be
Sent: Saturday, February 23, 2019 16:43
To: easybuild@lists.ugent.be
Subject: Re: [easybuild] Assertion error for PyTorch-0.3.1 with intelcuda-2017b

On 22/02/2019 22:10, Vanzo, Davide wrote:
gt; Hello all!
gt;
gt; I am trying to build PyTorch-0.3.1 with intelcuda-2017b but I am 
hitting
gt; the error I reported below. Has anybody seen anything similar before?
gt;
gt; 
/tmp/PyTorch/0.3.1/intelcuda-2017b-Python-3.6.3/pytorch-0.3.1/torch/lib/THD/base/data_channels/Store.cpp(44):
gt; error: no instance of overloaded function "std::vectorlt;_Tp,
gt; _Allocgt;::push_back [with _Tp=pollfd, 
_Alloc=std::allocatorpollfd]"
gt; matches the argument list
gt;  argument types are: ({...})
gt;  object type is: std::vectorpollfd, 
std::allocatorpollfd=""gt;
gt;  fds.push_back({ .fd = _listen_socket, .events = POLLIN });
gt;  ^
gt; 
/opt/easybuild/software/Core/GCCcore/6.4.0/include/c++/6.4.0/bits/stl_vector.h(932):
gt; note: this candidate was rejected because arguments do not match
gt;  push_back(value_typeamp;amp; __x)
gt;  ^
gt; 
/opt/easybuild/software/Core/GCCcore/6.4.0/include/c++/6.4.0/bits/stl_vector.h(914):
gt; note: this candidate was rejected because arguments do not match
gt;  push_back(const value_typeamp; __x)
gt;  ^
gt; 
/tmp/PyTorch/0.3.1/intelcuda-2017b-Python-3.6.3/pytorch-0.3.1/torch/lib/THD/base/data_channels/Store.cpp(44):
gt; internal error: assertion failed at: "shared/cfe/edgcpfe/exprutil.c",
gt; line 747
gt;  fds.push_back({ .fd = _listen_socket, .events = POLLIN });
gt;  ^
gt; compilation aborted for
gt; 
/tmp/PyTorch/0.3.1/intelcuda-2017b-Python-3.6.3/pytorch-0.3.1/torch/lib/THD/base/data_channels/Store.cpp
gt; (code 4)


Can you share the easyconfig file, so we can try to reproduce this?

It's probably the Intel compilers being a bit picky here, wouldn't be
the first time...

Looks like it may just need some convincing with a cast or something?

My C++ is too rusty to quickly figure this out though...

Maybe this helps:
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fr%2Fcpp_questions%2Fcomments%2F6cohwk%2Fno_instance_of_overloaded_function_vector_problem%2Famp;data=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C54ba0b2837ec49b51b4e08d699d066af%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636865517700155185amp;sdata=ry7KfirvByw9Q6JhqwByAllUBn%2BGCFQekZV2kDCLZUs%3Damp;reserved=0
.


regards,

Kenneth
/pollfd,/pollfd/kenneth.ho...@ugent.be/easybuild-requ...@lists.ugent.be

[easybuild] Assertion error for PyTorch-0.3.1 with intelcuda-2017b

[Vanzo, Davide]

Hello all!

I am trying to build PyTorch-0.3.1 with intelcuda-2017b but I am hitting the 
error I reported below. Has anybody seen anything similar before?


/tmp/PyTorch/0.3.1/intelcuda-2017b-Python-3.6.3/pytorch-0.3.1/torch/lib/THD/base/data_channels/Store.cpp(44):
 error: no instance of overloaded function "std::vector<_Tp, _Alloc>::push_back 
[with _Tp=pollfd, _Alloc=std::allocator]" matches the argument list
argument types are: ({...})
object type is: std::vector>
fds.push_back({ .fd = _listen_socket, .events = POLLIN });
^
/opt/easybuild/software/Core/GCCcore/6.4.0/include/c++/6.4.0/bits/stl_vector.h(932):
 note: this candidate was rejected because arguments do not match
push_back(value_type&& __x)
^
/opt/easybuild/software/Core/GCCcore/6.4.0/include/c++/6.4.0/bits/stl_vector.h(914):
 note: this candidate was rejected because arguments do not match
push_back(const value_type& __x)
^
/tmp/PyTorch/0.3.1/intelcuda-2017b-Python-3.6.3/pytorch-0.3.1/torch/lib/THD/base/data_channels/Store.cpp(44):
 internal error: assertion failed at: "shared/cfe/edgcpfe/exprutil.c", line 747
fds.push_back({ .fd = _listen_socket, .events = POLLIN });
^
compilation aborted for 
/tmp/PyTorch/0.3.1/intelcuda-2017b-Python-3.6.3/pytorch-0.3.1/torch/lib/THD/base/data_channels/Store.cpp
 (code 4)


--
Davide Vanzo, PhD
Sr. HPC Research Scientist
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre

RE: [easybuild] foss-2016b/foss-2018b

[Vanzo, Davide]

Hi Thomas,

I do not think there is such a feature available in any module management 
system, but people that have more experience with flat module naming schemes 
may give you better insights than me.

We went with a hierarchical module naming scheme for the exact reason of 
avoiding that type of issues.

--
Davide Vanzo, PhD
Application Developer
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre


On 2018-12-12 08:56:04-06:00 easybuild-requ...@lists.ugent.be wrote:

Hi

We have installed a lot of modules with the foss-2016b toolchain on our HPC
cluster.
And there are a several modules installed with the foss-2018b toolchain.

But some endusers have problems to load some modules if a module is installed
with the foss-2016b and foss-2018b toolchain.

use-case: an user has loaded the module BCFtools-1.8/foss-2016b. but then he
wants to load the Python module which is installed with the foss-2016b and
foss-2018b toolchain. If he executes' module load Python', then it
automatically loads the module with the foss-2018b toolchain.

Is there a way to provide it automatically loads the module with foss-2016b if
there are already modules loaded with the foss-2016b toolchain?

Thomas Eylenbosch
Ext: Gluo N.V.

BASF Agricultural Solutions Belgium NV
Technologiepark 38
B-9052 Ghent (Zwijnaarde)
BELGIUM
E-mail:  thomas.eylenbosch@agro.basf-se.com

[easybuild] EasyBeer and dinner @ SC18

[Vanzo, Davide]

If you are at SC18 and would like to join us for a beer and some dinner, please 
let me know by replying to this email.

Time: 6:30 pm

Location:

The Crafty Irishman Public House
1800 Main St, Dallas, TX 75201
(972) 707-7589
https://goo.gl/maps/S8ws56kGArN2

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] Next EasyBuild conf call: Wed Oct 31st, 5pm CEST

[Vanzo, Davide]

Dear EasyBuilders,

Notes on today's conf call are available at
https://github.com/easybuilders/easybuild/wiki/Conference-call-notes-20181031

Info regarding the Next conf call will arrive as soon as our BDFL will plan our 
lives.

Best regards
--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre


On 2018-10-30 12:01:47-05:00 easybuild-requ...@lists.ugent.be wrote:

Dear EasyBuilders,
The 113th EasyBuild conf call is planned for tomorrow, Wed 31 Oct 2018, at 
17:00 CEST, see
https://plus.google.com/events/c6fgrmjuqigogsq5fghdn2cit48

Current agenda:
* Q
Suggestions for additional topics are welcome.
More information on the EasyBuild conf calls is available at
https://github.com/easybuilders/easybuild/wiki/Conference-calls .
Best regards

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre

[easybuild] Next EasyBuild conf call: Wed Oct 31st, 5pm CEST

[Vanzo, Davide]

Dear EasyBuilders,
The 113th EasyBuild conf call is planned for tomorrow, Wed 31 Oct 2018, at 
17:00 CEST, see
https://plus.google.com/events/c6fgrmjuqigogsq5fghdn2cit48

Current agenda:
* Q
Suggestions for additional topics are welcome.
More information on the EasyBuild conf calls is available at
https://github.com/easybuilders/easybuild/wiki/Conference-calls .
Best regards

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre

[easybuild] RE: Python-3.7.0 No module named '_ssl'

[Vanzo, Davide]

John,

Could you please attach the full log?

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre


On 2018-07-13 07:53:56-05:00 easybuild-requ...@lists.ugent.be wrote:
Using EasyBuild 3.6.0 and Python-3.7.0 easyconfig from github. All modules 
build without issue but sanity check fails.

== FAILED: Installation ended unsuccessfully (build directory: 
/app/easybuild/build/Python/3.7.0/foss-2016b):build failed (first 300 chars): 
Sanity check failed: sanity check command python -c 'import _ssl' exited 
withcode 1 (output: Traceback (most recent call last):
  File "", line 1, in 
ModuleNotFoundError: No module named '_ssl'
)


--
John Dey
HPC Operations
Scientific Computing
O 206.667.4308
M 360.649.2731
E jf...@fredhutch.org

[/Users/john/Library/Containers/com.microsoft.Outlook/Data/Library/Caches/Signatures/signature_1878591012]
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. 
N.,
 Mail Stop J3-516
Seattle, WA 98109
fredhutch.org

Re: [easybuild] Next EasyBuild conf call: Wed July 11th 2018, 5pm CEST

[Vanzo, Davide]

Dear EasyBuilders,

If you missed today's conf call, here you can find some notes:
https://github.com/easybuilders/easybuild/wiki/Conference-call-notes-20180711

The next conf call is scheduled for Wed July 25th 2018, see
https://plus.google.com/events/cpq2nqo8spt1u9uemvjkhgat9t8

Best regards

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre


On 2018-07-10 10:31:42-05:00 easybuild-requ...@lists.ugent.be wrote:

Dear EasyBuilders,

The next EasyBuild conf call is planned for tomorrow, Wed July 11th 2018, at 
5pm CEST, see also
https://plus.google.com/events/ctbcs6ocbe2gq8f55kslbqfs87g

Current agenda:
* Upcoming EasyBuild 3.6.2 release
* Q
As usual, suggestions for additional topics are welcome.
More information on the EasyBuild conf calls is available at
https://github.com/easybuilders/easybuild/wiki/Conference-calls

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre

[easybuild] Next EasyBuild conf call: Wed July 11th 2018, 5pm CEST

[Vanzo, Davide]

Dear EasyBuilders,

The next EasyBuild conf call is planned for tomorrow, Wed July 11th 2018, at 
5pm CEST, see also
https://plus.google.com/events/ctbcs6ocbe2gq8f55kslbqfs87g

Current agenda:
* Upcoming EasyBuild 3.6.2 release
* Q
As usual, suggestions for additional topics are welcome.
More information on the EasyBuild conf calls is available at
https://github.com/easybuilders/easybuild/wiki/Conference-calls

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre

RE: Re: [easybuild] Minimal vs full toolchain, Qt, CUDA etc.

[Vanzo, Davide]

Maik,

I double Jack's explanation on why QT5 is built at the highest toolchain level. 
Similarly in our stack we moved Python to the GCC/iccifort level by removing 
the packages that depend on MPI/FFTW and install Qt5 with the GCC/iccofort 
toolchains.
Moving Python to a lower toolchain in the official easyconfig repo is something 
that we have been discussing for a while and we are making steps in that 
directions. In the meantime nothing stops you to do are we and Jack did.

As for CUDA, the use of toolchains like `fosscuda` and `intelcuda` allow to 
avoid the need of adding version suffixes for MPI and all other software built 
with CUDA support. In our case we prefer to avoid version suffixes as much as 
we can since they introduce unnecessary complication for our users. Obviously 
this is possible only if you are using hierarchical modules.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre


On 2018-06-20 10:19:58-05:00 easybuild-requ...@lists.ugent.be wrote:

[oops meant to send to the list]


 Forwarded Message 
Subject:Re: [easybuild] Minimal vs full toolchain, Qt, CUDA etc.
Date:   Wed, 20 Jun 2018 09:56:06 -0500
From:   Jack Perdue lt;j-per...@tamu.edugt;
To: Maik Schmidt lt;maik.schm...@tu-dresden.degt;



Howdy Maik,

Here we use a stripped down Python as a builddependency to build:

 amp;nbsp; Qt5-5.10.1-GCCcore-6.4.0-Python-2.7.14-bare.eb
and
 amp;nbsp; Qt5-5.10.1-GCCcore-6.4.0-Python-3.6.3-bare.eb

By default, the full Python needs MPI/maths (for numpy) so
if you build Qt with the regular Python you have to promote
the toolchain.amp;nbsp; Python-bare just provides the basics.

Works fairly well.

There are other initiatives in EB to clean this up
using Python-core and the like.amp;nbsp; But this is what
we've been using for now.

As for CUDA I wondered the same the answer was that
OpenMPI has hooks for CUDA so if you include CUDA early
in the toolchain (while building OpenMPI) then you get
some MPI-enabled CUDA.amp;nbsp; I/we haven't much experience
with that yet (curious to see what TensorFlow can do) but
that's the reasoning for that (though I do wish it was easier
like you suggest)

As ever, there are examples at:

 amp;nbsp; 
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.siliconslick.com%2Feasybuild%2Feasyconfigs%2Famp;amp;data=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C969e7c72f65249da661e08d5d6c145e8%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636651047945281105amp;amp;sdata=uZgbznCATjG8G%2BC6o5Pxkvz1UtS2qIJ8NiLSqYVn9Ys%3Damp;amp;reserved=0

See ada and terra.

Jack Perdue
Lead Systems Administrator
High Performance Research Computing
TAMU Division of Research
j-per...@tamu.edu
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhprc.tamu.eduamp;amp;data=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C969e7c72f65249da661e08d5d6c145e8%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636651047945281105amp;amp;sdata=7DXnIKRWuF6BXcxiI1YSxFxdv7M8W8Gdb116krNrxUQ%3Damp;amp;reserved=0
HPRC Helpdesk: h...@hprc.tamu.edu

On 06/20/2018 09:21 AM, Maik Schmidt wrote:
amp;gt; Hi, I've been asked by one of our users why Qt5 is built with the 
full
amp;gt; toolchain (foss or intel) even though it does not really use MPI or
amp;gt; MKL for that matter. I've looked at the dependencies of Qt itself 
and
amp;gt; apparently he is correct, why is this not GCCcore? There's no 
reason
amp;gt; to use the full toolchain here, right?
amp;gt;
amp;gt; On another note, what has been the reasoning behind introducing an
amp;gt; entire new toolchain only to add CUDA? it really makes not much 
sense
amp;gt; to me, because then I have to build a lot of duplicate software 
that
amp;gt; doesn't even need CUDA just to support this toolchain (foss vs
amp;gt; fosscuda)... e.g. why would I need a HDF5 -fosscuda if it is 
exactly
amp;gt; the same as the -foss version?
amp;gt;
amp;gt; The solution with just adding a versionsuffix and CUDA as a 
dependency
amp;gt; to software requiring it seemed much cleaner to me, but maybe I'm
amp;gt; missing something here...
amp;gt;
amp;gt; Thanks for your input,
amp;gt;
amp;gt; Maik
amp;gt;

lt;/maik.schm...@tu-dresden.degt;lt;/j-per...@tamu.edugt;

RE: [easybuild] Packages lacking intel fortran runtime

[Vanzo, Davide]

Hi Joachim!

Yes, we have encountered the same issue you described before. Our way of 
dealing with it is to expose to the users only the iccifort toolchain module, 
while icc and ifort are hidden.

Definitely not the most elegant solution if you still want to let the users 
select icc or ifort separately, but we do not have that need anyway.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre


On 2018-04-12 08:22:32-05:00 easybuild-requ...@lists.ugent.be wrote:

Hi,

There is a longstanding issue on our site regarding the intel fortran runtime 
for some packages.  I assume other sites have the same, but am not sure it was 
ever discussed here.

We are using hierarchical modules in production.  If a package is build with an 
intel toolchain, you can load it by either loading the

  icc, impi

or

 ifort, impi

component of that toolchain.  If a package (e.g. 
Caffe-1.0-intel-2017a-Python-2.7.13.eb) requires the Fortran runtime but it is 
loaded via the icc route, you get runtime errors because of the missing 
libifport.so.

A possible fix could be to load either of the ifort or intel modules inside the 
package (e.g. Caffe) module.  I assume that would need to be done in the 
easy-block or further down inside the easybuild framework.

Any comments or insights?  Apologies if that was discussed before.

Best wishes
   Joachim

[easybuild] Build Octave against parallel HDF5 library

[Vanzo, Davide]

Hello EasyBuilders!

Has anybody successfully managed to build Octave against HDF5 compiled with MPI?
It is well known that this causes issues since the compiler cannot find the MPI 
headers (Octave does not care if HDF5 is serial or not). For this reason the 
typical solution is to build against the serial HDF5. However I would much 
prefer having only one build of HDF5.

If you are there, make a noise!

Thanks!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

RE: [easybuild] Issue with building numpy with intel-2016b

[Vanzo, Davide]

Yup, that was it again.

Thanks Kenneth!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On 2018-01-05 09:10:32-06:00 easybuild-requ...@lists.ugent.be wrote:

Hi Davide,

On 04/01/2018 22:36, Vanzo, Davide wrote:
Hello EasyBuilders!

Background: at our site we install Python (2.x.y and 3.x.y) without numpy so 
that we can bump it down from foss to GCC and from intel to iccifort. Then 
numpy is built as a separate module with the foss and intel toolchains. Same 
for scipy, mpi4py and pandas.

For some reason that I cannot understand I am getting the error below when I 
try to install numpy with the intel-2016b toolchain. If I try to run the same 
python command manually, it works fine. Any idea?

Attached you can find the full log with debug and the easyconfig file I am 
using.

The "exit 11" points in the direction of a segmentation fault, so did you 
properly replace libintlc.so.5 in the icc, ifort *and* imkl installations 
included in the intel/2016b toolchain, as discussed in a previous post (cfr. 
https://lists.ugent.be/wws/arc/easybuild/2018-01/msg00012.html<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.ugent.be%2Fwws%2Farc%2Feasybuild%2F2018-01%2Fmsg00012.html=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C2b92d86e44b747007fc208d5544e659b%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636507618316711814=09rnuxhxP8Zyx9SxOLq8aDkOiLGONHe7AqZY4flUaVk%3D=0>)?

Maybe the problem only occurs with numpy because that needs imkl while your 
Python build doesn't since it was built with iccifort?


regards,

Kenneth


Thanks!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.accre.vanderbilt.edu=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C2b92d86e44b747007fc208d5544e659b%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636507618316711814=LCsXbLOwy0mkEOcpyK97xFYxefkslQiuk8mY8xzxaNQ%3D=0>

RE: [easybuild] Python-2.7.12-iccifort-2016.3.210-GCC-5.4.0-2.26 build segfault

[Vanzo, Davide]

Kenneth,

That did the trick.
Thank you!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On 2018-01-02 14:34:58-06:00 easybuild-requ...@lists.ugent.be wrote:

Hi Davide,

On 02/01/2018 21:29, Vanzo, Davide wrote:
Hello EasyBuilders!

I am trying to build Python-2.7.12-iccifort-2016.3.210-GCC-5.4.0-2.26.eb (from 
the intel-2016b easyconfig file I have simply removed the python packages that 
depend on MPI and MKL) on CentOS 7.2.1511 and for some reason it fails with the 
error below. I have never seen this error when I built it under CentOS 6.x.

Does it ring any bell to any of you?

This sounds a lot like the issue we encountered in 
https://github.com/easybuilders/easybuild-easyconfigs/issues/3646<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feasybuilders%2Feasybuild-easyconfigs%2Fissues%2F3646=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C60492f13dc124c9b145508d552203a60%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636505220979932081=F7lWMjqwLJ1n6n1mXsiXt%2B37t2DV4VGIzY87FI5zwqw%3D=0>,
which basically boils down to this version of the Intel compilers not being 
compatible with CentOS 7.2.

The official (!) fix is to copy libintlc.so.5 from a more recent version of the 
Intel compilers, see also 
https://bugzilla.redhat.com/show_bug.cgi?id=1377895<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D1377895=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C60492f13dc124c9b145508d552203a60%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636505220979932081=D6yK6055NdMSdv7paJpQIMxRNc1NjbupWsOskRAr6pM%3D=0>
 and 
https://software.intel.com/en-us/articles/intel-compiler-version-16-not-compatible-with-recent-libcso6<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsoftware.intel.com%2Fen-us%2Farticles%2Fintel-compiler-version-16-not-compatible-with-recent-libcso6=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C60492f13dc124c9b145508d552203a60%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636505220979932081=ECsUSf3or8X%2BQh17hgDSGOxMJeYK7TCrGD21vOvu7Bw%3D=0>
 .


regards,

Kenneth

Attached you can find the full log.

Thanks!



Modules/posixmodule.o: In function `posix_tmpnam':
/mnt/ramdisk/Python/2.7.12/iccifort-2016.3.210/Python-2.7.12/./Modules/posixmodule.c:7631:
 warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
Modules/posixmodule.o: In function `posix_tempnam':
posixmodule.c:(.text+0x4fba): warning: the use of `tempnam' is dangerous, 
better use `mkstemp'
icc -L/accre/arch/easybuild/software/Core/icc/2016.3.210/lib/intel64 
-L/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/bzip2/1.0.6/lib 
-L/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/zlib/1.2.8/li\
b -L/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/libreadline/6.3/lib 
-L/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/ncurses/6.0/lib 
-L/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/SQLi\
te/3.13.0/lib 
-L/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/Tk/8.6.5/lib 
-L/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/GMP/6.1.1/lib 
-L/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/l\
ibffi/3.2.1/lib64 
-L/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/libffi/3.2.1/lib 
-Xlinker -export-dynamic -o python \
Modules/python.o \
-L. -lpython2.7 -ldl -liomp5 -lpthread -lutil 
/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/libreadline/6.3/lib/libreadline.a
 /accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/ncurse\
s/6.0/lib/libncurses.a   -lm
LD_LIBRARY_PATH=/mnt/ramdisk/Python/2.7.12/iccifort-2016.3.210/Python-2.7.12:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/libffi/3.2.1/lib64:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/libffi\
/3.2.1/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/GMP/6.1.1/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/Tk/8.6.5/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/X11/201608\
19/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/fontconfig/2.12.1/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/expat/2.2.0/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/fre\
etype/2.6.5/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/libpng/1.6.24/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/SQLite/3.13.0/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.\
4.0/Tcl/8.6.5/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/libreadline/6.3/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/ncurses/6.0/lib:/accre/arch/easybuild/software/Compiler/GCCcore/\
5.4.0/zlib/1.2.8/lib:/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/bzip2/1.0.6/lib:/accre/arch/easybuild/software/Core/ifort/2016.3.210/compilers_and_li

RE: [easybuild] OpenMPI 1.10.3 - mpifort wrong search path for libgfortran.so

[Vanzo, Davide]

Kenneth,

Yes, the correct path is in both LD_LIBRARY_PATH and in LIBRARY_PATH.

Attached you can find the outputs of

$ LD_DEBUG=all mpifort multitask_mpi.f90

on two different servers that have the same minimal OS but one works and the 
other fails.

The only difference is the CPU architecture but that should not be a concern. 
Also I have used the same easyconfigs. The easyblocks in the working server 
come from the develop branch, while on the failing server they are from 3.4.1

The strange thing is that in the failing system it looks for a symbol 
(vfprintf) that is not even mentioned in the working system. But that's all I 
can see from that output.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On 2017-11-20 13:15:14-06:00 Kenneth Hoste wrote:

Hi Davide,

On 20/11/2017 16:06, Vanzo, Davide wrote:
Hello EasyBuilders!

I have started building the foss-2016b toolchain on a new server and I am 
stumbling against an error that I have never seen before and for which I cannot 
pinpoint the cause.

When I try to compile a fortran source with mpifort (or mpif90), I get the 
error you see below. The thing I do not understand is why the linker searches 
for /usr/lib64/libgfortran.so (which does not exist for obvious reasons) 
instead of the equivalent library built by EasyBuild with GCCcore-5.4.0 (which 
exists).
The LD_LIBRARY_PATH env var contains the path to the GCCcore directory where 
libgfortran.so is actually located. Why is ld ignoring this path?
Any suggestion?

I think ld(.gold) actually checks $LIBRARY_PATH when linking binaries, while 
$LD_LIBRARY_PATH is checked at runtime.

But I assume $LIBRARY_PATH also includes the location to libgfortran.so provide 
by GCCcore?

Do you get any useful output when running "LD_DEBUG=1 mpifort 
multitask_mpi.f90" (see 
http://www.bnikolic.co.uk/blog/linux-ld-debug.html<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.bnikolic.co.uk%2Fblog%2Flinux-ld-debug.html=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C8d702dfec3f9483274df08d5304b0597%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636468021133946943=eGc9RbcsbQlbsxg0mNsrmzxhibtv7ZznYIL3hq%2FsoNw%3D=0>)?


regards,

Kenneth

RE: [easybuild] OpenMPI 1.10.3 - mpifort wrong search path for libgfortran.so

[Vanzo, Davide]

John,

Thank you for your feedback but your point is exactly what I meant when I said 
that the /usr/lib64/libgfortran.so does not exist for obvious reasons.
I am working on a minimal CentOS 7.2 installation (hence no gfortran installed 
with the OS) and that is why I cannot understand why it still searches for it 
in the system path...

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On 2017-11-20 13:19:42-06:00 John Dey wrote:

The system you used to built foss-2016b on,  probably had gcc FORTRAN 
installed.  The local package was found by pkg_config or LD_LIBRARY_PATH and 
that is how your foss-2016b is now configured.  Its super important to build 
Easyconfig on systems that are as stripped down as possible otherwise system 
libraries will pollute your modules.

[easybuild] OpenMPI 1.10.3 - mpifort wrong search path for libgfortran.so

[Vanzo, Davide]

Hello EasyBuilders!

I have started building the foss-2016b toolchain on a new server and I am 
stumbling against an error that I have never seen before and for which I cannot 
pinpoint the cause.

When I try to compile a fortran source with mpifort (or mpif90), I get the 
error you see below. The thing I do not understand is why the linker searches 
for /usr/lib64/libgfortran.so (which does not exist for obvious reasons) 
instead of the equivalent library built by EasyBuild with GCCcore-5.4.0 (which 
exists).
The LD_LIBRARY_PATH env var contains the path to the GCCcore directory where 
libgfortran.so is actually located. Why is ld ignoring this path?

Any suggestion?

Thanks!



$ mpifort multitask_mpi.f90
/accre/arch/easybuild/software/Compiler/GCCcore/5.4.0/binutils/2.26/bin/ld.gold:
 error: cannot open /usr/lib64/libgfortran.so: No such file or directory
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_date_and_time'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_st_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_transfer_integer_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_string_trim'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_transfer_character_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_transfer_integer_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_transfer_integer_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_transfer_character_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_transfer_integer_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_transfer_character_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_transfer_character_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_string_trim'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function timestamp_: error: undefined 
reference to '_gfortran_st_write_done'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function p0_receive_output_: error: undefined 
reference to '_gfortran_st_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function p0_receive_output_: error: undefined 
reference to '_gfortran_st_write_done'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function p0_receive_output_: error: undefined 
reference to '_gfortran_st_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function p0_receive_output_: error: undefined 
reference to '_gfortran_st_write_done'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function p0_receive_output_: error: undefined 
reference to '_gfortran_st_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function p0_receive_output_: error: undefined 
reference to '_gfortran_st_write_done'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function MAIN__: error: undefined reference 
to '_gfortran_stop_string'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function MAIN__: error: undefined reference 
to '_gfortran_transfer_real_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function MAIN__: error: undefined reference 
to '_gfortran_stop_string'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function MAIN__: error: undefined reference 
to '_gfortran_transfer_real_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function MAIN__: error: undefined reference 
to '_gfortran_stop_string'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function MAIN__: error: undefined reference 
to '_gfortran_transfer_real_write'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function MAIN__: error: undefined reference 
to '_gfortran_stop_string'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function main: error: undefined reference to 
'_gfortran_set_args'
/tmp/ccpSxqE6.o:multitask_mpi.f90:function main: error: undefined reference to 
'_gfortran_set_options'
collect2: error: ld returned 1 exit status


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

RE: [easybuild] RE: EasyBuild at SC17

[Vanzo, Davide]

Markus,

I have booked a table and I have added a few more seats than the ones that 
participated in the poll to accommodate extra guests. We will be ok up to 15 
people.

In any case, I would appreciate to know if there are other guests in advance so 
that in case I can always give a call to the brewery and add one more table.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On 2017-11-11 13:19:50-06:00 Markus Geimer wrote:

Once again with Tim's new email address in CC...

Markus

On 11/11/2017 07:49 PM, Markus Geimer wrote:
gt; Hi Davide,
gt;
gt;gt; *Wednesday, Nov 15, 7:00pm - /EasyBuild Meet and Greet/*
gt;gt; /Rock Bottom Brewery - 1001 16th Street/
gt;gt; 
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgoo.gl%2Fmaps%2FRC2LFnwvBwyamp;data=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C3b6ee90259cc4a34cbc508d529392d16%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636460247898009117amp;sdata=Ksv4zeOxXxrGKojM8OgJq1mn4KFlwnVOXlynWmP1aM8%3Damp;reserved=0
gt;gt;
gt;gt; Share a pint of beer and plenty of food in honor of all the 
EasyBuild
gt;gt; community and our beloved BDFL Kenneth.
gt;
gt; I assume you reserved a table?  Did you plan for an exact match of
gt; people with the doodle poll?  Tim Browne (CC) and I may potentially
gt; be interested in joining as well.  However, at least I can only
gt; tell you at relatively short notice...
gt;
gt; I'll pass by on Monday at your booth for the EasyCake ;-)  Maybe I
gt; already know more at that time.
gt;
gt; CU,
gt; Markus
gt;

--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone:  +49-2461-61-1773
Fax:+49-2461-61-6656
E-Mail: m.gei...@fz-juelich.de
WWW:
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.fz-juelich.de%2Fjscamp;data=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C3b6ee90259cc4a34cbc508d529392d16%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636460247898009117amp;sdata=qvbxBfEqsg03jJ7jHjk3vvPSDFhtMU2uUUy5LZ5WUBM%3Damp;reserved=0




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

[easybuild] RE: EasyBuild at SC17

[Vanzo, Davide]

Good morning EasyBuilders!

So for those of you that are in Denver, here is the list of EasyBuild events at 
SC17.



Monday, Nov 13, 8:00pm - Happy birthday Easybuild!
Vanderbilt University, booth #463

Let's get together to celebrate on the exact day of the first release of 
EasyBuild 5 years ago. Come and grab some EasyCake, which will be the best cake 
ever released!



Tuesday-Wednedsay, Nov 14-15, 2:00pm - EasyBuild, building software with ease
Vanderbilt University, booth #463

EasyBuild presentations. I would strongly appreciate if you would like to stop 
by and provide your experience and help answering questions.



Wednesday, Nov 15, 7:00pm - EasyBuild Meet and Greet
Rock Bottom Brewery - 1001 16th Street
https://goo.gl/maps/RC2LFnwvBwy

Share a pint of beer and plenty of food in honor of all the EasyBuild community 
and our beloved BDFL Kenneth.



And again, feel free to contact me directly through Slack at any time (vanzod).

See you soon!


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On 2017-11-10 10:47:41-06:00 Vanzo, Davide wrote:

Howdy EasyBuilders!

Do not forget to sign up for our EasyBuild Meet and Greet event at SC17!

https://doodle.com/poll/qxs4c2rv6akb7644

The final date and time of the event will be selected tomorrow based on your 
availability.

Happy building!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On 2017-11-07 09:31:28-06:00 Vanzo, Davide wrote:

Hello EasyBuilders!

As you may know, the SC17 conference in Denver, CO is ready to start next week.

EasyBuild will also be present with some presentations that I gladly offered to 
host at our Vanderbilt University stand (#463). Please come by to share your 
EasyThoughts, grab a sticker or just to say hello.

I would also like to organize an EasyBuild Meet and Greet event at Freshcraft 
(https://goo.gl/maps/KDPeVs7tHby) one of those nights to have a beer in honor 
of our dear DBFL Kenneth that unfortunately will not be among us. Please let me 
know your availability here:

https://doodle.com/poll/qxs4c2rv6akb7644

I will follow up once a date and time will be picked.

Please do not hesitate to contact me directly through the EasyBuild Slack team. 
My username is vanzod.

See you soon!


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

[easybuild] RE: EasyBuild at SC17

[Vanzo, Davide]

Howdy EasyBuilders!

Do not forget to sign up for our EasyBuild Meet and Greet event at SC17!

https://doodle.com/poll/qxs4c2rv6akb7644

The final date and time of the event will be selected tomorrow based on your 
availability.

Happy building!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On 2017-11-07 09:31:28-06:00 Vanzo, Davide wrote:

Hello EasyBuilders!

As you may know, the SC17 conference in Denver, CO is ready to start next week.

EasyBuild will also be present with some presentations that I gladly offered to 
host at our Vanderbilt University stand (#463). Please come by to share your 
EasyThoughts, grab a sticker or just to say hello.

I would also like to organize an EasyBuild Meet and Greet event at Freshcraft 
(https://goo.gl/maps/KDPeVs7tHby) one of those nights to have a beer in honor 
of our dear DBFL Kenneth that unfortunately will not be among us. Please let me 
know your availability here:

https://doodle.com/poll/qxs4c2rv6akb7644

I will follow up once a date and time will be picked.

Please do not hesitate to contact me directly through the EasyBuild Slack team. 
My username is vanzod.

See you soon!


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

[easybuild] EasyBuild at SC17

[Vanzo, Davide]

Hello EasyBuilders!

As you may know, the SC17 conference in Denver, CO is ready to start next week.

EasyBuild will also be present with some presentations that I gladly offered to 
host at our Vanderbilt University stand (#463). Please come by to share your 
EasyThoughts, grab a sticker or just to say hello.

I would also like to organize an EasyBuild Meet and Greet event at Freshcraft 
(https://goo.gl/maps/KDPeVs7tHby) one of those nights to have a beer in honor 
of our dear DBFL Kenneth that unfortunately will not be among us. Please let me 
know your availability here:

https://doodle.com/poll/qxs4c2rv6akb7644

I will follow up once a date and time will be picked.

Please do not hesitate to contact me directly through the EasyBuild Slack team. 
My username is vanzod.

See you soon!


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

[easybuild] RE: specific toolchain for CMake needed?

[Vanzo, Davide]

Hi Marteen,

Welcome to the club!

In our cluster we provide CMake at the GCCcore level. In this way we have a 
single install for a given version of the foss and intel toolchains, since they 
both share the GCCcore toolchain. I cannot foresee any significant issue in 
building it with the dummy toolchain but we prefer to limit as much as possible 
the software installed at that level.

One good reason for this is that, if you are using minimal toolchains,  EB does 
not look for available dependencies in the dummy toolchain by default. This can 
be overridden either in the dependency specification in the easyconfig or 
through the --add-dummy-to-minimal-toolchains option.

For all the other dependencies that provide libraries, we build them at the 
GCCcore level so that we have single install for both foss and intel toolchains 
since they are binary compatible. The only exception are libraries that may 
show different performances if built with a specific compiler (not all 
compilers optimize at the same way), like math libraries. In this case we 
prefer to build them with the GCC or iccifort toolchains.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On 2017-08-15 14:12:40-05:00 Maarten Bosmans wrote:

Hi List,



I'm trying to get to know EasyBuild. So far there's been lots of little 
surprises and frustrations, but as I'm slowly getting used to the whole system, 
it looks like a nice way to manage building (scientific) software from source.



A couple of things I still do not fully understand. One of them is why build 
tools like CMake and M4 are recompiled for each toolchain. In most easyconfigs 
where CMake is in the builddependencies it is specified without toolchain. So 
it seems that the installation of toolchain-specific CMakes is mainly because 
the limited availability of CMake easyconfigs using the dummy toolchain. Am I 
missing something, or would it make more sense to make an easyconfig for each 
CMake version using the dummy toolchain and not for other toolchains?



The same thing goes for flex and bison, but these packages actually have a lib 
part, so for some use cases it could actually make sense to predicate their 
version on the toolchain.



Maarten Bosmans

[easybuild] Next Easybuild conf call: Wed August 2nd, 5pm CEST

[Vanzo, Davide]

Notes for today's conf call are available at 
https://github.com/easybuilders/easybuild/wiki/Conference-call-notes-20170802

The next conf call is planned for Wed August 16th, see also 
https://plus.google.com/u/0/events/crrvutfl9ifmnjb02v94d26g7qg  and our beloved 
BDFL should be back hosting.

Regards,

Not Kenneth nor Damian


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

RE: [easybuild] Errors with bootstrap_eb.py for easybuild 3.3.1

[Vanzo, Davide]

Clyde,

I stumbled into the same problem last Friday too and I am trying to figure out 
what is the source of the problem.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On 2017-07-21 17:25:40-05:00 Clyde Jones wrote:

Hi
 I've been trying to install easybuild using the bootstrap script and I am 
getting the following errors.  It looks like there is a configuration option 
that the script is expecting but is not getting set.  This fails on Centos7 
(7.3) and Ubuntu 16.04
What should I be setting?  This worked properly for 3.3.0
$ env | sort
APPS_PREFIX=/gstore/apps
BASH_ENV=/gstore/apps/lmod/lmod/init/bash
BASH_FUNC_ml()=() {  eval $($LMOD_DIR/ml_cmd "$@")
BASH_FUNC_module()=() {  eval $($LMOD_CMD bash "$@") && eval 
$(${LMOD_SETTARG_CMD:-:} -s sh)
EASYBUILD_PREFIX=/gstore/apps
EASYBUILD_ROBOT_PATHS=/home/gredsys/packages/
HOME=/home/gredsys
HOSTNAME=704ff3bc58b2
LESSOPEN=||/usr/bin/lesspipe.sh %s
LMOD_CMD=/gstore/apps/lmod/lmod/libexec/lmod
LMOD_DIR=/gstore/apps/lmod/lmod/libexec
LMOD_FULL_SETTARG_SUPPORT=no
LMOD_PKG=/gstore/apps/lmod/lmod
LMOD_SETTARG_CMD=:
LMOD_VERSION=7.5.15
LMOD_sys=Linux
LUAROCKS_PREFIX=/gstore/apps/luarocks/2.4.2
LUAROCKS_VER=2.4.2
LUA_CPATH=/gstore/apps/luarocks/2.4.2/lib/lua/5.1/?.so;;
LUA_PATH=/gstore/apps/luarocks/2.4.2/share/lua/5.1/?.lua;/gstore/apps/luarocks/2.4.2/share/lua/5.1/?/init.lua;;
LUA_VER=5.1.4
MANPATH=/gstore/apps/lmod/lmod/share/man::
MODULEPATH=/gstore/apps/modules::/gstore/apps/modulefiles:/gstore/apps/modulefiles/all:/gstore/apps/lmod/lmod/modulefiles/Core
MODULEPATH_ROOT=/gstore/apps/modulefiles
MODULESHOME=/gstore/apps/lmod/lmod
PATH=/tmp/tmpNM6MoY/bin:/tmp/tmpNM6MoY/lib:/tmp/tmpNM6MoY/lib64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/gstore/apps/luarocks/2.4.2/bin/:/gstore/apps/luarocks/2.4.2/lib/:/gstore/apps/luarocks/2.4.2/bin/:/gstore/apps/luarocks/2.4.2/lib/:/gstore/apps/luarocks/2.4.2/bin/:/gstore/apps/luarocks/2.4.2/lib/
PWD=/tmp/tmpNM6MoY/eb_stage1/lib/python2.7/site-packages
PYTHONPATH=/tmp/tmpNM6MoY/lib/python2.7/site-packages/:/tmp/tmpNM6MoY/lib/python2.7/site-packages/distribute-0.6.49-py2.7.egg:
SHLVL=1
TERM=xterm
USER=
_=/usr/bin/env
_ModuleTable001_=X01vZHVsZVRhYmxlXz17WyJNVHZlcnNpb24iXT0zLFsiY19yZWJ1aWxkVGltZSJdPTcyMDAsWyJjX3Nob3J0VGltZSJdPTAuMDA4NjM5ODEyNDY5NDgyNCxkZXB0aFQ9e30sZmFtaWx5PXt9LG1UPXt9LG1wYXRoQT17Ii9nc3RvcmUvYXBwcy9tb2R1bGVzIiwiIiwiL2dzdG9yZS9hcHBzL21vZHVsZWZpbGVzIiwiL2dzdG9yZS9hcHBzL21vZHVsZWZpbGVzL2FsbCIsIi9nc3RvcmUvYXBwcy9sbW9kL2xtb2QvbW9kdWxlZmlsZXMvQ29yZSIsfSxbInN5c3RlbUJhc2VNUEFUSCJdPSIvZ3N0b3JlL2FwcHMvbW9kdWxlczo6L2dzdG9yZS9hcHBzL21vZHVsZWZpbGVzOi9nc3RvcmUvYXBwcy9tb2R1bGVmaWxlcy9hbGw6L2dzdG9yZS9hcHBzL2xtb2QvbG1vZC9tb2R1bGVmaWxlcy9D
_ModuleTable002_=b3JlIix9
_ModuleTable_Sz_=2
}
}
[gredsys@704ff3bc58b2 tmp]$ python bootstrap_eb.py $APPS_PREFIX
[[INFO]] EasyBuild bootstrap script (version 20170706.01, MD5: 
e3595314c419ce935a5caaf70032801e)
[[INFO]] Found Python 2.7.5 (default, Nov  6 2016, 00:28:07) ; [GCC 4.8.5 
20150623 (Red Hat 4.8.5-11)]
[[INFO]] Installation prefix /gstore/apps
[[INFO]] Found module command '/gstore/apps/lmod/lmod/libexec/lmod' via 
$LMOD_CMD (Lmod), so using it.
[[INFO]] No suitable setuptools installation found, proceeding with stage 0...
[[INFO]] +++ STAGE 0: installing distribute via included (patched) 
distribute_setup.py...
Downloading 
https://easybuilders.github.io/easybuild/files/distribute-0.6.49-patched1.tar.gz
Extracting in /tmp/tmpoIozZ7
Now working in /tmp/tmpoIozZ7/distribute-0.6.49
Installing Distribute
[[INFO]] Installed setuptools version 0.6 
(/tmp/tmpNM6MoY/lib/python2.7/site-packages/distribute-0.6.49-py2.7.egg/setuptools/__init__.pyc)
[[INFO]] +++ STAGE 1: installing EasyBuild in temporary dir with easy_install...
[[INFO]] installing EasyBuild with 'easy_install --quiet --upgrade 
--prefix=/tmp/tmpNM6MoY/eb_stage1 easybuild'
[[INFO]] running post install command 'easy_install --upgrade 
--prefix=/tmp/tmpNM6MoY/eb_stage1 vsc-base'
[[INFO]] +++ STAGE 2: installing EasyBuild in /gstore/apps with EasyBuild from 
stage 1...
Traceback (most recent call last):
  File "bootstrap_eb.py", line 1014, in 
main()
  File "bootstrap_eb.py", line 817, in main
stage2(tmpdir, templates, install_path, distribute_egg_dir, sourcepath)
  File "bootstrap_eb.py", line 687, in stage2
easybuild_main()
  File 
"/tmp/tmpNM6MoY/eb_stage1/lib/python2.7/site-packages/easybuild_framework-3.3.1-py2.7.egg/easybuild/main.py",
 line 189, in main
eb_go = eboptions.parse_options(args=args)
  File 
"/tmp/tmpNM6MoY/eb_stage1/lib/python2.7/site-packages/easybuild_framework-3.3.1-py2.7.egg/easybuild/tools/options.py",
 line 1061, in parse_options
raise EasyBuildError("Failed to parse configuration options: %s" % err)
easybuild.tools.build_log.EasyBuildError: "Failed to parse 

Re: [easybuild] Gromacs GPU and view a complete log

[Vanzo, Davide]

Javier,

Yes, I also have the libnvidia-ml.so library in /usr/lib64/nvidia. The patch 
seem to point to a custom path. It would probably be worth hearing Ake.

Usually codes that use CUDA can be compiled in a node that does not have GPUs, 
as long as CUDA is available for linking. In this case though, since it needs 
the NVML libraries which are shipped with the driver, I suspect you will have 
to compile on the GPU node since the drivers should not be installed without 
any GPU.


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Jul 12 2017, at 3:23 pm, Javier Antonio Ruiz Bosch  wrote:

Hi, easybuilders.



My HPC has a GPU node that consists of 2 Nvidia K80 GPU cards. I recently need 
to install GROMACS to run it on that GPU node. I searched a few days ago and I 
did not found easyconfig of GROMACS for GPU so I prepared one using the 
toolchain foss and as a dependency I added cuda. Four days ago, akesandgren 
(github) added an easyconfig of GROMACS for GPU using the toolchain goolfc, so 
can I also install it using that easyconfig perhaps making some changes to the. 
patch file because my nvidia libraries are in user/lib64/nvidia not in 
user/lib/nvidia-367 not in user/lib/nvidia-375?



If at the time of starting the installation with easybuild should I do it from 
the server master or from the GPU node so the installation process can detects 
the GPU cards and use their drivers and library that are only in this GPU node. 
Or there is another way to do it.



Another question is how can I see the outputs of the cmake at the configuration 
step even when all is fine, it is only written to the log when the installation 
fails, in case it doesn’t fail this output is not written to the log.



Regards.

Javier.

Re: [easybuild] Local user intstallation

[Vanzo, Davide]

So, I gave it a try and it works fine.

The only annoyance (since it is more a matter of style) is that once I add the 
$EB_GLOBAL_PREFIX/modules/all to the $MODULEPATH, Lmod output for "module 
avail" gets pretty crowded.

--
Davide Vanzo, PhD
Application Developer
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Jun 27 2017, at 10:43 am, Vanzo, Davide <davide.va...@vanderbilt.edu> wrote:
Thank you all for the great suggestions!

DV


On Jun 27 2017, at 3:14 am, Miguel Dias Costa <migueldiasco...@gmail.com> wrote:


On 27/06/17 15:42, Alan O'Cais wrote:
That is a neat trick with the sourcepaths that I will try out today.

I have to credit boegel, he told me about it, I had simply assumed that would 
not work :)


I think there is a bit more to a user install space than what's been mentioned 
when you have a HMNS, see 
https://github.com/hpcugent/easybuild-framework/issues/2143

ah... because there are then two roots to the hierarchy, the global one and the 
local one?

Miguel


On 27 June 2017 at 03:09, Fotis Georgatos 
<fo...@mail.cern.ch<mailto:fo...@mail.cern.ch>> wrote:

Hi Miguel,

Yeap! i’d also fully applause this approach, it is very similar to what we’ve 
been doing since 2012!
- $MODULEPATH can include both global and local builds, you just need to decide 
their exact order
  * fyi. recent versions of Lmod have in ./contrib a pair of use.own.eb 
modulefiles, implementing the above
- the global install, is nothing more than by yet another user, which just 
happens to be steered by admin
- i also encourage people to add a versionsuffix with their initials and maybe 
a date, to keep things apart

The trick with EASYBUILD_SOURCEPATH is quite nice, it could also be used within 
use.own.eb/* modulefiles,
assuming that it is always *prepend*, as you described.

F.

On Jun 27, 2017, at 1:57 AM, Miguel Costa 
<migueldiasco...@gmail.com<mailto:migueldiasco...@gmail.com>> wrote:
> Hello Davide,
>
> I also had doubts initially because of what you mention and also using global 
> source paths, but this works for us:
>
> - create and manage a cluster-wide software stack with an easybuild user, 
> using EASYBUILD_PREFIX=$EB_GLOBAL_PREFIX
>
> - for normal users
>
>   - set EASYBUILD_PREFIX=$HOME/.local/easybuild but add 
> $EB_GLOBAL_PREFIX/modules/all to the $MODULEPATH (so that it finds the 
> already globally installed software)
>
>   - set 
> EASYBUILD_SOURCEPATH=$HOME/.local/easybuild/sources:$EB_GLOBAL_PREFIX/sources,
>  in this order (so that it downloads to local but looks in global). This was 
> recently clarified in the documentation, last line in 
> https://easybuild.readthedocs.io/en/latest/Configuration.html?highlight=sourcepath#sourcepath
>
> I think this is enough.
>
> Miguel
>
> P.S. I encourage users to always add a versionsuffix in case they are 
> rebuilding a package that already exists globally, otherwise which one gets 
> picked depends on the order of the folders in $MODULEPATH


--
echo "sysadmin know better bash than english" | sed s/min/mins/ \
  | sed 's/better bash/bash better/' # signal detected in a CERN forum










--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de<mailto:a.oc...@fz-juelich.de>
WWW:http://www.fz-juelich.de/ias/jsc/EN




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

[easybuild] Local user intstallation

[Vanzo, Davide]

Hello guys,

We would like to allow users to install software locally in their home 
directories via EB. Is there a way to configure EB to install the software and 
the modules in a local directory if the user is just a regular user but allow a 
subset of users (admins) to install directly in the cluster-wide software stack?

I gave it a try with --installpath but that does not help since the user would 
have to rebuild the whole toolchain stack instead of using the one already 
available on the cluster.

--
Davide Vanzo, PhD
Application Developer
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] install dir on gpfs

[Vanzo, Davide]

Todd,

We have all EasyBuild built software on GPFS and I have never experienced that 
kind of issue.
Unfortunately I do not have any suggestion on where to start looking at since I 
am not much a storage expert.

--
Davide Vanzo, PhD
Application Developer
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Jun 20 2017, at 4:07 pm, Heywood, Todd  wrote:

P.s. This issue does NOT occur if using g++ from a GCC built outside of the 
Easybuild framework.

From: 
> on 
behalf of Todd Heywood >
Reply-To: "easybuild@lists.ugent.be" 
>
Date: Tuesday, June 20, 2017 at 4:52 PM
To: "easybuild@lists.ugent.be" 
>
Subject: [easybuild] install dir on gpfs

Has anyone installed easybuild on GPFS, not NFS? We just migrated our easybuild 
installation to GPFS, and now when we use an Easybuild-built compiler to build 
software (on GPFS), the “configure” step fails when tests the compiler to 
create an executable, and tests whether the executable works. You can see the 
issue here:

xyz@bnbmgmt2:~/tmp$ g++ hello.cpp ; ./a.out

-bash: ./a.out: cannot execute binary file

xyz@bnbmgmt2:~/tmp$ ./a.out # it work a couple seconds later

Hello World!heywood@bnbmgmt2:~/tmp$

xyz@bnbmgmt2:~/tmp$

xyz@bnbmgmt2:~/tmp$ g++ hello.cpp ; file a.out

a.out: data

xyz@bnbmgmt2:~/tmp$ file a.out # gives expected output a couple seconds later

a.out: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked 
(uses shared libs), for GNU/Linux 2.6.18, not stripped

xyz@bnbmgmt2:~/tmp$

Thanks,

T

[easybuild] R renders with Xlib instead of cairo

[Vanzo, Davide]

We provide R 3.3.3 built with both intel and foss toolchains and in both cases 
we observe that the bitmap type is set to "Xlib" instead of "cairo", as it 
should be since the cairo capability is TRUE. Here it is for the unbelievers:

> capabilities("png")
png
TRUE
> capabilities("cairo")
cairo
TRUE
> capabilities("X11")
X11
TRUE
> options('bitmapType')
$bitmapType
[1] "Xlib"

According to the R documentation [A] the default should be "cairo" since it is 
available.
Do you think it is a bug in R or something relative to how it has been built?

[A] https://stat.ethz.ch/R-manual/R-devel/library/grDevices/html/x11.html


--
Davide Vanzo, PhD
Application Developer
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] Xvfb

[Vanzo, Davide]

Markus,

Thanks a lot!
If it works do you mind if I upload to the EB repo?

--
Davide Vanzo, PhD
Application Developer
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On Jun 10 2017, at 3:14 am, Markus Geimer  wrote:

Davide,

> Does anybody have (or have tried) to build Xvfb?
> I need it and I would rather not reinvent the wheel or help to finish it
> if possible.

You can get it by building the XServer package. Here is an easyconfig
that is working for me, including one support package that is not yet
in EB: https://gist.github.com/geimer/f375a45bbfe85309aaf1d720d464d80f

You may have to tweak toolchains/versions according to your environment.

Hth,
Markus

--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49-2461-61-1773
Fax: +49-2461-61-6656
E-Mail: m.gei...@fz-juelich.de
Web: http://www.fz-juelich.de/jsc



Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

[easybuild] Python 3 fails to install setuptools

[Vanzo, Davide]

Hello again,

I am facing an issue when installing setuptools-23.1.0 as part of  
Python-3.5.2-foss-2016b.eb. It fails with a permission denied despite it has 
full write permissions on the installation path:

creating build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/dependency_links.txt -> 
build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/entry_points.txt -> 
build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/requires.txt -> 
build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/top_level.txt -> 
build/bdist.linux-x86_64/egg/EGG-INFO
copying setuptools.egg-info/zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
creating dist
creating 'dist/setuptools-23.1.0-py3.5.egg' and adding 
'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Creating 
/gpfs22/easybuild/centos6/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/Python/3.5.2/lib/python3.5/site-packages/site.py
Processing setuptools-23.1.0-py3.5.egg
Copying setuptools-23.1.0-py3.5.egg to 
/gpfs22/easybuild/centos6/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/Python/3.5.2/lib/python3.5/site-packages
error: [Errno 13] Permission denied: 
'/gpfs22/easybuild/centos6/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/Python/3.5.2/lib/python3.5/site-packages/setuptools-23.1.0-py3.5.egg'
(at 
easybuild/software/Core/EasyBuild/3.1.2/lib/python2.7/site-packages/easybuild_framework-3.1.2-py2.7.egg/easybuild/tools/run.py:449
 in parse_cmd_output)
== 2017-05-10 11:58:07,532 easyblock.py:2518 WARNING build failed (first 300 
chars): cmd " 
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/Python/3.5.2/bin/python
 setup.py install 
--prefix=/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/Python/3.5.2 
" exited with exitcode 1 and output:
running install
running bdist_egg
running egg_info
writing top-level names to
== 2017-05-10 11:58:07,532 easyblock.py:276 INFO Closing log for application 
name Python version 3.5.2


According to the setuptools documentation this error can also occur when a 
previous version of setuptools is installed and the egg ant pth files are not 
deleted before installing the new version. I checked by installing a bare 
version of the same Python and, although it has setuptools-20.10.1 installed, 
none of the conflicting files mentioned in the documentation are present:

$ find . -name sys.path
$ find . -name setuptools*
./lib/python3.5/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl
./lib/python3.5/site-packages/pip/utils/setuptools_build.py
./lib/python3.5/site-packages/pip/utils/__pycache__/setuptools_build.cpython-35.pyc
./lib/python3.5/site-packages/setuptools
./lib/python3.5/site-packages/setuptools-20.10.1.dist-info


Anybody has encoutered this issue before?

Thanks!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] Mesa with SWR on non-AVX processor

[Vanzo, Davide]

Jack,

Thank you for the feedback.

In my path to enlightenment I stumbled across the following article, which 
seems to suggest that using swrast is not such a great idea:

http://www.phoronix.com/scan.php?page=article=mesa_81_llvmpipe=1

Any comment on that?

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On May 10 2017, at 12:01 pm, Jack Perdue <j-per...@tamu.edu> wrote:

AFAIK, we haven't had any problems with this change
(things seem to be working users aren't yelling):

< --with-gallium-drivers='swrast,swr'\
---
 > --with-gallium-drivers='swrast'\

jack

On 05/10/2017 11:45 AM, Vanzo, Davide wrote:
> Hello fellow easybuilders,
>
> I am working on creating a minimal toolchain set of modules for X11
> and Qt for GUI-based software in the foss-2016b toolchain. When trying
> to install Mesa-12.0.2-GCC-5.4.0-2.26.eb (created by
> using Mesa-12.0.2-foss-2016b.eb) it fails when building the libswrAVX2
> since the LLVM library has been built on a Nehalem machine which does
> not have support for AVX (see log at the end of the email).
>
> Clearly one solution would be to avoid building SWR as part of Mesa
> but I have no idea of the implication of this choice since I am not
> very familiar with this type of libraries.
>
> Any suggestion?
>
> Thanks!
>
>
> PS: Here is the error log:
>
> CXX rasterizer/jitter/libswrAVX2_la-blend_jit.lo
> CXX rasterizer/jitter/libswrAVX2_la-builder.lo
> CXX rasterizer/jitter/libswrAVX2_la-builder_misc.lo
> CXX rasterizer/jitter/libswrAVX2_la-fetch_jit.lo
> CXX rasterizer/jitter/libswrAVX2_la-JitManager.lo
> CXX rasterizer/jitter/libswrAVX2_la-streamout_jit.lo
> CXX rasterizer/memory/libswrAVX2_la-ClearTile.lo
> CXX rasterizer/memory/libswrAVX2_la-LoadTile.lo
> CXX rasterizer/memory/libswrAVX2_la-StoreTile.lo
> rasterizer/core/threads.cpp: In function ‘void
> WorkOnFifoBE(SWR_CONTEXT*, uint32_t, uint64_t&, TileSet&, uint32_t,
> uint32_t)’:
> rasterizer/core/threads.cpp:467:26: warning: unused variable
> ‘numWorkItems’ [-Wunused-variable]
> uint32_t numWorkItems = tile.getNumQueued();
> ^
> CXX rasterizer/scripts/libswrAVX2_la-gen_knobs.lo
> CXX rasterizer/jitter/libswrAVX2_la-builder_x86.lo
> CXX rasterizer/jitter/libswrAVX2_la-builder_gen.lo
> CXX swr_loader.lo
> CXX rasterizer/common/libswrAVX_la-formats.lo
> CXX rasterizer/common/libswrAVX_la-rdtsc_buckets.lo
> rasterizer/jitter/builder_misc.cpp: In member function ‘llvm::Value*
> Builder::PMAXSD(llvm::Value*, llvm::Value*)’:
> rasterizer/jitter/builder_misc.cpp:862:77: error: ‘x86_sse41_pmaxsd’
> is not a member of ‘llvm::Intrinsic’
> Function* pmaxsd =
> Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_sse41_pmaxsd);
> ^
> rasterizer/jitter/builder_misc.cpp: In member function ‘llvm::Value*
> Builder::PMINSD(llvm::Value*, llvm::Value*)’:
> rasterizer/jitter/builder_misc.cpp:891:77: error: ‘x86_sse41_pminsd’
> is not a member of ‘llvm::Intrinsic’
> Function* pminsd =
> Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_sse41_pminsd);
> ^
> make[5]: *** [rasterizer/jitter/libswrAVX2_la-builder_misc.lo] Error 1
> make[5]: *** Waiting for unfinished jobs
> rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
> Builder::VPMINSD(llvm::Value*, llvm::Value*)’:
> rasterizer/jitter/builder_x86.cpp:85:71: error: ‘x86_avx2_pmins_d’ is
> not a member of ‘llvm::Intrinsic’
> Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_avx2_pmins_d);
> ^
> rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
> Builder::VPMAXSD(llvm::Value*, llvm::Value*)’:
> rasterizer/jitter/builder_x86.cpp:92:71: error: ‘x86_avx2_pmaxs_d’ is
> not a member of ‘llvm::Intrinsic’
> Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_avx2_pmaxs_d);
> ^
> rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
> Builder::VPMOVSXBD(llvm::Value*)’:
> rasterizer/jitter/builder_x86.cpp:148:71: error: ‘x86_avx2_pmovsxbd’
> is not a member of ‘llvm::Intrinsic’
> Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_avx2_pmovsxbd);
> ^
> rasterizer/jitter/builder_x86.cpp: In member function ‘llvm::Value*
> Builder::VPMOVSXWD(llvm::Value*)’:
> rasterizer/jitter/builder_x86.cpp:155:71: error: ‘x86_avx2_pmovsxwd’
> is not a member of ‘llvm::Intrinsic’
> Function *func = Intrinsic::getDeclaration(JM()->mpCurrentModule,
> Intrinsic::x86_avx2_pmovsxwd);
> ^
> make[5]: *** [rasterizer/jitter/libswrAVX2_

Re: [easybuild] Lmod integration - Families

[Vanzo, Davide]

I am using intel-2016b.
However, it comes out is is an Lmod problem. I get the same message when I try 
to load the hidden module intel/.2016b. But I cannot figure out why. I will ask 
on the Lmod list.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 24 2017, at 3:26 am, Markus Geimer  wrote:

Davide,

> I am using your SwatHMNS module and everything is working fine until I
> try to build something that requires the imkl module. The error I see is
> the following:
>
> Lmod has detected the following error: These module(s) exist but cannot be
> loaded as requested: "imkl/11.3.3.210", "IntelMPI/5.1.3.181"
>
> It looks like EB is trying to load the IntelMPI and imkl in the opposite
> order. If I manually try to load the (hidden) intel/2016b module
> everything gets loaded correctly. Any idea?

Which toolchain are you using? The order in which the
dependencies are listed in the toolchain easyconfig should
be the order in which modules are loaded. If not, we are
in trouble (I think)...

Markus

--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49-2461-61-1773
Fax: +49-2461-61-6656
E-mail: m.gei...@fz-juelich.de
WWW: http://www.fz-juelich.de/jsc/



Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Lmod integration - Families

[Vanzo, Davide]

Maxime,
in your case isn't the use of "intel" as alternative name for the iccifort 
module creating conflicts with the intel toolchain module?

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 16 2017, at 9:35 am, Maxime Boissonneault 
 wrote:

If you want an example for this, you can have a look at our easyconfig
for iccifort :
https://github.com/ComputeCanada/easybuild-easyconfigs/blob/computecanada-master/easybuild/easyconfigs/i/iccifort/iccifort-2017.1.eb

Maxime

On 17-03-16 10:24, Markus Geimer wrote:
> Davide,
>
>> as a matter of fact I was thinking too about renaming iccifort to
>> something a little more familiar to the users. The problem is that if I
>> do so I need to modify all other easyconfig dependencies whenever they
>> depend on iccifort. Or do you have a better way?
> You can do so via the 'modaltsoftname' easyconfig parameter and
> a custom module naming scheme (needed since 'iccifort' is a
> toolchain). But then the easyconfigs can still refer to 'iccifort'.
>
> @Kenneth: We really, really, really, need a way to install icc
> and ifort as a bundle such that only a single compiler module
> remains... (Note: Not the full PSXE, just the compilers)
>
>> So if I understand correctly your suggestion, I may want to consider
>> using the --recursive_module_unload flag only when building GCC and
>> iccifort, sonce those are my only two base compilers, correct?
> Yes, you can put
>
> recursive_module_unload = True
>
> in the easyconfigs of 'icc', 'ifort', 'iccifort', and 'GCC'. Then
> the recursive unloading is done for the compilers only.
>
> Markus
>
>
>> On Mar 16 2017, at 9:04 am, Markus Geimer  wrote:
>>
>> Davide,
>>
>> yes, 'iccifort' needs to be visible. In fact, in my setup I am
>> hiding 'icc' and 'ifort' and only have 'iccifort' non-hidden
>> (well, it is actually also renamed to 'Intel', but that is a
>> different story...). The 'recursive_module_unload' is not
>> necessarily a global setting; you can also put it in an the
>> corresponding easyconfig where you want this behavior, e.g.,
>> in all compiler modules which are the lowest level of the
>> hierarchy and the recursive unloading shouldn't do any harm.
>>
>> Markus
>>
>>
>
> --
> Dr. Markus Geimer
> Juelich Supercomputing Centre
> Institute for Advanced Simulation
> Forschungszentrum Juelich GmbH
> 52425 Juelich, Germany
>
> Phone: +49-2461-61-1773
> Fax: +49-2461-61-6656
> E-mail: m.gei...@fz-juelich.de
> WWW: http://www.fz-juelich.de/jsc/
>
>
>
> 
> 
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
> 
> 
>

--
-
Maxime Boissonneault
Analyste de calcul - Calcul Québec, Université Laval
Président - Comité de coordination du soutien à la recherche de Calcul Québec
Team lead - Research Support National Team, Compute Canada
Instructeur Software Carpentry
Ph. D. en physique

Re: [easybuild] Lmod integration - Families

[Vanzo, Davide]

Thank you guys! That is really great.

+1 on the idea of having a single module for icc and ifort.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 16 2017, at 9:25 am, Markus Geimer  wrote:

Davide,

> as a matter of fact I was thinking too about renaming iccifort to
> something a little more familiar to the users. The problem is that if I
> do so I need to modify all other easyconfig dependencies whenever they
> depend on iccifort. Or do you have a better way?

You can do so via the 'modaltsoftname' easyconfig parameter and
a custom module naming scheme (needed since 'iccifort' is a
toolchain). But then the easyconfigs can still refer to 'iccifort'.

@Kenneth: We really, really, really, need a way to install icc
and ifort as a bundle such that only a single compiler module
remains... (Note: Not the full PSXE, just the compilers)

> So if I understand correctly your suggestion, I may want to consider
> using the --recursive_module_unload flag only when building GCC and
> iccifort, sonce those are my only two base compilers, correct?

Yes, you can put

recursive_module_unload = True

in the easyconfigs of 'icc', 'ifort', 'iccifort', and 'GCC'. Then
the recursive unloading is done for the compilers only.

Markus

> On Mar 16 2017, at 9:04 am, Markus Geimer  wrote:
>
> Davide,
>
> yes, 'iccifort' needs to be visible. In fact, in my setup I am
> hiding 'icc' and 'ifort' and only have 'iccifort' non-hidden
> (well, it is actually also renamed to 'Intel', but that is a
> different story...). The 'recursive_module_unload' is not
> necessarily a global setting; you can also put it in an the
> corresponding easyconfig where you want this behavior, e.g.,
> in all compiler modules which are the lowest level of the
> hierarchy and the recursive unloading shouldn't do any harm.
>
> Markus
>
>

--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49-2461-61-1773
Fax: +49-2461-61-6656
E-mail: m.gei...@fz-juelich.de
WWW: http://www.fz-juelich.de/jsc/



Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] GCCcore build in HMNS

[Vanzo, Davide]

OT



From: easybuild-requ...@lists.ugent.be 
[mailto:easybuild-requ...@lists.ugent.be] On Behalf Of Vanzo, Davide
Sent: Wednesday, March 8, 2017 2:09 PM
To: easybuild@lists.ugent.be
Subject: Re: [easybuild] GCCcore build in HMNS



Shahzeb,

Looks good to me. It is expected that GCCcore depends on dummy toolchain built 
software since that is the only toolchain below GCCcore. In other words, you 
need whatever has been built with the system compiler and binutils in order to 
compile GCCcore. Once you have GCCcore built, at this point you will use it to 
compile again M4/Bison/flex/zlib/binutils within the GCCcore toolchain so that 
whatever is built with GCC is completely independent from the underlying OS 
software stack.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
www.accre.vanderbilt.edu<http://www.accre.vanderbilt.edu>

On Mar 8 2017, at 12:58 pm, Siddiqui, Shahzeb 
<shahzeb.siddi...@pfizer.com<mailto:shahzeb.siddi...@pfizer.com>> wrote:

Hello,



I want to find out if the build for GCCcore 5.4.0 is suppose to look like this 
under HMNS. I have copied all the files from easybuild repo into my local 
directory because I have disabled robot paths.  One thing I don’t understand 
why there is M4 and binutils built with GCCcore toolchain while it also is a 
dependency in GCCcore. When I tried to build GCCcore-5.4.0 the dependency picks 
up M4 and binutils from dummy toolchain.



I want to hide GCCcore and its deps, is this the correct way to do this.





hpcswadm@hpcv18$eb GCCcore-5.4.0.eb -D --hidden 
--hide-deps=M4,Bison,flex,zlib,binutils

== temporary log file in case of crash /tmp/eb-mZkRzA/easybuild-xv6sEZ.log

Dry run: printing build status of easyconfigs and dependencies

CFGS=/hpc/hpcswadm/easybuild

* [ ] $CFGS/M4/M4-1.4.17.eb (module: Core | M4/.1.4.17)

* [ ] $CFGS/Bison/Bison-3.0.4.eb (module: Core | Bison/.3.0.4)

* [ ] $CFGS/flex/flex-2.6.0.eb (module: Core | flex/.2.6.0)

* [ ] $CFGS/zlib/zlib-1.2.8.eb (module: Core | zlib/.1.2.8)

* [ ] $CFGS/binutils/binutils-2.26.eb (module: Core | binutils/.2.26)

* [ ] $CFGS/GCCcore/GCCcore-5.4.0.eb (module: Core | GCCcore/.5.4.0)



Regards,



Shahzeb Siddiqui

HPC Linux Engineer

B2220-447.2

Groton, CT

Re: [easybuild] GCCcore build in HMNS

[Vanzo, Davide]

Shahzeb,
Looks good to me. It is expected that GCCcore depends on dummy toolchain built 
software since that is the only toolchain below GCCcore. In other words, you 
need whatever has been built with the system compiler and binutils in order to 
compile GCCcore. Once you have GCCcore built, at this point you will use it to 
compile again M4/Bison/flex/zlib/binutils within the GCCcore toolchain so that 
whatever is built with GCC is completely independent from the underlying OS 
software stack.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
www.accre.vanderbilt.edu

On Mar 8 2017, at 12:58 pm, Siddiqui, Shahzeb  
wrote:

Hello,



I want to find out if the build for GCCcore 5.4.0 is suppose to look like this 
under HMNS. I have copied all the files from easybuild repo into my local 
directory because I have disabled robot paths.  One thing I don’t understand 
why there is M4 and binutils built with GCCcore toolchain while it also is a 
dependency in GCCcore. When I tried to build GCCcore-5.4.0 the dependency picks 
up M4 and binutils from dummy toolchain.



I want to hide GCCcore and its deps, is this the correct way to do this.





hpcswadm@hpcv18$eb GCCcore-5.4.0.eb -D --hidden 
--hide-deps=M4,Bison,flex,zlib,binutils

== temporary log file in case of crash /tmp/eb-mZkRzA/easybuild-xv6sEZ.log

Dry run: printing build status of easyconfigs and dependencies

CFGS=/hpc/hpcswadm/easybuild

* [ ] $CFGS/M4/M4-1.4.17.eb (module: Core | M4/.1.4.17)

* [ ] $CFGS/Bison/Bison-3.0.4.eb (module: Core | Bison/.3.0.4)

* [ ] $CFGS/flex/flex-2.6.0.eb (module: Core | flex/.2.6.0)

* [ ] $CFGS/zlib/zlib-1.2.8.eb (module: Core | zlib/.1.2.8)

* [ ] $CFGS/binutils/binutils-2.26.eb (module: Core | binutils/.2.26)

* [ ] $CFGS/GCCcore/GCCcore-5.4.0.eb (module: Core | GCCcore/.5.4.0)



Regards,



Shahzeb Siddiqui

HPC Linux Engineer

B2220-447.2

Groton, CT

Re: [easybuild] FOSS vs CUDA

[Vanzo, Davide]

Maxime,
your point it totally legitimate. My approach is less about philosophy and more 
about practicality.
We picked the foss toolchain instead of the goolf toolchain because of its more 
collaborative nature and scheduled release. The problem is that if we now start 
using a goolfc toolchain, we could not get the benefit of reusing most of the 
software built with foss since we build with minimal toolchains. Hence I 
proposed of starting a fosscuda toolchain that is aligned with the foss 
release. That's it.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 2 2017, at 5:30 pm, Maxime Boissonneault 
 wrote:

Hi,

I've seen a couple emails about CUDA recently, and I was a bit surprised
to see work done about FOSS and CUDA.

Isn't the whole point of FOSS to be free and open source ? CUDA is not
open source. Won't die-hard fan of FOSS object to having CUDA in a FOSS
toolchain ?

I personally don't really care, I just want the best performance for my
users (which is why we don't go with FOSS in the first place, since MKL
gives better performances than OpenBLAS).

I just thought I'ld raise the question.

--
-
Maxime Boissonneault
Analyste de calcul - Calcul Québec, Université Laval
Président - Comité de coordination du soutien à la recherche de Calcul Québec
Team lead - Research Support National Team, Compute Canada
Instructeur Software Carpentry
Ph. D. en physique

Re: [easybuild] GROMACS with gpu support

[Vanzo, Davide]

Shahzeb,
the fosscuda toolchain is still a work in progress and no PR have been released 
yet.
Hopefully I will get something working in the next months.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On Mar 1 2017, at 2:00 pm, Siddiqui, Shahzeb <shahzeb.siddi...@pfizer.com> 
wrote:

This might be a dumb question, I don’t see a foss toolchain, the only closest 
one is goolfc which uses gompic toolchain. Which toolchain should we use?



From: easybuild-requ...@lists.ugent.be 
[mailto:easybuild-requ...@lists.ugent.be] On Behalf Of Vanzo, Davide
Sent: Wednesday, March 1, 2017 2:48 PM
To: easybuild@lists.ugent.be
Cc: Oliver Stueker
Subject: Re: [easybuild] GROMACS with gpu support



Maxime and Oliver,

since GROMACS with GPU acceleration is in my software list for the fosscuda 
toolchain I will be working on, I would be interested in sharing the pain if 
somebody is already working on it.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu<http://www.accre.vanderbilt.edu>

[https://link.nylas.com/open/6n4bjcakifea4azkemwwns1da/local-37bc18e0-2b36]

On Mar 1 2017, at 1:43 pm, Maxime Boissonneault 
<maxime.boissonnea...@calculquebec.ca<mailto:maxime.boissonnea...@calculquebec.ca>>
 wrote:

Hi Shahzeb,
I am putting you in contact with my colleague Oliver Stueker. He is actually 
working to modify the recipe for GROMACS GPU. Maybe he can help.

Maxime

On 17-03-01 14:38, Siddiqui, Shahzeb wrote:

Hello,



I am curious how I would build GROMACS with GPU support, I only see GROMACS 
build with multithreaded and hybrid approach. I notice this is just a tweak to 
toolchainopts that sets whether to build with openmp and mpi. This seems fine, 
but not sure how I would build with cuda.



Do I need to load a cuda specific toolchain like gompic or intelcuda.



Regards,







Shahzeb Siddiqui

HPC Linux Engineer

B2220-447.2

Groton, CT

Re: [easybuild] GROMACS with gpu support

[Vanzo, Davide]

Maxime and Oliver,
since GROMACS with GPU acceleration is in my software list for the fosscuda 
toolchain I will be working on, I would be interested in sharing the pain if 
somebody is already working on it.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu


On Mar 1 2017, at 1:43 pm, Maxime Boissonneault 
 wrote:
Hi Shahzeb,
I am putting you in contact with my colleague Oliver Stueker. He is actually 
working to modify the recipe for GROMACS GPU. Maybe he can help.

Maxime

On 17-03-01 14:38, Siddiqui, Shahzeb wrote:

Hello,



I am curious how I would build GROMACS with GPU support, I only see GROMACS 
build with multithreaded and hybrid approach. I notice this is just a tweak to 
toolchainopts that sets whether to build with openmp and mpi. This seems fine, 
but not sure how I would build with cuda.



Do I need to load a cuda specific toolchain like gompic or intelcuda.



Regards,







Shahzeb Siddiqui

HPC Linux Engineer

B2220-447.2

Groton, CT

Re: [easybuild] Playlist from 2nd EasyBuild User Meeting

[Vanzo, Davide]

Alan,
thank you from this side of the pond!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Feb 16 2017, at 4:46 am, Alan O'Cais  wrote:
Dear all,

I finished editing the videos from the 2nd User Meeting. You can find the full 
playlist at
https://www.youtube.com/playlist?list=PLVA9BuLC1j-yfxp2w-wraAGDCmhjb3o5Y

Enjoy!

Alan

--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de
WWW:http://www.fz-juelich.de/ias/jsc/EN




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Lmod integration - Families

[Vanzo, Davide]

Damian and Alex,
I am still on a pre-production stage and I am trying to find the best way to 
implement it in production such that the users experience will be the best 
possible. So I was just wandering what other groups do in order to avoid 
reinventing the wheel.
Your idea of not exposing the toolchain to users is already a good feedback. Do 
you add the family attribute manually after EB installed the default module for 
a specific package?
For this purpose it would be nice to have a function in EB that would allow 
from the easyconfig file (or even better in a easyconfig patch files)  to add 
additional lines to the generated module for a more easy to manage localization.

PS: You are right, I will not be able to join you guys at the user meeting. I 
will surely watch any recordings or slides you will make available.

Davide


On Feb 6 2017, at 12:05 pm, Alvarez, Damian <d.alva...@fz-juelich.de> wrote:

I’ll cover it a bit, but not too deeply. In any case, I don’t think Davide will 
come.



Anyway, what are the issues you are seeing? I think that using a toolchain 
based module naming scheme this would work similarly to the hierarchical scheme 
(ie: it will prevent you from loading a compiler on top of another one)



Damian



From: <easybuild-requ...@lists.ugent.be> on behalf of Alan O'Cais 
<a.oc...@fz-juelich.de>
Reply-To: "easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
Date: Monday 6 February 2017 at 18:22
To: "easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
Subject: Re: [easybuild] Lmod integration - Families



We use families for  compilers and MPI implementations in a hierarchical 
scheme. We do not expose the toolchains to normal users. Damian will probably 
cover this a little in his presentation on Wednesday.



Alan



On 6 February 2017 at 16:57, Vanzo, Davide 
<davide.va...@vanderbilt.edu<mailto:davide.va...@vanderbilt.edu>> wrote:

Hello all,

one of the most useful features provided by Lmod is the ability to assign a 
family to a subset of modules in order to avoid loading potentially conflicting 
software. This gets a little complicated when dealing with the toolchain-based 
model of EB. I was wondering if anybody is using families and how.

Thank you.

--
Davide Vanzo, PhD
Application Developer

Adjunct Assistant Professor of Chemical and Biomolecular Engineering

Advanced Computing Center for Research and Education (ACCRE)

Vanderbilt University - Hill Center 201
www.accre.vanderbilt.edu<http://www.accre.vanderbilt.edu>





--

Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de<mailto:a.oc...@fz-juelich.de>
WWW:http://www.fz-juelich.de/ias/jsc/EN




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

[easybuild] Lmod integration - Families

[Vanzo, Davide]

Hello all,
one of the most useful features provided by Lmod is the ability to assign a 
family to a subset of modules in order to avoid loading potentially conflicting 
software. This gets a little complicated when dealing with the toolchain-based 
model of EB. I was wondering if anybody is using families and how.
Thank you.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
www.accre.vanderbilt.edu

Re: [easybuild] Raw output for debugging

[Vanzo, Davide]

Yes, no luck even with the debug flag. Here are the last last log lines before 
my SIGINT:

== 2017-01-31 09:07:15,484 easyblock.py:2263 DEBUG Not skipping test step 
(skippable: True, skip: None, skipsteps: [], module_only: False, force: False
== 2017-01-31 09:07:15,484 build_log.py:216 INFO testing...
== 2017-01-31 09:07:15,485 easyblock.py:2271 INFO Starting test step
== 2017-01-31 09:07:15,485 templates.py:151 DEBUG config: 

== 2017-01-31 09:07:15,485 templates.py:175 DEBUG version found in easyconfig 
is 5.24.0
== 2017-01-31 09:07:15,485 templates.py:219 DEBUG name: github_account, config: 
None
== 2017-01-31 09:07:15,485 templates.py:219 DEBUG name: name, config: Perl
== 2017-01-31 09:07:15,486 templates.py:219 DEBUG name: version, config: 5.24.0
== 2017-01-31 09:07:15,486 templates.py:219 DEBUG name: versionsuffix, config:
== 2017-01-31 09:07:15,486 templates.py:219 DEBUG name: versionprefix, config:
== 2017-01-31 09:07:15,486 templates.py:151 DEBUG config: 

== 2017-01-31 09:07:15,486 templates.py:175 DEBUG version found in easyconfig 
is 5.24.0
== 2017-01-31 09:07:15,487 templates.py:219 DEBUG name: github_account, config: 
None
== 2017-01-31 09:07:15,487 templates.py:219 DEBUG name: name, config: Perl
== 2017-01-31 09:07:15,487 templates.py:219 DEBUG name: version, config: 5.24.0
== 2017-01-31 09:07:15,487 templates.py:219 DEBUG name: versionsuffix, config:
== 2017-01-31 09:07:15,487 templates.py:219 DEBUG name: versionprefix, config:
== 2017-01-31 09:07:15,488 easyblock.py:2274 INFO Running method test_step part 
of step test
== 2017-01-31 09:07:15,488 run.py:137 DEBUG run_cmd: running cmd export 
LC_ALL=C && make test (in /mnt/ramdisk/Perl/5.24.0/GCC-5.4.0-2.26/perl-5.24.0)

DV


On Jan 31 2017, at 9:31 am, Kenneth Hoste <kenneth.ho...@ugent.be> wrote:
Hi Davide,

On 31/01/2017 16:01, Vanzo, Davide wrote:
Kenneth,
yes, for the configure and build steps I have the output in the log filed 
(sorry for the confusion before), but I don't see anything for the test step. 
For example, for both Tcl and Perl the last thing I see is:

== 2017-01-30 17:55:58,057 easyblock.py:2274 INFO Running method test_step part 
of step test

If the test hangs (as happened in both cases for me) I have no clue what is 
going wrong.

Have you tried running with "eb --debug" to get a debug log?

The test step for Tcl is nothing more than a 'make test' afaik.

A non-debug log probably only shows which commands were run after they 
completed (we should probably change that).


regards,

Kenneth

DV


On Jan 31 2017, at 4:09 am, Kenneth Hoste 
<kenneth.ho...@ugent.be><mailto:kenneth.ho...@ugent.be> wrote:
Hi Davide,

On 30/01/2017 23:08, Vanzo, Davide wrote:
Hello world,
I would find quite useful to have an option (or maybe make it a default 
behavior) to get the full raw standard output and error from whatever EB is 
running (make, test, etc.). For example, while installing Tcl it was hanging at 
the test section. After trying manually, I realized it was the httpd test that 
was hanging because of the iptables rules on that port. If I would have had 
access to the raw output I could have figured it out without having to redo the 
test manually outside EB. This would be even more useful when developing an 
easyconfig or easyblock.
Have I missed the option or is it something that should be implemented on EB?

This output should be available in the log file that EasyBuild collects?
The location of the (temporary) log file is printed at the start of EB.

With the hanging Tcl installation, you should be able to just dive into the log 
file after hitting Ctrl-C?

If needed, you could pass --debug to get a debug log, but the info you're 
looking for should be there by default already...


regards,

Kenneth


Thanks!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu<http://www.accre.vanderbilt.edu>

Re: [easybuild] Raw output for debugging

[Vanzo, Davide]

Kenneth,
yes, for the configure and build steps I have the output in the log filed 
(sorry for the confusion before), but I don't see anything for the test step. 
For example, for both Tcl and Perl the last thing I see is:

== 2017-01-30 17:55:58,057 easyblock.py:2274 INFO Running method test_step part 
of step test

If the test hangs (as happened in both cases for me) I have no clue what is 
going wrong.

DV


On Jan 31 2017, at 4:09 am, Kenneth Hoste <kenneth.ho...@ugent.be> wrote:
Hi Davide,

On 30/01/2017 23:08, Vanzo, Davide wrote:
Hello world,
I would find quite useful to have an option (or maybe make it a default 
behavior) to get the full raw standard output and error from whatever EB is 
running (make, test, etc.). For example, while installing Tcl it was hanging at 
the test section. After trying manually, I realized it was the httpd test that 
was hanging because of the iptables rules on that port. If I would have had 
access to the raw output I could have figured it out without having to redo the 
test manually outside EB. This would be even more useful when developing an 
easyconfig or easyblock.
Have I missed the option or is it something that should be implemented on EB?

This output should be available in the log file that EasyBuild collects?
The location of the (temporary) log file is printed at the start of EB.

With the hanging Tcl installation, you should be able to just dive into the log 
file after hitting Ctrl-C?

If needed, you could pass --debug to get a debug log, but the info you're 
looking for should be there by default already...


regards,

Kenneth


Thanks!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu<http://www.accre.vanderbilt.edu>

[easybuild] Raw output for debugging

[Vanzo, Davide]

Hello world,
I would find quite useful to have an option (or maybe make it a default 
behavior) to get the full raw standard output and error from whatever EB is 
running (make, test, etc.). For example, while installing Tcl it was hanging at 
the test section. After trying manually, I realized it was the httpd test that 
was hanging because of the iptables rules on that port. If I would have had 
access to the raw output I could have figured it out without having to redo the 
test manually outside EB. This would be even more useful when developing an 
easyconfig or easyblock.
Have I missed the option or is it something that should be implemented on EB?

Thanks!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

[easybuild] foss + CUDA toolchain

[Vanzo, Davide]

Hello world,
I really like the idea of using foss and intel toolchain as common toolchain 
with other sites. However since we also have GPU nodes on our cluster we would 
need a foss toolchain with CUDA. I know that there is goolfc already out there 
but that does not comply to the a/b release frequency as foss does. Plus the 
fact that it is not "fossc" it does not make it obvious to users what it is.
I was then wondering if any other may benefit from such toolchain and if it 
would be a good idea starting it.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] Python-2.7.12-foss-2016b undefined references

[Vanzo, Davide]

Alan,
thanks for pointing me to that PR. That "--with-termlib" is the problem. With 
that option the symbols are stripped on a separate library file (libtinfo.a) 
and the linking fails when building against the ncurses-6.0.eb build.

Since this is not a local problem I have created a dedicated issue here:

https://github.com/hpcugent/easybuild-easyconfigs/issues/4049

DV


On Jan 18 2017, at 3:12 pm, Alan O'Cais <a.oc...@fz-juelich.de> wrote:
This sounds like it might be somehow related to 
https://github.com/hpcugent/easybuild-easyconfigs/pull/3545

On 18 Jan 2017 9:05 pm, "Vanzo, Davide" 
<davide.va...@vanderbilt.edu<mailto:davide.va...@vanderbilt.edu>> wrote:
When trying to build Python-2.7.12-foss-2016b it fails with the error below.

=

gcc -L/usr/software/software/Core/GCCcore/5.4.0/lib64 
-L/usr/software/software/Core/GCCcore/5.4.0/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/OpenBLAS/0.2.18-LAPACK-3.6.1/lib
 
-L/usr/software/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/ScaLAPACK/2.0.2-OpenBLAS-0.2.18-LAPACK-3.6.1/lib
 -L/usr/software/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/FFTW/3.3.4/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/bzip2/1.0.6/lib 
-L/usr/software/software/Core/zlib/1.2.8/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib 
-L/usr/software/software/Core/ncurses/6.0/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/SQLite/3.13.0/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/Tk/8.6.5/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/GMP/6.1.1/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/libffi/3.2.1/lib64 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/libffi/3.2.1/lib -Xlinker 
-export-dynamic -o python \
Modules/python.o \
-L. -lpython2.7 -ldl -lm -lpthread -lutil 
/usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib/libreadline.a
 /usr/software/software/Core/ncurses/6.0/lib/libncurses.a   -lm
./libpython2.7.so<http://libpython2.7.so>: error: undefined reference to 'tputs'
./libpython2.7.so<http://libpython2.7.so>: error: undefined reference to 'tgoto'
./libpython2.7.so<http://libpython2.7.so>: error: undefined reference to 
'tgetnum'
./libpython2.7.so<http://libpython2.7.so>: error: undefined reference to 'PC'
./libpython2.7.so<http://libpython2.7.so>: error: undefined reference to 'BC'
./libpython2.7.so<http://libpython2.7.so>: error: undefined reference to 'UP'
./libpython2.7.so<http://libpython2.7.so>: error: undefined reference to 
'tgetent'
./libpython2.7.so<http://libpython2.7.so>: error: undefined reference to 
'tgetstr'
./libpython2.7.so<http://libpython2.7.so>: error: undefined reference to 
'tgetflag'
collect2: error: ld returned 1 exit status
make: *** [python] Error 1

==

The weird thing is that libreadline.a contains such symbols when checking with 
readelf. Has anyone had this problem before?

On a side note, while digginig into libreadline and ncurses, I noticed that 
although we want libreadline linked to our custom version of ncurses 6.0, it 
gets actually linked to the system ncurses library. This can be solved by 
modifying the easyconfig to read as follows:

preconfigopts = 'env LDFLAGS="-lncurses -L$EBROOTNCURSES/lib"'

Is it just me having this problem or can be something worth enhancing?


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu<http://www.accre.vanderbilt.edu>




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Python-2.7.12-foss-2016b undefined references

[Vanzo, Davide]

Ward,
the symbols exist in the libreadline.a, not in libncurses.a as you can see here:

$ readelf --syms 
/usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib/libreadline.a
 | grep 'tputs\|tgoto\|tgetnum\|PC\|BC\|UP\|tgetent\|tgetstr\|tgetflag'
   97:  0 NOTYPE  GLOBAL DEFAULT  UND tputs
  127:  0 NOTYPE  GLOBAL DEFAULT  UND tgoto
   70:  0 NOTYPE  GLOBAL DEFAULT  UND tgetnum
   97:  0 NOTYPE  GLOBAL DEFAULT  UND PC
   99:  0 NOTYPE  GLOBAL DEFAULT  UND BC
  100:  0 NOTYPE  GLOBAL DEFAULT  UND UP
  101:  0 NOTYPE  GLOBAL DEFAULT  UND tgetent
  102:  0 NOTYPE  GLOBAL DEFAULT  UND tgetstr
  104:  0 NOTYPE  GLOBAL DEFAULT  UND tgetflag
  118:  0 NOTYPE  GLOBAL DEFAULT  UND tputs

$ readelf --syms /usr/software/software/Core/ncurses/6.0/lib/libncurses.a | 
grep 'tputs\|tgoto\|tgetnum\|PC\|BC\|UP\|tgetent\|tgetstr\|tgetflag'
   28:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp
   41:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp
   14:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp
   25:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp

Davide

On Jan 18 2017, at 2:56 pm, Ward Poelmans <wpoel...@gmail.com> wrote:

Hi Davide,

On Wed, Jan 18, 2017 at 9:05 PM, Vanzo, Davide
<davide.va...@vanderbilt.edu> wrote:
> When trying to build Python-2.7.12-foss-2016b it fails with the error below.
>
> =
> /usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib/libreadline.a
> /usr/software/software/Core/ncurses/6.0/lib/libncurses.a -lm
> ./libpython2.7.so: error: undefined reference to 'tputs'
> ./libpython2.7.so: error: undefined reference to 'tgoto'
> ./libpython2.7.so: error: undefined reference to 'tgetnum'
> ./libpython2.7.so: error: undefined reference to 'PC'
> ./libpython2.7.so: error: undefined reference to 'BC'
> ./libpython2.7.so: error: undefined reference to 'UP'
> ./libpython2.7.so: error: undefined reference to 'tgetent'
> ./libpython2.7.so: error: undefined reference to 'tgetstr'
> ./libpython2.7.so: error: undefined reference to 'tgetflag'
> collect2: error: ld returned 1 exit status
> make: *** [python] Error 1
>
> ==
>
> The weird thing is that libreadline.a contains such symbols when checking
> with readelf. Has anyone had this problem before?

Yes, I've seen this before. That's why we link libreadline to ncurses.
I'm surprise you get this error when both the static archive for
libreadline and ncurses are include.

So you are sure that those symbols exists in libncurses.a and it is defined?

Can you try what happens if you put the paths for libncurses and
libreadline before -lpython2.7?

> On a side note, while digginig into libreadline and ncurses, I noticed that
> although we want libreadline linked to our custom version of ncurses 6.0, it
> gets actually linked to the system ncurses library. This can be solved by
> modifying the easyconfig to read as follows:
>
> preconfigopts = 'env LDFLAGS="-lncurses -L$EBROOTNCURSES/lib"'
>
> Is it just me having this problem or can be something worth enhancing?

That's definitely a bug. Please open an issue on github (or pull request).

Ward

Re: [easybuild] Minimal toolchain issue with Python-3.5.2-foss-2016b

[Vanzo, Davide]

Makes totally sense. Thanks for clarifying this for me Alan.

DV


On Jan 16 2017, at 3:44 pm, Alan O'Cais <a.oc...@fz-juelich.de> wrote:
Minimal toolchains doesn't take existing modules into account unless asked to 
with '--use-existing-modules'.

It also doesn't include dummy in the search unless told to...which makes me 
believe it's building ncurses with dummy only because one of the easyconfigs in 
the dep list has an explicit dependency on it. That almost certainly comes from 
gettext, for gettext to function at Core level it needs ncurses at Core.

On 16 Jan 2017 10:08 pm, "Vanzo, Davide" 
<davide.va...@vanderbilt.edu<mailto:davide.va...@vanderbilt.edu>> wrote:
Hello guys,
I am installing Python-3.5.2-foss-2016b with minimal toolchain and I am facing 
the following issue.
As you can see from the output below, EB tries to install the ncurses 6.0 with 
the dummy toolchain even if the same version is already installed with the 
GCCcore 5.4.0 toolchain (and correctly reported by Lmod).
Opposite thing with numactl. EB wants to install the GCCcore version even if 
the GCC 5.4.0-2.26 is already installed.
My understanding of the minimal toolchain was that it tries to find an already 
installed version from the highest related toolchain down to dummy before 
installing it for the highest available toolchain (foss in this case). Has 
something changed or am I completely misunderstanding its behavior?


$ eb Python-3.5.2-foss-2016b.eb -D --minimal-toolchain --robot=$PWD
== temporary log file in case of crash /tmp/eb-z4kRE6/easybuild-a27hVB.log
Dry run: printing build status of easyconfigs and dependencies
* [x] /usr/software/ebfiles_repo/M4/M4-1.4.17.eb (module: Core | M4/.1.4.17)
* [x] /usr/software/ebfiles_repo/Bison/Bison-3.0.4.eb (module: Core | 
Bison/.3.0.4)
* [x] /usr/software/ebfiles_repo/flex/flex-2.6.0.eb (module: Core | flex/.2.6.0)
* [x] /usr/software/ebfiles_repo/zlib/zlib-1.2.8.eb (module: Core | zlib/.1.2.8)
* [x] /usr/software/ebfiles_repo/binutils/binutils-2.26.eb (module: Core | 
binutils/.2.26)
* [x] /usr/software/ebfiles_repo/GCCcore/GCCcore-5.4.0.eb (module: Core | 
GCCcore/.5.4.0)
* [x] /usr/software/ebfiles_repo/M4/M4-1.4.17-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | M4/.1.4.17)
* [x] /usr/software/ebfiles_repo/zlib/zlib-1.2.8-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | zlib/.1.2.8)
* [x] /usr/software/ebfiles_repo/Bison/Bison-3.0.4-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | Bison/.3.0.4)
* [x] /usr/software/ebfiles_repo/flex/flex-2.6.0-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | flex/.2.6.0)
* [x] /usr/software/ebfiles_repo/binutils/binutils-2.26-GCCcore-5.4.0.eb 
(module: Compiler/GCCcore/5.4.0 | binutils/.2.26)
* [x] /usr/software/ebfiles_repo/GCC/GCC-5.4.0-2.26.eb (module: Core | 
GCC/5.4.0-2.26)
* [x] /usr/software/ebfiles_repo/ncurses/ncurses-6.0-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | ncurses/.6.0)
* [x] /usr/software/ebfiles_repo/Tcl/Tcl-8.6.5-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | Tcl/8.6.5)
* [x] /usr/software/ebfiles_repo/bzip2/bzip2-1.0.6-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | bzip2/.1.0.6)
* [x] /usr/software/ebfiles_repo/libreadline/libreadline-6.3-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | libreadline/.6.3)
* [x] /usr/software/ebfiles_repo/SQLite/SQLite-3.13.0-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | SQLite/3.13.0)
* [x] /usr/software/ebfiles_repo/Tk/Tk-8.6.5-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | Tk/.8.6.5)
* [x] /usr/software/ebfiles_repo/libffi/libffi-3.2.1-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | libffi/.3.2.1)
* [x] /usr/software/ebfiles_repo/Autoconf/Autoconf-2.69-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | Autoconf/.2.69)
* [ ] /home/vanzod/OpenSSL-1.1.0c-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | OpenSSL/.1.1.0c)
* [x] /usr/software/ebfiles_repo/Automake/Automake-1.15-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | Automake/.1.15)
* [ ] 
/gpfs0/software/centos6/software/Core/EasyBuild/software/EasyBuild/3.0.2/lib/python2.6/site-packages/easybuild_easyconfigs-3.0.2-py2.6.egg/easybuild/easyconfigs/n/ncurses/ncurses-6.0.eb
 (module: Core | ncurses/.6.0)
* [ ] 
/gpfs0/software/centos6/software/Core/EasyBuild/software/EasyBuild/3.0.2/lib/python2.6/site-packages/easybuild_easyconfigs-3.0.2-py2.6.egg/easybuild/easyconfigs/g/gettext/gettext-0.19.8.eb
 (module: Core | gettext/.0.19.8)
* [x] /usr/software/ebfiles_repo/libtool/libtool-2.4.6-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | libtool/.2.4.6)
* [x] /usr/software/ebfiles_repo/Autotools/Autotools-20150215-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | Autotools/.20150215)
* [x] /usr/software/ebfiles_repo/GMP/GMP-6.1.1-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | GMP/.6.1.1)
* [ ] 
/gpfs0/software/centos6/software/Core/EasyBuild/software/EasyBuild/3.0.2/lib/python2.6/site-packages/easybuild_easyconfigs-3.0.2-py2.6

[easybuild] Minimal toolchain issue with Python-3.5.2-foss-2016b

[Vanzo, Davide]

Hello guys,
I am installing Python-3.5.2-foss-2016b with minimal toolchain and I am facing 
the following issue.
As you can see from the output below, EB tries to install the ncurses 6.0 with 
the dummy toolchain even if the same version is already installed with the 
GCCcore 5.4.0 toolchain (and correctly reported by Lmod).
Opposite thing with numactl. EB wants to install the GCCcore version even if 
the GCC 5.4.0-2.26 is already installed.
My understanding of the minimal toolchain was that it tries to find an already 
installed version from the highest related toolchain down to dummy before 
installing it for the highest available toolchain (foss in this case). Has 
something changed or am I completely misunderstanding its behavior?


$ eb Python-3.5.2-foss-2016b.eb -D --minimal-toolchain --robot=$PWD
== temporary log file in case of crash /tmp/eb-z4kRE6/easybuild-a27hVB.log
Dry run: printing build status of easyconfigs and dependencies
* [x] /usr/software/ebfiles_repo/M4/M4-1.4.17.eb (module: Core | M4/.1.4.17)
* [x] /usr/software/ebfiles_repo/Bison/Bison-3.0.4.eb (module: Core | 
Bison/.3.0.4)
* [x] /usr/software/ebfiles_repo/flex/flex-2.6.0.eb (module: Core | flex/.2.6.0)
* [x] /usr/software/ebfiles_repo/zlib/zlib-1.2.8.eb (module: Core | zlib/.1.2.8)
* [x] /usr/software/ebfiles_repo/binutils/binutils-2.26.eb (module: Core | 
binutils/.2.26)
* [x] /usr/software/ebfiles_repo/GCCcore/GCCcore-5.4.0.eb (module: Core | 
GCCcore/.5.4.0)
* [x] /usr/software/ebfiles_repo/M4/M4-1.4.17-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | M4/.1.4.17)
* [x] /usr/software/ebfiles_repo/zlib/zlib-1.2.8-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | zlib/.1.2.8)
* [x] /usr/software/ebfiles_repo/Bison/Bison-3.0.4-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | Bison/.3.0.4)
* [x] /usr/software/ebfiles_repo/flex/flex-2.6.0-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | flex/.2.6.0)
* [x] /usr/software/ebfiles_repo/binutils/binutils-2.26-GCCcore-5.4.0.eb 
(module: Compiler/GCCcore/5.4.0 | binutils/.2.26)
* [x] /usr/software/ebfiles_repo/GCC/GCC-5.4.0-2.26.eb (module: Core | 
GCC/5.4.0-2.26)
* [x] /usr/software/ebfiles_repo/ncurses/ncurses-6.0-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | ncurses/.6.0)
* [x] /usr/software/ebfiles_repo/Tcl/Tcl-8.6.5-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | Tcl/8.6.5)
* [x] /usr/software/ebfiles_repo/bzip2/bzip2-1.0.6-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | bzip2/.1.0.6)
* [x] /usr/software/ebfiles_repo/libreadline/libreadline-6.3-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | libreadline/.6.3)
* [x] /usr/software/ebfiles_repo/SQLite/SQLite-3.13.0-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | SQLite/3.13.0)
* [x] /usr/software/ebfiles_repo/Tk/Tk-8.6.5-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | Tk/.8.6.5)
* [x] /usr/software/ebfiles_repo/libffi/libffi-3.2.1-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | libffi/.3.2.1)
* [x] /usr/software/ebfiles_repo/Autoconf/Autoconf-2.69-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | Autoconf/.2.69)
* [ ] /home/vanzod/OpenSSL-1.1.0c-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | OpenSSL/.1.1.0c)
* [x] /usr/software/ebfiles_repo/Automake/Automake-1.15-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | Automake/.1.15)
* [ ] 
/gpfs0/software/centos6/software/Core/EasyBuild/software/EasyBuild/3.0.2/lib/python2.6/site-packages/easybuild_easyconfigs-3.0.2-py2.6.egg/easybuild/easyconfigs/n/ncurses/ncurses-6.0.eb
 (module: Core | ncurses/.6.0)
* [ ] 
/gpfs0/software/centos6/software/Core/EasyBuild/software/EasyBuild/3.0.2/lib/python2.6/site-packages/easybuild_easyconfigs-3.0.2-py2.6.egg/easybuild/easyconfigs/g/gettext/gettext-0.19.8.eb
 (module: Core | gettext/.0.19.8)
* [x] /usr/software/ebfiles_repo/libtool/libtool-2.4.6-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | libtool/.2.4.6)
* [x] /usr/software/ebfiles_repo/Autotools/Autotools-20150215-GCC-5.4.0-2.26.eb 
(module: Compiler/GCC/5.4.0-2.26 | Autotools/.20150215)
* [x] /usr/software/ebfiles_repo/GMP/GMP-6.1.1-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | GMP/.6.1.1)
* [ ] 
/gpfs0/software/centos6/software/Core/EasyBuild/software/EasyBuild/3.0.2/lib/python2.6/site-packages/easybuild_easyconfigs-3.0.2-py2.6.egg/easybuild/easyconfigs/x/XZ/XZ-5.2.2-GCC-5.4.0-2.26.eb
 (module: Compiler/GCC/5.4.0-2.26 | XZ/.5.2.2)
* [x] 
/usr/software/ebfiles_repo/OpenBLAS/OpenBLAS-0.2.18-GCC-5.4.0-2.26-LAPACK-3.6.1.eb
 (module: Compiler/GCC/5.4.0-2.26 | OpenBLAS/0.2.18-LAPACK-3.6.1)
* [ ] 
/gpfs0/software/centos6/software/Core/EasyBuild/software/EasyBuild/3.0.2/lib/python2.6/site-packages/easybuild_easyconfigs-3.0.2-py2.6.egg/easybuild/easyconfigs/n/numactl/numactl-2.0.11-GCCcore-5.4.0.eb
 (module: Compiler/GCCcore/5.4.0 | numactl/.2.0.11)
* [x] /usr/software/ebfiles_repo/hwloc/hwloc-1.11.3-GCC-5.4.0-2.26.eb (module: 
Compiler/GCC/5.4.0-2.26 | hwloc/.1.11.3)
* [x] 

Re: [easybuild] Intel Parallel Studio components naming convention

[Vanzo, Davide]

Ole,
I think that this is more a docs issue than the specific easyconfig files. Also 
because it applies to any software that requires a license.
I will fork the docs repo, add the info and generate a PR.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Nov 4 2016, at 2:21 am, Ole Holm Nielsen <ole.h.niel...@fysik.dtu.dk> wrote:

Hi Davide,

When you update EB files, could you kindly add an explanation of the
variable INTEL_LICENSE_FILE in each EB file, given the fact that it
doesn't seems to be documented elsewhere and everybody will need to
customize it?

My suggestion of text for adding to all Intel EB files:

# Intel license_file must be specified. Select a long-term valid name.
# Can be overridden by setting the environment variable
# INTEL_LICENSE_FILE before running eb.
# Absolute path example:
# license_file = HOME + '/licenses/intel/license.lic'
# License server example:
# license_file = port-number@license-server

On 11/03/2016 11:10 PM, Vanzo, Davide wrote:
> as you have probably seen I am updating the easybuild files for the new
> Intel Parallel Studio XE 2017 Update 1.
> I started working on the other tools in the suite other than the main
> ones (compilers, MPI and MKL) and I am facing a naming dilemma. All main
> components easyconfigs use versioning in the 2017.1 form. That is great
> because it allows me to use version_major and version_minor to generate
> the source tarball name. The additional components instead use the
> "update1" version naming which brakes consistency across the whole
> suite. What would you think about adopting a uniform naming convention
> across the whole suite (and maybe also for the toolchains that use a
> preceding zero to the minor version number)?

Thanks,
Ole

[easybuild] Intel Parallel Studio components naming convention

[Vanzo, Davide]

Hello,
as you have probably seen I am updating the easybuild files for the new Intel 
Parallel Studio XE 2017 Update 1.
I started working on the other tools in the suite other than the main ones 
(compilers, MPI and MKL) and I am facing a naming dilemma. All main components 
easyconfigs use versioning in the 2017.1 form. That is great because it allows 
me to use version_major and version_minor to generate the source tarball name. 
The additional components instead use the "update1" version naming which brakes 
consistency across the whole suite. What would you think about adopting a 
uniform naming convention across the whole suite (and maybe also for the 
toolchains that use a preceding zero to the minor version number)?

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] Has anyone created EB files for the Intel 2017 compilers?

[Vanzo, Davide]

Pablo,
you can find the updated easyconfigs in this PR:

https://github.com/hpcugent/easybuild-easyconfigs/pull/3757

Don't forget to apply the easyblocks changes from this PR:

https://github.com/hpcugent/easybuild-easyblocks/pull/1012

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] Has anyone created EB files for the Intel 2017 compilers?

[Vanzo, Davide]

Pablo,

I am trying to install the 2017.1 version and I will be happy to share as soon 
as I fix a little issue with the MKL.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Nov 3 2016, at 6:23 am, Pablo Escobar Lopez  
wrote:
Hi Ole,

Could you share your iomkl easyconfig using the latest intel2017 compiler?

regards,
Pablo.

2016-11-03 12:07 GMT+01:00 Fotis Georgatos 
>:
Hi Ole, all,

On Nov 2, 2016, at 2:33 PM, Ole Holm Nielsen 
> wrote:
> On 11/02/2016 12:52 PM, Åke Sandgren wrote:
>> env INTEL_LICENSE_FILE=port@host eb intel-2016a.eb ...
>
> Fantastic, this worked for me!  I wonder where the usage of license_file 
> and/or INTEL_LICENSE_FILE within EB might be documented?

It SHOULD ;-)

There’s one caveat with the port@host approach:

One day, Intel’s practices will force you to setup a second license server
(either because you need to setup a new flexlm server version or,
because Intel has the nasty habit of silently deprecating old components in new 
licenses
and you need to point to a test instance without touching your production 
instance yet)

So, it’s more beneficial to point to a file, so that you can easily redirect 
clients
to a new server instance, from a *single* point of control - without touching 
the modulefiles at all!
If you have 100s of Intel modulefiles in module buildset trees, you know what 
I’m talking about!

Also one more potential caveat: although it IS possible to drop in that one 
just the first 2 lines
of the license file (ie. license server coordinates) don’t do that either and 
use the whole thing:
I’ve met situations where certain Intel components (not all) have bugs and 
don’t cooperate well.
I no longer have the Intel ticket numbers but you can take my word for it: I’ve 
spend endless hours around it.

Last but not least, something I’ve not tried yet: you could be pointing instead 
to a licensefile
sitting on /dev/shm, presumably populated at node boot time, so that you can 
have more granularity
of control - at node level rather than cluster. This would permit you fi. to 
modify node behaviour
in a rolling update fashion. I haven’t yet been hit by the absence of this but 
it’s a nice to have.

Fotis

--
echo "sysadmin know better bash than english" | sed s/min/mins/ \
  | sed 's/better bash/bash better/' # signal detected in a CERN forum










--
Pablo Escobar López
HPC systems engineer
sciCORE, University of Basel
SIB Swiss Institute of Bioinformatics
http://scicore.unibas.ch

Re: [easybuild] Comments in patches

[Vanzo, Davide]

Hi Kenneth,

Thanks for the reply. I truly appreciate all your effort and my initial comment 
was not meant to be a rant but just a feedback.
In terms of policy, I believe that one of the things that would be very 
important to add to patches headers is if it is an EB related pacth (as in the 
numpy case) or if it is a bug fix patch taken from a specific source (I have 
seen some with it).

Davide


On Nov 2 2016, at 8:49 am, Kenneth Hoste  wrote:

Hi Davide,

I think the numpy patch for MKL is required because the libraries that provide 
BLAS/LAPACK support in Intel MKL are spread out across multiple directories, 
which is not taken into account by the numpy build procedure.

As was said, we have tried to make sure some context/comments are included in 
every patch file recently (and we could/should probably enforce that via a 
dedicated test in the easyconfigs test suite...).

regards,

Kenneth

> On 29 Oct 2016, at 18:54, Fotis Georgatos  wrote:
>
>
>> On Oct 28, 2016, at 8:36 AM, Ward Poelmans  wrote:
>> Your absolutely right and we try to do this rigorously for all new
>> patches. However for older patches, this is not always the case (as you
>> have found).
>>
>> If you figure it out, don't hesitate to send a PR with a comment on what
>> it does.
>
> +1
>
> Also, several of these patches come from 3rd party sources, findable via a 
> URL;
>
> I was also trying in the early days to at least leave that as minimum 
> reference,
> since it often describes the context of the patch (what, why, how etc) and, 
> it is
> especially important for understanding which patch suites which software 
> version.
> If possible, please include those URLs in your headers!
>
> Fotis
>
> --
> echo "sysadmin know better bash than english" | sed s/min/mins/ \
> | sed 's/better bash/bash better/' # signal detected in a CERN forum
>
>
>
>
>
>
>

Re: [easybuild] Problem with include and lib paths for python modules

[Vanzo, Davide]

Hi Kenneth,
I will look into Quassel, although it still seem that Quassel Core still needs 
to be hosted on a system that is always connected too. I will have to arrange 
something... Anyway, thanks for the suggestion.

Back to our problem. For numpy I used a modified version of 
numpy-1.8.2-foss-2016a-Python-2.7.11.eb where the only changes were in the 
toolchain (goolf instead of foss) and Python version. Attached you can find the 
easyconfig and the resulting lua module. The path to the numpy include 
directory does not appear on CPATH once loading the module.

$ ml goolf
$ ml numpy
$ ml

Currently Loaded Modules:
 1) GCCcore/.5.4.0   4) numactl/.2.0.11   7) OpenBLAS/0.2.19-LAPACK-3.6.1   
   10) goolf/2016.10  13) ncurses/.6.0  16) SQLite/3.15.0  19) 
libffi/.3.2.122) numpy/1.11.2-Python-2.7.12
 2) binutils/.2.26   5) hwloc/.1.11.4 8) FFTW/3.3.5 
   11) bzip2/.1.0.6   14) libreadline/.7.0  17) Tk/8.6.6   20) 
OpenSSL/.1.0.2j
 3) GCC/5.4.0-2.26   6) OpenMPI/1.10.49) 
ScaLAPACK/2.0.2-OpenBLAS-0.2.19-LAPACK-3.6.1  12) zlib/.1.2.815) Tcl/8.6.6  
   18) GMP/.6.1.1 21) Python/2.7.12

$ echo $CPATH | tr ':' '\n'
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/OpenSSL/1.0.2j/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/GMP/6.1.1/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Tk/8.6.6/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/SQLite/3.15.0/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Tcl/8.6.6/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/libreadline/7.0/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/ncurses/6.0/include
/opt/easybuild/software/Compiler/GCCcore/5.4.0/zlib/1.2.8/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/bzip2/1.0.6/include
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/ScaLAPACK/2.0.2-OpenBLAS-0.2.19-LAPACK-3.6.1/include
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/FFTW/3.3.5/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/OpenBLAS/0.2.19-LAPACK-3.6.1/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/OpenMPI/1.10.4/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/hwloc/1.11.4/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/numactl/2.0.11/include
/opt/easybuild/software/Compiler/GCCcore/5.4.0/binutils/2.26/include
/opt/easybuild/software/Core/GCCcore/5.4.0/include


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Nov 2 2016, at 8:28 am, Kenneth Hoste <kenneth.ho...@ugent.be> wrote:

Hi Davide,

> On 1 Nov 2016, at 18:07, Vanzo, Davide <davide.va...@vanderbilt.edu> wrote:
>
> Hello,
> I am moving this discussion over here since it is hard to follow up on the 
> IRC channel (I don't have a computer online 24/7 to set up a bouncer and the 
> time zone differential from Europe does not help either).

You could use Quassel, then you have a 'server' component that is always 
online, and you use a client to connect to it (talk to Pablo on the #easybuild 
IRC channel, he can hook you up). That doesn't fix the timezone issue though. 
;-)

So, it makes sense to move the discussion here.

> I am trying to build ScientificPython and it fails with the following error:
>
> Include/Scientific/arrayobject.h:2:30: fatal error: numpy/oldnumeric.h: No 
> such file or directory
>
> The reason can be found in the fact that in toolchain.py it looks for 
> /include, /lib and /lib64 to obtain the paths 
> that need to be passed to the compiler.

That's not entirely accurate... The toolchain mechanism does define the build 
environment ($CC, $CFLAGS, ...), but $CPATH (which specifies the locations that 
the compiler should consider for header files) is defined in the generated 
module files.

> Although those are the default paths for the majority of programs, python 
> modules have a different path hierarchy and it fails because for numpy the 
> headers are in /lib/python2.7/site-packages/numpy/core/include. See 
> the following debug log extract for evidence.
>
> So, how would you suggest to proceed? One way would be to have a recursive 
> search starting from  coupled with a more tight check on the 
> lib/include paths based not only on the directory name but also on its 
> content (are there .a, .o or .h files?). That would also solve the problem of 
> false positive entries like /lib that does not contain library 
> files at all.

I'm actually quite surprised this only pops up now... Doesn't the module file 
for numpy correctly define $CPATH? (can't easily check myself atm)

Which easyconfig file are you using exactly? Maybe this is just an issue with a

[easybuild] Problem with include and lib paths for python modules

[Vanzo, Davide]

Hello,
I am moving this discussion over here since it is hard to follow up on the IRC 
channel (I don't have a computer online 24/7 to set up a bouncer and the time 
zone differential from Europe does not help either).

I am trying to build ScientificPython and it fails with the following error:

  Include/Scientific/arrayobject.h:2:30: fatal error: 
numpy/oldnumeric.h: No such file or directory

The reason can be found in the fact that in toolchain.py it looks for 
/include, /lib and /lib64 to obtain the paths 
that need to be passed to the compiler. Although those are the default paths 
for the majority of programs, python modules have a different path hierarchy 
and it fails because for numpy the headers are in 
/lib/python2.7/site-packages/numpy/core/include. See the following 
debug log extract for evidence.

So, how would you suggest to proceed? One way would be to have a recursive 
search starting from  coupled with a more tight check on the 
lib/include paths based not only on the directory name but also on its content 
(are there .a, .o or .h files?). That would also solve the problem of false 
positive entries like /lib that does not contain library files at 
all.

Davide



== 2016-11-01 12:05:51,970 toolchain.py:218 DEBUG set_variables: toolchain 
variables. Do nothing.
== 2016-11-01 12:05:51,970 toolchain.py:647 DEBUG prepare: set additional 
variables onlymod=False
== 2016-11-01 12:05:51,970 toolchain.py:270 DEBUG get_software_root software 
root /opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12 for Python 
was found in environment
== 2016-11-01 12:05:51,971 toolchain.py:270 DEBUG get_software_root software 
root 
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12
 for numpy was found in environment
== 2016-11-01 12:05:51,971 variables.py:542 DEBUG Passthrough to LISTCLASS 
element function append_subdirs
== 2016-11-01 12:05:51,971 variables.py:267 DEBUG _is_protected: False value 
None (None)
== 2016-11-01 12:05:51,971 variables.py:297 DEBUG nappend: value None newvalue 
[] position None
== 2016-11-01 12:05:51,971 variables.py:186 DEBUG append_subdirs: base 
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12 subdirs 
['include']
== 2016-11-01 12:05:51,972 variables.py:198 DEBUG append_subdirs: added 
directory /opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12/include
== 2016-11-01 12:05:51,972 variables.py:542 DEBUG Passthrough to LISTCLASS 
element function append_subdirs
== 2016-11-01 12:05:51,972 variables.py:267 DEBUG _is_protected: False value 
None (None)
== 2016-11-01 12:05:51,972 variables.py:297 DEBUG nappend: value None newvalue 
[] position None
== 2016-11-01 12:05:51,972 variables.py:186 DEBUG append_subdirs: base 
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12 subdirs ['lib64', 
'lib']
== 2016-11-01 12:05:51,973 variables.py:200 WARNING flags_for_subdirs: 
directory /opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12/lib64 
was not found
== 2016-11-01 12:05:51,973 variables.py:198 DEBUG append_subdirs: added 
directory /opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12/lib
== 2016-11-01 12:05:51,973 variables.py:542 DEBUG Passthrough to LISTCLASS 
element function append_subdirs
== 2016-11-01 12:05:51,973 variables.py:267 DEBUG _is_protected: False value 
None (None)
== 2016-11-01 12:05:51,973 variables.py:297 DEBUG nappend: value None newvalue 
[] position None
== 2016-11-01 12:05:51,974 variables.py:186 DEBUG append_subdirs: base 
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12
 subdirs ['include']
== 2016-11-01 12:05:51,974 variables.py:200 WARNING flags_for_subdirs: 
directory 
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12/include
 was not found
== 2016-11-01 12:05:51,974 variables.py:542 DEBUG Passthrough to LISTCLASS 
element function append_subdirs
== 2016-11-01 12:05:51,974 variables.py:267 DEBUG _is_protected: False value 
None (None)
== 2016-11-01 12:05:51,974 variables.py:297 DEBUG nappend: value None newvalue 
[] position None
== 2016-11-01 12:05:51,975 variables.py:186 DEBUG append_subdirs: base 
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12
 subdirs ['lib64', 'lib']
== 2016-11-01 12:05:51,975 variables.py:200 WARNING flags_for_subdirs: 
directory 
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12/lib64
 was not found
== 2016-11-01 12:05:51,975 variables.py:198 DEBUG append_subdirs: added 
directory 
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12/lib

Re: [easybuild] Comments in patches

[Vanzo, Davide]

Ward,
I sure will.

Davide


On Oct 28 2016, at 2:36 am, Ward Poelmans <ward.poelm...@ugent.be> wrote:

On 27-10-16 20:57, Vanzo, Davide wrote:
> Hello all,
> I don't know you but I would find very useful to have some introductory
> comment on the patch files that explain what it actually does. For
> example I am trying to build numpy 1.11.2 with GCC 5.4.0 and most of the
> easyconfig files in the default repo use some patch files to use MKL. I
> am having some hard time understanding the reason for all those changes...

Hi Davide,

Your absolutely right and we try to do this rigorously for all new
patches. However for older patches, this is not always the case (as you
have found).

If you figure it out, don't hesitate to send a PR with a comment on what
it does.

Ward

Re: [easybuild] Proposal for collecting use cases

[Vanzo, Davide]

Kenneth,
thanks for supporting my idea. It has been very helpful to see how you use EB 
at UGent and I am curious to know more about other centers.
When we will reach production I will certainly contribute to the section.

Davide


On Oct 23 2016, at 2:03 pm, Kenneth Hoste <kenneth.ho...@ugent.be> wrote:
Hi Davide,

On 21/10/16 23:23, Vanzo, Davide wrote:

Hello guys,
We are experimenting with EB to figure out the best way to use it in our 
cluster environment. One of the (very good) things about EB is that it is very 
flexible and each center can adopt a different strategy to build software via 
EB. This is pretty evident in this mailing list.
It would then be very helpful for new centers that are planning to deploy 
software with EB to have a section of the documentation that collects 
configuration and use cases from multiple centers that already adopt EB in 
production.

This is an excellent idea!
I have been wanting to add a list of sites that are using EasyBuild to the 
documentation, but I never got around to it...

Your proposal takes that one step further, and I think you're right, this would 
be very valuable for other sites, both newcomers and people already using 
EasyBuild for a while.

I spent some time on putting this together, and fleshed out a page for 
HPC-UGent already which can be used as example.

See https://github.com/hpcugent/easybuild/pull/264.
Please take a look, and let me know what you think.

If anyone want to add a page for their site as well, please send me it to me 
one way or another (a pull request, via mail, etc.).


regards,

Kenneth



Have a great weekend!

--
Davide Vanzo, PhD
Application Developer
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu<http://www.accre.vanderbilt.edu>

Re: [easybuild] easybuild -devel files for users?

[Vanzo, Davide]

Absolutely. Just trying to get a feeling on your thoughts about this potential 
broader implementation of the functionality.

DV

On October 22, 2016 10:38:08 Jack Perdue <j-per...@tamu.edu> wrote:

On 10/22/2016 09:09 AM, Vanzo, Davide wrote:

>From a user perspective it would be more interesting to have this
option under Lmod than EB. The end user is most likely to interact
only with the environment modules manager than EB directly. The idea
would be that the user loads all the modules he/she needs according to
a specific toolchain and can get all the linking flags by calling
something like:

$module -L
In this way the user can use command substitution directly into
Makefiles. More importantly, this should also take into account
ordering the lib flags accordingly since sometimes order is essential
for successful linking.
That is something that we discussed internally since this
functionality is provided by our current modules manager (although in
a very primitive way) and would be lost when transitioning to Lmod.

That would all be up to Robert for the most part.
But even if such an option existed it would still need
EB to feed it what it needs to work.

jack

[easybuild] Proposal for collecting use cases

[Vanzo, Davide]

Hello guys,
We are experimenting with EB to figure out the best way to use it in our 
cluster environment. One of the (very good) things about EB is that it is very 
flexible and each center can adopt a different strategy to build software via 
EB. This is pretty evident in this mailing list.
It would then be very helpful for new centers that are planning to deploy 
software with EB to have a section of the documentation that collects 
configuration and use cases from multiple centers that already adopt EB in 
production.

Have a great weekend!

--
Davide Vanzo, PhD
Application Developer
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] Configure options as list

[Vanzo, Davide]

Apparently I was wrong. EB does four builds, each configured with the 
corresponding configopts line.

DV


On Oct 20 2016, at 1:03 pm, Vanzo, Davide <davide.va...@vanderbilt.edu> wrote:
Hello,
It is clear to me that in an easyconfig for a ConfigureMake easyblock the 
configopts variable contains the options for the configuration step. However it 
is not fully clear to me what happens when instead of a single string 
configopts is a list of strings, like in this case taken from 
FFTW-3.3.5-gompi-2016.07.eb:

configopts = [
   common_configopts + " --enable-single --enable-sse2 --enable-mpi",
   common_configopts + " --enable-long-double --enable-mpi",
   common_configopts + " --enable-quad-precision",
   common_configopts + " --enable-sse2 --enable-mpi",  # default as last
]

My best guess at the moment is that EB will try one string at a time and, if 
the configure step fails, it will try the next in the list. Am I correct?

I tried to look in the doc but I could not find an answer. Unless I missed it 
(higly probable) I would suggest adding a couple of lines under 
"Configure/build/install command options" .

Thanks!

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] Automatic detection of hidden modules

[Vanzo, Davide]

Markus,
you are absolutely right.
The only way to do what I want is the following:

$ eb GCC-5.4.0-2.26.eb -D --hide-deps=M4,Bison,flex,zlib,binutils,GCCcore 
--hide-toolchains=GCCcore

or to specify the dependencies that need to be hidden in the 
EASYBUILD_HIDE_DEPS environment variable.
I think that at the moment it is good enough for me.

DV


On Oct 18 2016, at 12:41 pm, Markus Geimer  wrote:

Alan, Davide,

> hidden = True is now supported within EB (see
> https://github.com/hpcugent/easybuild-framework/pull/1837), just add
> that to your GCCcore eb file

Since GCCcore is a toolchain, '--hide-deps' or 'hidden = True' alone
doesn't help. 'hidden = True' only applies to the installation itself,
but not necessarily when the package is used as a dependency (see
below). And '--hide-deps' only applies if GCCcore is used as a
dependency, but not when used as a toolchain. Therefore, EB 2.9.0
also supports a '--hide-toolchains' option, which should help in this
case. (Note that GCCcore needs to be marked as hidden in both ways.)

> If so, is there any chance we will see such feature (maybe not by
> default but as an eb option flag) in the future?
>
> If not, what would you suggest? Obviously I could manually edit all
> easyconfig files by adding the hiddendependencies but that may be
> quite annoying in the long term.

If you list your easyconfig archive and/or "golden repo" before the
easyconfig dir of your EB install in robot path, EasyBuild will look
at those first and correctly handles packages including 'hidden=True'
even w/o listing them in hiddendependencies. However, for other
robot-path setups, an 'automatically consider hidden modules' feature
is still missing in EasyBuild. Let me use Kenneth's favorite in such
cases: "It should not be too hard to implement this..." ;-)

Markus

--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49-2461-61-1773
Fax: +49-2461-61-6656
E-Mail: m.gei...@fz-juelich.de
Web: http://www.fz-juelich.de/jsc



Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Automatic detection of hidden modules

[Vanzo, Davide]

Great! Thank you guys.

DV

On Oct 18 2016, at 12:36 pm, Bart Oldeman <bart.olde...@mcgill.ca> wrote:
Also you can put that list in a configuration file, e.g. I have a config.cfg 
file with
hide-deps = icc,ifort
and use
EASYBUILD_CONFIGFILES
to point to that file. That is good for the environment.

On 18 October 2016 at 13:26, Alan O'Cais 
<a.oc...@fz-juelich.de<mailto:a.oc...@fz-juelich.de>> wrote:
hidden = True
is now supported within EB (see 
https://github.com/hpcugent/easybuild-framework/pull/1837), just add that to 
your GCCcore eb file

We have a long list of software packages hidden by default, this is the setting 
you also might suit you:
-bash-4.2$ echo $EASYBUILD_HIDE_DEPS
ANTLR,APR,APR-util,AT-SPI2-ATK,AT-SPI2-core,ATK,Autoconf,Automake,Bison,CUSP,Coreutils,DB,DBus,DocBook-XML,Dyninst,ETSF_IO,FFmpeg,FLTK,FTGL,GCCcore,GDAL,GL2PS,GLEW,GLib,GLPK,GPC,GObject-Introspection,GTI,GTK+,GTS,Gdk-Pixbuf,Ghostscript,GraphicsMagick,GtkSourceView,HarfBuzz,JUnit,JasPer,LibTIFF,LibUUID,Libint,M4,Mesa,NASM,OPARI2,OTF2,PCRE,PDT,PROJ,Pango,PnMPI,PyCairo,PyGObject,PyQt,Qhull,Qt,Qt5,S-Lang,SCons,SIP,SQLite,SWIG,Serf,Szip,Tcl,Tk,UDUNITS,XML-Parser,XZ,XKeyboardConfig,YAXT,Yasm,adwaita-icon-theme,ant,assimp,binutils,byacc,bzip2,cairo,dbus-glib,damageproto,eudev,expat,g2clib,g2lib,gc,glproto,gperf,guile,grib_api,gsettings-desktop-schemas,fixesproto,fontsproto,fontconfig,freeglut,freetype,gettext,icc,ifort,inputproto,intltool,itstool,jhbuild,kbproto,libGLU,libICE,libSM,libX11,libXau,libXaw,libXcursor,libXdamage,libXdmcp,libXext,libXfixes,libXfont,libXft,libXi,libXinerama,libXmu,libXpm,libXrandr,libXrender,libXt,libXtst,libcerf,libcroco,libctl,libdap,libdrm,libdwarf,libelf,libevent,libffi,libfontenc,libgd,libgeotiff,libglade,libidn,libjpeg-turbo,libmatheval,libpng,libpciaccess,libpthread-stubs,libreadline,librsvg,libtool,libunistring,libunwind,libyaml,libxcb,libxkbcommon,libxml2,libxslt,makedepend,motif,ncurses,pixman,pkg-config,pkgconfig,popt,pscom,qrupdate,randrproto,recordproto,renderproto,scrollkeeper,texinfo,util-linux,wxPropertyGrid,wxWidgets,x264,xbitmaps,xcb-proto,xcb-util,xcb-util-image,xcb-util-keysyms,xcb-util-renderutil,xcb-util-wm,xextproto,xineramaproto,xorg-macros,xprop,xproto,xtrans,zlib



On 18 October 2016 at 18:18, Vanzo, Davide 
<davide.va...@vanderbilt.edu<mailto:davide.va...@vanderbilt.edu>> wrote:
Hello,
Although we wold love to use EB in production there is still one thing that 
stops us from doing so. Since we need to keep the modules list in Lmod as clean 
as possible it is essential for us to be able to tell EB to automatically look 
for dependencies within hidden modules even if not explicitly listed as 
hiddendependencies in the easyconfig files. Let me show you with the case I am 
struggling with.

I want to install GCC but keep the GCCcore module hidden. As you can see from 
the output below if I ask to hide GCCcore it gets built twice and two modules 
get generated: one hidden and one visible. My understanding is that it happens 
because when rebuilding all dependencies with GCCcore (instead of the system 
GCC) they don't have GCCcore within a hiddendependencies list and since EB does 
not automatically scan for hidden modules it builds GCCcore again. Am I right?

If so, is there any chance we will see such feature (maybe not by default but 
as an eb option flag) in the future?

If not, what would you suggest? Obviously I could manually edit all easyconfig 
files by adding the hiddendependencies but that may be quite annoying in the 
long term.

In addition, what do you think about being able to specify the "hide-deps" list 
directly into the easyconfig?

Thanks!


$ eb GCC-5.4.0-2.26.eb -D --hide-deps=M4,Bison,flex,zlib,binutils,GCCcore
== temporary log file in case of crash /tmp/eb-bLnY30/easybuild-wrqlUH.log
Dry run: printing build status of easyconfigs and dependencies
CFGS=/opt/easybuild/software/EasyBuild/2.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-2.9.0-py2.7.egg/easybuild/easyconfigs
* [ ] $CFGS/m/M4/M4-1.4.17.eb (module: Core | M4/.1.4.17)
* [ ] $CFGS/b/Bison/Bison-3.0.4.eb (module: Core | Bison/.3.0.4)
* [ ] $CFGS/f/flex/flex-2.6.0.eb (module: Core | flex/.2.6.0)
* [ ] $CFGS/z/zlib/zlib-1.2.8.eb (module: Core | zlib/.1.2.8)
* [ ] $CFGS/b/binutils/binutils-2.26.eb (module: Core | binutils/.2.26)
* [ ] $CFGS/g/GCCcore/GCCcore-5.4.0.eb (module: Core | GCCcore/.5.4.0)
* [ ] $CFGS/g/GCCcore/GCCcore-5.4.0.eb (module: Core | GCCcore/5.4.0)
* [ ] $CFGS/z/zlib/zlib-1.2.8-GCCcore-5.4.0.eb (module: Compiler/GCCcore/5.4.0 
| zlib/.1.2.8)
* [ ] $CFGS/m/M4/M4-1.4.17-GCCcore-5.4.0.eb (module: Compiler/GCCcore/5.4.0 | 
M4/.1.4.17)
* [ ] $CFGS/b/Bison/Bison-3.0.4-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | Bison/.3.0.4)
* [ ] $CFGS/f/flex/flex-2.6.0-GCCcore-5.4.0.eb (module: Compiler/GCCcore/5.4.0 
| flex/.2.6.0)
* [ ] $CFGS/b/binutils/binutils-2.26-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | binutils/.2.26)
* [ ] $CFGS/g/G

[easybuild] Automatic detection of hidden modules

[Vanzo, Davide]

Hello,
Although we wold love to use EB in production there is still one thing that 
stops us from doing so. Since we need to keep the modules list in Lmod as clean 
as possible it is essential for us to be able to tell EB to automatically look 
for dependencies within hidden modules even if not explicitly listed as 
hiddendependencies in the easyconfig files. Let me show you with the case I am 
struggling with.

I want to install GCC but keep the GCCcore module hidden. As you can see from 
the output below if I ask to hide GCCcore it gets built twice and two modules 
get generated: one hidden and one visible. My understanding is that it happens 
because when rebuilding all dependencies with GCCcore (instead of the system 
GCC) they don't have GCCcore within a hiddendependencies list and since EB does 
not automatically scan for hidden modules it builds GCCcore again. Am I right?

If so, is there any chance we will see such feature (maybe not by default but 
as an eb option flag) in the future?

If not, what would you suggest? Obviously I could manually edit all easyconfig 
files by adding the hiddendependencies but that may be quite annoying in the 
long term.

In addition, what do you think about being able to specify the "hide-deps" list 
directly into the easyconfig?

Thanks!


$ eb GCC-5.4.0-2.26.eb -D --hide-deps=M4,Bison,flex,zlib,binutils,GCCcore
== temporary log file in case of crash /tmp/eb-bLnY30/easybuild-wrqlUH.log
Dry run: printing build status of easyconfigs and dependencies
CFGS=/opt/easybuild/software/EasyBuild/2.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-2.9.0-py2.7.egg/easybuild/easyconfigs
* [ ] $CFGS/m/M4/M4-1.4.17.eb (module: Core | M4/.1.4.17)
* [ ] $CFGS/b/Bison/Bison-3.0.4.eb (module: Core | Bison/.3.0.4)
* [ ] $CFGS/f/flex/flex-2.6.0.eb (module: Core | flex/.2.6.0)
* [ ] $CFGS/z/zlib/zlib-1.2.8.eb (module: Core | zlib/.1.2.8)
* [ ] $CFGS/b/binutils/binutils-2.26.eb (module: Core | binutils/.2.26)
* [ ] $CFGS/g/GCCcore/GCCcore-5.4.0.eb (module: Core | GCCcore/.5.4.0)
* [ ] $CFGS/g/GCCcore/GCCcore-5.4.0.eb (module: Core | GCCcore/5.4.0)
* [ ] $CFGS/z/zlib/zlib-1.2.8-GCCcore-5.4.0.eb (module: Compiler/GCCcore/5.4.0 
| zlib/.1.2.8)
* [ ] $CFGS/m/M4/M4-1.4.17-GCCcore-5.4.0.eb (module: Compiler/GCCcore/5.4.0 | 
M4/.1.4.17)
* [ ] $CFGS/b/Bison/Bison-3.0.4-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | Bison/.3.0.4)
* [ ] $CFGS/f/flex/flex-2.6.0-GCCcore-5.4.0.eb (module: Compiler/GCCcore/5.4.0 
| flex/.2.6.0)
* [ ] $CFGS/b/binutils/binutils-2.26-GCCcore-5.4.0.eb (module: 
Compiler/GCCcore/5.4.0 | binutils/.2.26)
* [ ] $CFGS/g/GCC/GCC-5.4.0-2.26.eb (module: Core | GCC/5.4.0-2.26)
== Temporary log file(s) /tmp/eb-bLnY30/easybuild-wrqlUH.log* have been removed.
== Temporary directory /tmp/eb-bLnY30 has been removed.


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

Re: [easybuild] caffe/mxnet/tensorflow easyconfigs

[Vanzo, Davide]

Hi all,
glad to hear there is somebody else interested in building DNN training 
software with EB.
I will be working on that too so please keep me in the loop and I will be glad 
to help out.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Oct 18 2016, at 8:16 am, Peretti-Pezzi Guilherme  wrote:

Hi

We used EB for all the dependencies (Python, Swig, PCRE, Bazel) when
building a GPU-enabled TF 0.9.0 on our Cray systems.

Building TF itself required a couple of ugly tweaks (see [1]), I was not
convinced that it would work so I didn’t bother to create the .eb for
that.

So far the users didn’t complain much about this build, so I assume it is
working. You can find the .eb files we used for Python, Swig, PCRE and
Bazel on [2].

I hope it helps.

--
Cheers,
G.

[1]
https://github.com/eth-cscs/TensorFlow/wiki/TensorFlow-0.9.0-Daint-GPU-buil
d
[2] https://github.com/eth-cscs/tools/tree/master/easybuild/easyconfigs

-Original Message-
From:  on behalf of Erik Smeets

Reply-To: "easybuild@lists.ugent.be" 
Date: Tuesday 18 October 2016 12:50
To: "easybuild@lists.ugent.be" 
Subject: [easybuild] caffe/mxnet/tensorflow easyconfigs

>Hello,
>
>Has anyone got (working) easyconfigs for caffe, mxnet and tensorflow they
>are willing to share? Any work in progress is also appreciated, so we
>could help with these. For Caffe I've already found
>http://www.siliconslick.com/easybuild/ebfiles_repo_cleaned/ada/Caffe/Caffe
>-rc3-intel-2016a-Python-2.7.11.eb and
>https://github.com/hpcugent/easybuild-easyconfigs/pull/3667/files. The
>first is not working (at least for us) and the second is foss toolchain
>based, while we prefer intel toolchain based.
>
>Thanks in advance.
>Kind regards,
>Erik Smeets
>
>-- The information contained in this communication and any attachments is
>confidential and may be privileged, and is for the sole use of the
>intended recipient(s). Any unauthorized review, use, disclosure or
>distribution is prohibited. Unless explicitly stated otherwise in the
>body of this communication or the attachment thereto (if any), the
>information is provided on an AS-IS basis without any express or implied
>warranties or liabilities. To the extent you are relying on this
>information, you are doing so at your own risk. If you are not the
>intended recipient, please notify the sender immediately by replying to
>this message and destroy all copies of this message and any attachments.
>The sender nor the company/group of companies he or she represents shall
>be liable for the proper and complete transmission of the information
>contained in this communication, or for any delay in its receipt.

Re: [easybuild] Version numbering for new CUDA toolchain

[Vanzo, Davide]

Eliot,
please proceed with the pull request whenever you want.
I am currently on hold until I get some advice on how to deal with naming MPI 
easyconfigs for different networks.
I will be happy to test your pull whenever available.

Davide


On Aug 24 2016, at 9:58 am, Eliot Eshelman <el...@microway.com> wrote:
Thanks for the comments, Davide! I'll use your numbering if no one else objects.

My builds all look good, so I'm ready to submit the pull request. Or were you 
about to do so yourself?

I was hoping Kenneth would jump in and tell us if we're right or wrong. We can 
always adjust before he merges.

Eliot



On 08/23/2016 02:27 PM, Vanzo, Davide wrote:
Eliot,

I started working on the same toolchain and I have the same problem since the 
toolchain version numbering is not very clear. The temporary number I chose is 
3.1.10 for the following reasons:


  *   Since one of the elements got a major update (i.e. CUDA), I bumped the 
toolchain major version to 3
  *   Since GCC got only a minor update with respect to gcccuda-2.6.10, I set 
the minor version to 1
  *   I set the tiny version to 10 to allow other users to downgrade if needed


That was what came out from my personal interpretation of the toolchain version 
rules, but I could be completely wrong.

--
Davide Vanzo, PhD
Application Developer

Adjunct Assistant Professor of Chemical and Biomolecular Engineering

Advanced Computing Center for Research and Education (ACCRE)

Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu<http://www.accre.vanderbilt.edu>

On Aug 23 2016, at 1:15 pm, Eliot Eshelman 
<el...@microway.com><mailto:el...@microway.com> wrote:

Hi Folks,

I could use help with version numbering on a new toolchain.

I manage a GPU cluster and am in the process of adding support for newer
versions of CUDA. I'm working on CUDA 7.5 along with the FOSS tools from
foss/2016a. When CUDA 8.0 comes out (soon) it can be paired with foss/2016b.

The existing toolchain is gcccuda-2.6.10, which includes CUDA 5.5 and
GCC 4.8.2. I'm talking about moving to CUDA 7.5 and GCC 4.9.3-2.25.

In that context, what version number should I give gcccuda? If CUDA 5
was major version 2, then CUDA 6 would be version 3 and CUDA 7 would be
version 4? Bump up to gcccuda-4.0.0? And to gcccuda-5.0.0 when CUDA 8.0
comes out?

Once this is settled I'll submit pull requests on github for gcccuda,
gompic, and goolfc with the newer version of CUDA and suitable GCC,
OpenMPI, etc.

Thanks!
Eliot

--
Eliot Eshelman
Microway, Inc.


--
Eliot Eshelman, Vice President
Strategic Accounts and HPC Initiatives

Microway, Inc.
12 Richards Road, Plymouth, MA 02360
(508) 732-5534
el...@microway.com<mailto:el...@microway.com>
http://www.microway.com

Re: [easybuild] Version numbering for new CUDA toolchain

[Vanzo, Davide]

Eliot,
I started working on the same toolchain and I have the same problem since the 
toolchain version numbering is not very clear. The temporary number I chose is 
3.1.10 for the following reasons:


  *   Since one of the elements got a major update (i.e. CUDA), I bumped the 
toolchain major version to 3
  *   Since GCC got only a minor update with respect to gcccuda-2.6.10, I set 
the minor version to 1
  *   I set the tiny version to 10 to allow other users to downgrade if needed

That was what came out from my personal interpretation of the toolchain version 
rules, but I could be completely wrong.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Aug 23 2016, at 1:15 pm, Eliot Eshelman  wrote:

Hi Folks,

I could use help with version numbering on a new toolchain.

I manage a GPU cluster and am in the process of adding support for newer
versions of CUDA. I'm working on CUDA 7.5 along with the FOSS tools from
foss/2016a. When CUDA 8.0 comes out (soon) it can be paired with foss/2016b.

The existing toolchain is gcccuda-2.6.10, which includes CUDA 5.5 and
GCC 4.8.2. I'm talking about moving to CUDA 7.5 and GCC 4.9.3-2.25.

In that context, what version number should I give gcccuda? If CUDA 5
was major version 2, then CUDA 6 would be version 3 and CUDA 7 would be
version 4? Bump up to gcccuda-4.0.0? And to gcccuda-5.0.0 when CUDA 8.0
comes out?

Once this is settled I'll submit pull requests on github for gcccuda,
gompic, and goolfc with the newer version of CUDA and suitable GCC,
OpenMPI, etc.

Thanks!
Eliot

--
Eliot Eshelman
Microway, Inc.

Re: [easybuild] Find hidden modules by default

[Vanzo, Davide]

Thank you guys for your replies. I'm happy to see I'm not the only one that 
would benefit from this.

I totally agree with Kenneth on how this should be implemented. Am I wrong to 
assume that a positive side effect of this approach is to make the 
"hiddendependencies" option for easyconfig files redundant since EB will 
automatically look for hidden dependencies if no visible ones are found for the 
ones listed in "dependencies" and "builddependencies"?

DV

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Aug 4 2016, at 5:11 am, Kenneth Hoste  wrote:

Hi Markus,

On 04/08/16 10:31, Markus Geimer wrote:
> All,
>
> It seems that with [2] in place, the dependency resolution already
> does "the right thing" as it parses the easyconfig files. But that
> obviously only works if the easyconfig contains the 'hidden'
> parameter. That said, maybe it is sufficient to ensure writing the
> 'hidden' info (wherever it comes from: 'hidden' flag, 'hide-deps',
> '--hidden' command-line option, ...) to the archived easyconfig?
> Or am I missing something?
That's probably only a partial solution, since you have no guarantee
that the archived easyconfig will be picked up the next time the same
installation is needed (the path to the archived easyconfigs is not in
the robot path, by default).

I *think* that making the robot consider also hidden modules is as
simple as updating the 'find_resolved_modules' function to optionally
check also whether the module is available as hidden (which involves
tweaking the dep_mod_name to inject a '.', or something like that).

You may need to make a similar change to make dry-run spit out the
correct info though, which probably means some refactoring to ensure
that both the robot and dry-run use the same function to check whether a
module is available or not...

regards,

Kenneth

>
> Markus
>
>
> [2] https://github.com/hpcugent/easybuild-framework/pull/1837
>
> On 08/04/16 08:58, Markus Geimer wrote:
>> Davide,
>>
>>> The most logical way of doing so seems to be by creating hidden
>>> modules for the dependencies we don't want the users to see. However,
>>> when installing other easyconfig packages from the default easyconfig
>>> files they cannot see the hidden modules and try to install them again.
>>> Is there a way to tell EB to automatically look for hidden packages
>>> without modifying the easyconfig files?
>> At this point not, unfortunately. A corresponding issue has
>> been created already quite some time ago [1], but since then
>> hasn't received much attention. Since I'm very interested in
>> this feature as well and would like to see it implemented
>> rather sooner than later, I recently started looking into it.
>> But I'm still far from a clean and working solution -- though
>> Kenneth claims that it shouldn't be too hard... But that's
>> what he is saying all the time ;-)
>>
>> Markus
>>
>>
>> [1] https://github.com/hpcugent/easybuild-framework/issues/1079
>
> --
> Dr. Markus Geimer
> Juelich Supercomputing Centre
> Institute for Advanced Simulation
> Forschungszentrum Juelich GmbH
> 52425 Juelich, Germany
>
> Phone: +49-2461-61-1773
> Fax: +49-2461-61-6656
> E-mail: m.gei...@fz-juelich.de
> WWW: http://www.fz-juelich.de/jsc/
>
>
>
> 
> 
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
> 
> 
>

[easybuild] Find hidden modules by default

[Vanzo, Davide]

Hello easybuilders,
I am testing EB + Lmod with hierarchical naming and I am trying to reconcile 
the huge amount of modules generated by EB with the need of reducing users' 
confusion by showing only the strictly necessary modules. The most logical way 
of doing so seems to be by creating hidden modules for the dependencies we 
don't want the users to see. However, when installing other easyconfig packages 
from the default easyconfig files they cannot see the hidden modules and try to 
install them again. Is there a way to tell EB to automatically look for hidden 
packages without modifying the easyconfig files?

Thank you

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

[easybuild] Hidden modules

[Vanzo, Davide]

Hello easybuilders,
I am testing EB + Lmod with hierarchical naming and I am trying to reconcile 
the huge amount of modules generated by EB with the need of reducing users' 
confusion by showing only the strictly necessary modules. The most logical way 
of doing so seems to be by creating hidden modules for the dependencies we 
don't want the users to see. However, when installing other easyconfig packages 
from the default easyconfig files they cannot see the hidden modules and try to 
install them again. Is there a way to tell EB to automatically look for hidden 
packages without modifying the easyconfig files?

Thank you

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu