[easybuild] PyPi legacy is gone.... and ideas on source_urls for Python extensions

[Jack Perdue]

As recently as a week or two ago, the "legacy"
PyPi site was still serving files from old source_urls.

That no longer seems to be the case.

And the new URLs seem to be a random mess.

Has anyone pondered how to deail with this?

--
Jack Perdue
Lead Systems Administrator
High Performance Research Computing
TAMU Division of Research
j-per...@tamu.eduhttp://hprc.tamu.edu
HPRC Helpdesk: h...@hprc.tamu.edu

[easybuild] Next EasyBuild conf call: Wed July 11th 2018, 5pm CEST

[Vanzo, Davide]

Dear EasyBuilders,

The next EasyBuild conf call is planned for tomorrow, Wed July 11th 2018, at 
5pm CEST, see also
https://plus.google.com/events/ctbcs6ocbe2gq8f55kslbqfs87g

Current agenda:
* Upcoming EasyBuild 3.6.2 release
* Q&A
As usual, suggestions for additional topics are welcome.
More information on the EasyBuild conf calls is available at
https://github.com/easybuilders/easybuild/wiki/Conference-calls

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.vanderbilt.edu/accre

Re: [easybuild] Loading toolchain module and CC/CXX env vars

[Alan O'Cais]

Hi Paul,

There's a simple easyblock that implments this (for the toolchain):
https://github.com/easybuilders/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/buildenv.py
You can see example uses in
https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs/b/buildenv

Alan

On Tue, 10 Jul 2018 at 15:56, Paul Melis 
mailto:paul.me...@surfsara.nl>> wrote:
Hi,

When developing software (in contrast to simply building and installing
it with EB) it makes sense to leverage the existing toolchains for
setting up a build environment. E.g. I load foss/2017b and other needed
EB-built dependencies while developing and use CMake to build by hand.
When the package gets to a stable point creating an EB config is then
(relatively) easy and predictable, based on the loaded modules used.
Rebuilding by hand is also a lot quicker than doing a full EB build and
install each time.

But loading a toolchain like foss/2017b doesn't completely set up the
environment in a way that makes it directly usable by CMake and friends.
Particularly, CC and CXX are not set so it's up to the build tool to
figure out which compiler to use.

CMake does this somewhat silly in that it searches for (in order) cc,
gcc, cl, bcc, ... when CC is not set (and similarly c++, g++, aCC, cl,
... when CXX is not set). But this is a breadth-first search of "cc" in
a set of directories. So in our case it actually finds /usr/bin/cc (and
/usr/bin/c++ next to it) which are old, GCC 4.8.5, and it doesn't find
the foss/2017b binaries. This appears to be because GCC installations
don't contain the "cc" command (even though they have a "c++" one),
which is what CMake searches for first.

Setting CC and CXX to the foss/2017b compilers will fix this, which kind
of makes me wonder why these aren't set by default when loading
GCC(core) and similar compiler modules? Has this ever been discussed?

Regards,
Paul

--

Paul Melis
| Visualization group leader & developer | SURFsara |
| Science Park 140 | 1098 XG Amsterdam |
| T 020 800 1312 | paul.me...@surfsara.nl | 
www.surfsara.nl |


--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de
WWW:http://www.fz-juelich.de/ias/jsc/EN




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

[easybuild] Loading toolchain module and CC/CXX env vars

[Paul Melis]

Hi,

When developing software (in contrast to simply building and installing 
it with EB) it makes sense to leverage the existing toolchains for 
setting up a build environment. E.g. I load foss/2017b and other needed 
EB-built dependencies while developing and use CMake to build by hand. 
When the package gets to a stable point creating an EB config is then 
(relatively) easy and predictable, based on the loaded modules used. 
Rebuilding by hand is also a lot quicker than doing a full EB build and 
install each time.


But loading a toolchain like foss/2017b doesn't completely set up the 
environment in a way that makes it directly usable by CMake and friends. 
Particularly, CC and CXX are not set so it's up to the build tool to 
figure out which compiler to use.


CMake does this somewhat silly in that it searches for (in order) cc, 
gcc, cl, bcc, ... when CC is not set (and similarly c++, g++, aCC, cl, 
... when CXX is not set). But this is a breadth-first search of "cc" in 
a set of directories. So in our case it actually finds /usr/bin/cc (and 
/usr/bin/c++ next to it) which are old, GCC 4.8.5, and it doesn't find 
the foss/2017b binaries. This appears to be because GCC installations 
don't contain the "cc" command (even though they have a "c++" one), 
which is what CMake searches for first.


Setting CC and CXX to the foss/2017b compilers will fix this, which kind 
of makes me wonder why these aren't set by default when loading 
GCC(core) and similar compiler modules? Has this ever been discussed?


Regards,
Paul

--

Paul Melis
| Visualization group leader & developer | SURFsara |
| Science Park 140 | 1098 XG Amsterdam |
| T 020 800 1312 | paul.me...@surfsara.nl | www.surfsara.nl |

Re: [easybuild] Cuda supporting toolchains

[Alan O'Cais]

Ok, I got to build with a patch from the developers:
https://github.com/easybuilders/easybuild-easyconfigs/pull/6568
I'm checking with them if it'll also work with Flang.
Maybe it's time for a full LLVM toolchain? Flang works with an LLVM fork though 
so in both cases we probably want to do rpath installs so they don't step on 
each others toes.

Alan

On Thu, 5 Jul 2018 at 13:27, Alan O'Cais 
mailto:a.oc...@fz-juelich.de>> wrote:
Have you tried to build polly and GPU support using the EB easyblock? I get 
some test failures with Clang 6.0.0. For the build you need:
```
usepolly = True
configopts = '-DPOLLY_ENABLE_GPGPU_CODEGEN=ON'

# Build capability to target GPUs
build_targets = ['X86', 'NVPTX']
```
and a CUDA dep (not sure if any of those are optional). It results in 9 failing 
tests (at least with an underlying GCC/7.3.0)

Alan


On Thu, 5 Jul 2018 at 10:33, Jure Pečar 
mailto:jurij.pe...@embl.de>> wrote:
On Mon, 19 Mar 2018 14:43:05 +
Joachim Hein mailto:joachim.h...@math.lu.se>> wrote:

> I am wondering how do we want to organise us in future?  Do we want to 
> continue with the goolfc idea or do we go for a “core” cuda and cuDNN?  I 
> feel this needs standardising soonish.

On this topic, does anyone have any opinions about making use of Magma instead 
of standard blas in whatever "cuda" toolchain we come up with?

Even further, is anyone willing to explore this magic:
http://spcl.inf.ethz.ch/Research/Parallel_Programming/Polly-ACC/
There was a talk at ISC claiming they achieve 60% of performance on unmodified 
C code compared to manually ported OpenACC code.


--

  Jurij Pečar
  HPC Engineer, IT Operations, IT Services
  EMBL Heidelberg, Meyerhofstraße 1, 69117, Heidelberg, Germany
  Room 13-401


--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de
WWW:http://www.fz-juelich.de/ias/jsc/EN


--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de
WWW:http://www.fz-juelich.de/ias/jsc/EN




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt