Re: [easybuild] 3rd EasyBuild User Meeting 2018 in Amsterdam (Tue Jan 30-Thu Feb 1 2018)

[John Donners]

Dear EasyBuilders, 

registration for the EasyBuild User Meeting 2018 in Amsterdam is open! 
You can register at: 

[ http://tiny.cc/eum18 | http://tiny.cc/eum18 ] 

* make sure to request an eduroam guest account for WiFi access (unless you 
have eduroam access already). 
* provide your exact name as on your passport if you would like to join the 
data center tour. 

More information about the event can be found at 

https://github.com/easybuilders/easybuild/wiki/3rd-EasyBuild-User-Meeting 

The agenda will follow soon. The SURFsara premises and the (modest) conference 
centre are 
only a 10-minute train ride away from Amsterdam Central. 

We hope to see you soon in Amsterdam! 

Cheers, 
Kenneth (HPC-UGent) 
John (SURFsara) 


From: "John Donners"  
To: "easybuild"  
Sent: Wednesday, 15 November, 2017 15:38:48 
Subject: [easybuild] 3rd EasyBuild User Meeting 2018 in Amsterdam (Tue Jan 
30-Thu Feb 1 2018) 

Dear EasyBuilders, 

we're happy to share that we have confirmed the dates & location for the 
3rd EasyBuild User Meeting early next year... 

location: Amsterdam (Netherlands), co-organised by HPC-UGent & 
SURFsara 
dates: Tue Jan 30th - Thu Feb 1st 2018 (*) 
attendance is free, but registration is required (will be available soon) 

The overall format of the EasyBuild User Meetings is to have a 50-50 
split between presentations & hands-on sessions, where presentations 
cover a variety of topics that are (loosely) related to EasyBuild, and 
the hands-on sessions allow you to work on aspects of EasyBuild you care 
about, with the help from other attendees, including EasyBuild 
developers & maintainers. 

As such, this event is intended for people who are already somewhat 
familiar with EasyBuild. 
You do *not* need to be an EasyBuild expert, but there will not be an 
EasyBuild introductory talk either. 
If you have used EasyBuild before to install software, and if you 
(roughly) know of the EasyBuild terminology (easyconfigs, easyblocks, 
toolchains, ...), then this event is intended for you. 

The agenda is still being planned, but we hope to have talks on: 

* EasyBuild: past, present & future (Kenneth Hoste, HPC-UGent) [confirmed] 
* module tools like Lmod & Tcl environment modules 
* site presentations by attendees on how they integrate EasyBuild in 
their workflow 
* confirmed: SURFsara (John Donners) 
* Singularity, CVMFS, Spack, Nix/Guix for HPC, ... 

The wiki page on the 3rd EasyBuild User Meeting at 
[ https://github.com/easybuilders/easybuild/wiki/3rd-EasyBuild-User-Meeting | 
https://github.com/easybuilders/easybuild/wiki/3rd-EasyBuild-User-Meeting ] 
will be updated as more information becomes available. 

Feel free to contact us for more information. 

regards, 

Kenneth (HPC-UGent, kenneth.ho...@ugent.be ) 
John (SURFsara, john.donn...@surfsara.nl ) 


(*) We have deliberately planned this event shortly before the FOSDEM'18 
open source meeting [2] in Brussels the weekend after, to allow 
attendees to combine both events easily. 
There will be an HPC devroom at FOSDEM'18, talk proposals can be 
submitted until Friday Nov 24th [3]. 

[1] [ https://www.ugent.be/hpc/en | https://www.ugent.be/hpc/en ] 
[2] [ https://www.surf.nl/en/about-surf/subsidiaries/surfsara/ | 
https://www.surf.nl/en/about-surf/subsidiaries/surfsara/ ] 
[3] [ https://fosdem.org/2018/ | https://fosdem.org/2018/ ] 
[4] [ https://hpc-bigdata-fosdem18.github.io/ | 
https://hpc-bigdata-fosdem18.github.io/ ] 

Re: [easybuild] Redhead/CentOS 7.4 and Intel compiler issues

[Alvarez, Damian]

There is a bug fix for it:

https://access.redhat.com/errata/RHBA-2017:3296

Damian

PS: I didn’t see the title of the subject before.. what was the original 
poster thinking about….? ;-)

From:  on behalf of "Strube, Alexandre" 

Reply-To: 
Date: Monday 20 November 2017 at 16:34
To: "easybuild@lists.ugent.be" 
Subject: Re: [easybuild] Redhead/CentOS 7.4 and Intel compiler issues

Correct. You can test if your system is affected by running the code attached 
to the issue on Redhat’s ticket system:


https://bugzilla.redhat.com/attachment.cgi?id=1334984

dr. Alexandre Strube
a.str...@fz-juelich.de
Jülich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Jülich, Germany
Phone: +49 2461 61-3866
Fax: +49 2461 61-6656


JSC is the coordinator of the
John von Neumann Institute for Computing (NIC)
and member of the
Gauss Centre for Supercomputing (GCS)


On 20. Nov 2017, at 16:17, Åke Sandgren  wrote:

The bug affects (as far as i understand it) code built by intel
compilers from before 17.0 to 18.0 running on systems with the new
glibc. I.e., regardless of where they were built. libc is almost always
dynamically loaded.

On 11/20/2017 04:14 PM, John-Paul Robinson wrote:

Could someone clarify if this bug also affects software built with Intel
on CentOS/RedHat 7.3 and then run on the 7.4 OS versions? Or is this
only affecting new builds on 7.4 with the Intel compilers.

In other words, does it break software built with Intel on prior OS
releases an run on the 7.4 release.  It's not clear from the bug reports
or work around.

Thanks for the clarification,

~jpr

On 11/13/2017 04:03 AM, Strube, Alexandre wrote:

If possible, you can upgrade glibc with this patch:

https://copr.fedorainfracloud.org/coprs/fweimer/glibc-rhel-7.5/

The other workaround is to set LD_BIND_NOW=1, but this kills SLURM.
So, if you meet this bug, and are using slurm, try something like

“srun --export=LD_BIND_NOW=1 ….”

in your batch script. It solves the issue.



dr. Alexandre Strube
a.str...@fz-juelich.de 

Jülich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Jülich, Germany
Phone: +49 2461 61-3866
Fax: +49 2461 61-6656


JSC is the coordinator of the
John von Neumann Institute for Computing (NIC)
and member of the
Gauss Centre for Supercomputing (GCS)


On 13. Nov 2017, at 09:21, Paul Melis 

> wrote:

A related link at intel, with references to relevant glibc bug
reports, is

https://software.intel.com/en-us/articles/intel-compiler-not-compatible-with-glibc-224-9-and-newer.

I don't think this has caused issues at our place yet, but it's
definitely screwing up the plans for an update to RHEL 7.4 :-/

Paul

On 12-11-17 20:59, Joachim Hein wrote:

We got bitten by:
https://software.intel.com/en-us/articles/inconsistent-program-behavior-on-red-hat-enterprise-linux-74-if-compiled-with-intel
We are running CentOS 7.4.  Many of our intel build apps are not
working any longer.
Best wishes
  Joachim
Sent from my nanoPad


--

Paul Melis
| Visualization group leader & developer | SURFsara |
| Science Park 140 | 1098 XG Amsterdam |
| T 020 800 1312 | paul.me...@surfsara.nl
 | www.surfsara.nl
> |



--
Ake Sandgren, HPC2N, Umea University, S-90187 Umea, Sweden
Internet: a...@hpc2n.umu.se   Phone: +46 90 7866134 
Fax: +46 90-580 14
Mobile: +46 70 7716134 WWW: http://www.hpc2n.umu.se





Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] gc3pie on heterogenous cluster

[Riccardo Murri]

(Jure Pečar, Fri, Dec 01, 2017 at 01:17:05PM +0100:)
> On Fri, 1 Dec 2017 12:34:42 +0100
> Riccardo Murri  wrote:
>
> > Do I understand correctly that you want to generate programmatically a 
> > number
> > of different values for `-C`?
>
> Yes, like nehalem, sandybridge, haswell, skylake, epyc ...
>
> So multiple backend definitions in gc3pie.conf would be an option. How do I 
> then tell eb --job which backend to use? --job-backend is a choice between 
> gc3pie and pbspython. I assume I can play with --job-backend-config and have 
> one backend per gc3pie.conf.arch file, right? I'll try that ...

That's one option.  Another one is to define different GC3Pie resources
in the same configuration file and use EB's `--job-target-resource`::

  ### gc3pie.conf
  [resource/nehalem]
  # ... generic SLURM config here
  sbatch = sbatch -C nehalem

  [resource/sandybridge]
  # ... (copy config from `nehalem`)
  sbatch = sbatch -C sandybridge

and then:

eb --job-backend gc3pie --job-target-resource nehalem ...

Hope this helps,
Riccardo

--
Riccardo Murri

S3IT: Services and Support for Science IT
University of Zurich

Re: [easybuild] gc3pie on heterogenous cluster

[Jure Pečar]

On Fri, 1 Dec 2017 12:34:42 +0100
Riccardo Murri  wrote:

> Do I understand correctly that you want to generate programmatically a number 
> of different values for `-C`? 

Yes, like nehalem, sandybridge, haswell, skylake, epyc ...

So multiple backend definitions in gc3pie.conf would be an option. How do I 
then tell eb --job which backend to use? --job-backend is a choice between 
gc3pie and pbspython. I assume I can play with --job-backend-config and have 
one backend per gc3pie.conf.arch file, right? I'll try that ...


-- 

  Jurij Pečar
  HPC Engineer, IT Operations, IT Services
  EMBL Heidelberg, Meyerhofstraße 1, 69117, Heidelberg, Germany
  Room 13-401

Re: [easybuild] Recipe for Torch ?

[Maxime Boissonneault]

  
  

 See
  
 
https://github.com/ComputeCanada/easybuild-easyconfigs/tree/computecanada-master/easybuild/easyconfigs/t/Torch
  
And
  
 
https://github.com/ComputeCanada/easybuild-easyblocks/blob/computecanada-master/easybuild/easyblocks/t/torch.py
  

  
I have not created a pr yet because since we don’t use the same toolchains as 
upstream Easybuild, it is rather hard to test everything through Travis.
  

  
Maxime
  

  
  

  
  
>   
> On déc. 1, 2017 at 5:26 a.m.,  mailto:jurij.pe...@embl.de)>  
> wrote:
>   
>   
>   
>  On Thu, 18 May 2017 07:52:06 -0400  
> Bart Oldeman    wrote:  
>
> >  Hi,  
> >   
> >  I have some notes here about Torch installation:  
>   
> ...  
>   
> >  I can try to create an easyconfig for this. It should not be too hard. I  
> >  remember there was an issue in the past with Lmod using a conflicting Lua, 
> >  
> >  but IIRC that was resolved.  
>
> Did you manage to get torch built by easybuild?  
>
>
> --  
>
>  Jurij Pečar  
>  HPC Engineer, IT Operations, IT Services  
>  EMBL Heidelberg, Meyerhofstraße 1, 69117, Heidelberg, Germany  
>  Room 13-401  
>  

Re: [easybuild] gc3pie on heterogenous cluster

[Riccardo Murri]

Hi Jurij,

> I'm looking at possible easybuild integrations to our slurm. Sice we have a 
> zoo of machines in the cluster, I would need to submit easybuild --job with 
> the use of sbatch -C flags so that each piece of software gets built for all 
> cpu generations we have.
>
> I don't see support for -C in gc3pie/slurm.

If you were to use a fixed value for `-C` (e.g., `-C lustre`), this
could be supported by setting

sbatch = sbatch -C lustre

in `gc3pie.conf`.

Do I understand correctly that you want to generate programmatically a number of
different values for `-C`?  Or do you want to re-run the same `eb ...`
command-line over a number of given `-C` values?

Ciao,
R
--
Riccardo Murri

S3IT: Services and Support for Science IT
University of Zurich

[easybuild] gc3pie on heterogenous cluster

[Jure Pečar]

Hi,

I'm building a CI pipeline that will eventually feed our hpc software 
repository from a git of .eb files.

I'm looking at possible easybuild integrations to our slurm. Sice we have a zoo 
of machines in the cluster, I would need to submit easybuild --job with the use 
of sbatch -C flags so that each piece of software gets built for all cpu 
generations we have.

I don't see support for -C in gc3pie/slurm.

How do you guys handle this? Is it doable by specifying different backends in 
gc3pie.conf? How do you then tell easybuild which backend to use?

As a last resort I'll just roll my own job files and feed them to sbatch.


-- 

  Jurij Pečar
  HPC Engineer, IT Operations, IT Services
  EMBL Heidelberg, Meyerhofstraße 1, 69117, Heidelberg, Germany
  Room 13-401