Re: [easybuild] RE: R-3.3.2-foss-2016.09 module load error

[Peretti-Pezzi Guilherme]

Hi

We’ve also spotted differences on the Jenkins shell under RH, maybe because its 
shell is not interactive. The profile was not being sourced, so we had to add 
it to the jenkins script:

source ~/.profile

We don’t use Lmod though, not sure this will help.

Cheers,
--
Guilherme Peretti-Pezzi
CSCS - Swiss National Supercomputing Centre


From: 
> on 
behalf of "Siddiqui, Shahzeb" 
>
Reply-To: "easybuild@lists.ugent.be" 
>
Date: Monday 20 February 2017 19:46
To: lmod-users 
>
Cc: "easybuild@lists.ugent.be" 
>
Subject: [easybuild] RE: R-3.3.2-foss-2016.09 module load error

Please see the attached file

From: Siddiqui, Shahzeb
Sent: Monday, February 20, 2017 1:46 PM
To: lmod-users
Cc: easybuild@lists.ugent.be
Subject: R-3.3.2-foss-2016.09 module load error

Hello,

I seem to be having issue loading module for R through Jenkins, however when I 
run it via Command Line, it seems to work. I have no clue why it is not working 
for this particular module.

I have a similar module for R/3.3.2 built with intel and that module works. I 
can’t figure out what klist error is.

Any ideas how to fix this?

Shahzeb Siddiqui
HPC Linux Engineer
B2220-447.2
Groton, CT

Re: [easybuild] Theano orte_init

[Peretti-Pezzi Guilherme]

Hi

We have a similar problem when building Theano on login nodes: MPI subsystem is 
not available, so the sanity check fails.

Our workaround (not approved by Kenneth) is to skip the sanity check, see here:
https://github.com/eth-cscs/production/pull/84/files#diff-4363a39e679c51f12df983df71d3eaa9R33

--
Cheers,
G.

From: 
> on 
behalf of Yann Sagon >
Reply-To: "easybuild@lists.ugent.be" 
>
Date: Wednesday 22 February 2017 14:38
To: "easybuild@lists.ugent.be" 
>
Subject: Re: [easybuild] Theano orte_init

No it's not submitted to a scheduler (the scheduler is configured in EasyBuild 
but I'm not using it, ie. not using --job)

2017-02-22 12:52 GMT+01:00 Robert Schmidt 
>:
As a default sanity check for python packages, easybuild will try to import the 
main module. In this case it seems like the import was enough to trigger an mpi 
failure.

It is a bit weird that the import is enough to cause that problem, but maybe 
there is something else in the environment that makes it think it is being run 
as part of an MPI job? Is the build being submitted to a scheduler?

On Wed, Feb 22, 2017 at 5:36 AM Yann Sagon 
> wrote:
Hello,

I'm back again with my problem (I had a similar problem with Rmpi), maybe 
someone has an idea of what is going on.

I tried to install Theano-0.8.2-foss-2016a-Python-2.7.11.eb and had the 
following error:


== 2017-02-22 11:19:25,788 build_log.py:147 ERROR EasyBuild crashed with an 
error (at ?:124 in __init__): Sanity check failed: Theano failed to install, 
cmd '/opt/ebsofts/MPI/GCC/4.9.3-2.25/OpenMPI/1.10.2/Python/2.7.11/bin/python -c 
"import theano"' (stdin: None) output: 
--
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  PMI2_Job_GetId failed failed
  --> Returned value (null) (14) instead of ORTE_SUCCESS

It worth be said that I have compiled OpenMPI with the following additional 
flags:

configopts += '--with-slurm --with-pmi '

Thanks for any suggestion

--
Yann SAGON
Ingénieur système HPC

24 Rue du Général-Dufour
1211 Genève 4 - Suisse
Tél. : +41 (0)22 379 7737
yann.sa...@unige.ch - 
www.unige.ch




--
Yann SAGON
Ingénieur système HPC

24 Rue du Général-Dufour
1211 Genève 4 - Suisse
Tél. : +41 (0)22 379 7737
yann.sa...@unige.ch - 
www.unige.ch

[easybuild] Advice on shared easybuild install

[Peretti-Pezzi Guilherme]

Dear all,

I¹m trying to set up shared easybuild installs here at CSCS and I¹d like
some advice on how to make all software(+modules) group writable.

I found some info on this PR that should do the job:
https://github.com/hpcugent/easybuild-framework/pull/902

I tried to set up umask but I end up with some inconsistent permissions:
they are OK for files under easybuild/modules/ and KO for directories
under easybuild/software/ (missing g+w).

Anyone here is using a shared setup and faced similar issues?

Thank you in advance if you have any advice.

Cheers,
Guilherme

[easybuild] [off-topic] Software Engineer Position at Swiss National Supercomputing Centre

[Peretti-Pezzi Guilherme]

Hi

I liked Riccardo’s idea so I’m also posting a job position here (since we’re 
using EB). Feel free to forward it to potential candidates.

Cheers,
Guilherme

==
The mission of the Scientific Computing Support Unit is to help users from all 
computational domains to make the most efficient use of CSCS leading-edge 
supercomputing resources and to build bridges between the user and various 
units at CSCS, at a larger scale also among the various Swiss computational 
research communities. In order to strengthen our team we are seeking a:

Software Engineer

In this role you will support the implementation and maintenance of our 
software development framework strategy. In addition you are expected to be 
comfortable with the full chain of scientific applications development tools 
and are able to deal with second level computer support for our portfolio of 
applications.

Desired Skills and Experience:

Applicants should have a Master's degree and a Ph.D. in Software Engineering or 
Computational Science. Demonstrated experience with application lifecycle 
management tools, with special emphasis on continuous integration and 
validation tools is highly appreciated. A proven record of parallel and HPC 
code, numerical libraries and/or scientific workflow development for or within 
a scientific discipline or teams is considered a strong asset. In particular, 
you should also have strong experience in software testing and deployment on 
large HPC systems and test-driven development frameworks, such as test suites 
and continuous integration tools.

Evidence of application tuning on specialized and accelerated hardware, for 
example GPU, Intel Xeon Phi, embedded systems and automatic building tools for 
scientific software is highly appreciated. Know-how of code development tools 
such as IDEs, debuggers, performance and monitoring tools is a must.

Besides your motivation to accept new challenges and to work independently in a 
problem solving way within a vibrant scientific environment, you are 
characterized by strong interpersonal communication and organizational skills. 
The ideal candidate is a team player and feels comfortable working in an 
international environment in the heart of Lugano, Canton of Ticino in 
Switzerland and is comfortable with a fast changing working environment. 
Excellent command of written and spoken English (our official working language) 
is a must.

Starting date: negotiable. You will benefit from a competitive salary based on 
experience and performance.

For further information please contact Dr. Guilherme Peretti-Pezzi, (peretti 
[at] cscs.ch), Group Lead Scientific Computing Support. (No 
applications please)

We look forward to receiving your complete online application, which we ask you 
to submit to Stephanie Frequente, CSCS Human Resources, via Trevano 131, 6900 
Lugano.

https://www.linkedin.com/jobs2/view/77914182?trk=jobs_biz_prem_srch

[easybuild] FW: 11th EasyBuild Hackathon hosted at CSCS in Lugano

[Peretti-Pezzi Guilherme]

Dear EasyBuilders,

We’re happy to announce the 11th EasyBuild Hackathon that will take place at 
CSCS in Lugano, just after the HPC Advisory Council (which also happens to be 
in Lugano).

This Hackathon will be mostly dedicated to hands on activities, however it will 
also include a short intro for new users. You can find more details in the 
message below.

If you are interested in a more comprehensive intro, you are advised to attend 
Kenneth’s talk at the HPC Advisory Council: 
http://www.hpcadvisorycouncil.com/events/2016/swiss-workshop/agenda.php

Do not hesitate to contact me or Kenneth for further information.

Looking forward to see you in Lugano!

Sincerely,
--
Guilherme Peretti-Pezzi, PhD
CSCS - Swiss National Supercomputing Centre
Via Trevano 131
CH-6900 Lugano Tel: +41 (91) 610 8231
Switzerland


From:  Giuffreda Maria Grazia >
Date: Thursday 18 February 2016 10:41
Subject: [users-hpc] 11th EasyBuild Hackathon hosted at CSCS in Lugano

Dear All,

The Swiss National Supercomputing Centre (CSCS) is delighted to invite you to 
attend the 11th EasyBuild Hackathon,
to be hosted on March 24 to 25, 2016 at CSCS, in Lugano, Switzerland.

EasyBuild is a software build andinstallation framework that allows you to 
manage (scientific) software on High Performance
Computing (HPC) systems in an efficient way.

This event is dedicated to help new & expert users to get the most out of 
EasyBuild. The agenda will cover a basic introduction,
an overview of the new features, the new Cray toolchains and the remaining time 
will be dedicated to hands-on activities.

Attendees are welcome (and advised) to bring their own set of applications and 
custom scientific workflows, so that we can integrate
it to EasyBuild and discuss/implement missing features.

The event will start at 9:30 am on Thursday, March 24 as well as on Friday, 
March 25, 2016; and end at 5:00 pm on both days.

More details and updates will be available on the following link
https://github.com/hpcugent/easybuild/wiki/11th-EasyBuild-hackathon

Speakers
• Kenneth Hoste (HPC-UGent, Belgium; EasyBuild developer and release manager)
• Petar Forai (IMP/IMBA, Austria; EasyBuild contributor)
• Guilherme Peretti-Pezzi (CSCS, Switzerland; EasyBuild contributor)

The registration is now open:
http://www.cscs.ch/events/event_detail/index.html?tx_seminars_pi1%5BshowUid%5D=131

Best regards,
Maria Grazia







CSCS Swiss National Supercomputing Centre
Maria Grazia Giuffreda, PhD | Associate Director
Via Trevano, 131 | CH-6900 Lugano
mgg[at]cscs.ch | Phone +41 91 610 82 23

http://www.cscs.ch | http://user.cscs.ch
http://twitter.com/#!/cscsch | http://www.youtube.com/cscsch

Ad Astra Per Aspera

Re: participation fees at the 11th EB hackathon (WAS: Re: [easybuild] FW: 11th EasyBuild Hackathon hosted at CSCS in Lugano)

[Peretti-Pezzi Guilherme]

Dear all,

Due to all the requests from the community (and in order to be aligned
with EB’s Code of Conduct :) CSCS won’t be charging any registration fee
for the Hackathon.

The webpage has been updated accordingly. Please be aware that the
registration is mandatory (deadline is March 18th).

Cheers,
G.




-Original Message-
From:  on behalf of Riccardo Murri

Reply-To: "easybuild@lists.ugent.be" 
Date: Thursday 18 February 2016 21:55
To: "easybuild@lists.ugent.be" 
Subject: participation fees at the 11th EB hackathon (WAS: Re: [easybuild]
FW: 11th EasyBuild Hackathon hosted at CSCS in Lugano)

>Hi Guilherme, all,
>
>> We’re happy to announce the 11th EasyBuild Hackathon that will take
>> place at CSCS in Lugano, just after the HPC Advisory Council (which
>> also happens to be in Lugano).
>>
>> This Hackathon will be mostly dedicated to hands on activities,
>> however it will also include a short intro for new users.
>
>Thanks for organizing this!
>
>However, I see that CSCS is charging participants 80. CHF per day, and I
>find this unfair towards Hackathon contributors and institutions that
>have hosted similar events in the past for free.
>
>After all, CSCS is going to profit eventually from work done at the
>Hackathon in terms of bug fixes and new features for EB; asking
>participants to pay in order to come to work in Lugano really feels like
>double-charging to me.
>
>Of course, the amount is not a problem (160. CHF is peanuts in an IT
>group's budget), but I do really feel these charges are undue.
>
>Ciao,
>R
>
>--
>Riccardo Murri
>http://www.s3it.uzh.ch/about/team/#Riccardo.Murri
>
>S3IT: Services and Support for Science IT
>University of Zurich
>Winterthurerstrasse 190, CH-8057 Zürich (Switzerland)
>
>Tel: +41 44 635 4297
>Fax: +41 44 635 6888

[easybuild] REMINDER: 11th EasyBuild Hackathon hosted at CSCS in Lugano

[Peretti-Pezzi Guilherme]

Hi all

This is just a gentle reminder that the registration is mandatory for the
Hackathon and the deadline is March 18th.

Cheers,
G.


-Original Message-
From: Peretti-Pezzi  Guilherme <pere...@cscs.ch>
Date: Friday 19 February 2016 19:05
To: "easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
Subject: Re: participation fees at the 11th EB hackathon (WAS: Re:
[easybuild] FW: 11th EasyBuild Hackathon hosted at CSCS in Lugano)

>Dear all,
>
>Due to all the requests from the community (and in order to be aligned
>with EB’s Code of Conduct :) CSCS won’t be charging any registration fee
>for the Hackathon.
>
>The webpage has been updated accordingly. Please be aware that the
>registration is mandatory (deadline is March 18th).
>
>Cheers,
>G.
>
>
>
>
>-Original Message-
>From: <easybuild-requ...@lists.ugent.be> on behalf of Riccardo Murri
><riccardo.mu...@uzh.ch>
>Reply-To: "easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
>Date: Thursday 18 February 2016 21:55
>To: "easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
>Subject: participation fees at the 11th EB hackathon (WAS: Re: [easybuild]
>FW: 11th EasyBuild Hackathon hosted at CSCS in Lugano)
>
>>Hi Guilherme, all,
>>
>>> We’re happy to announce the 11th EasyBuild Hackathon that will take
>>> place at CSCS in Lugano, just after the HPC Advisory Council (which
>>> also happens to be in Lugano).
>>>
>>> This Hackathon will be mostly dedicated to hands on activities,
>>> however it will also include a short intro for new users.
>>
>>Thanks for organizing this!
>>
>>However, I see that CSCS is charging participants 80. CHF per day, and I
>>find this unfair towards Hackathon contributors and institutions that
>>have hosted similar events in the past for free.
>>
>>After all, CSCS is going to profit eventually from work done at the
>>Hackathon in terms of bug fixes and new features for EB; asking
>>participants to pay in order to come to work in Lugano really feels like
>>double-charging to me.
>>
>>Of course, the amount is not a problem (160. CHF is peanuts in an IT
>>group's budget), but I do really feel these charges are undue.
>>
>>Ciao,
>>R
>>
>>--
>>Riccardo Murri
>>http://www.s3it.uzh.ch/about/team/#Riccardo.Murri
>>
>>S3IT: Services and Support for Science IT
>>University of Zurich
>>Winterthurerstrasse 190, CH-8057 Zürich (Switzerland)
>>
>>Tel: +41 44 635 4297
>>Fax: +41 44 635 6888
>

Re: [easybuild] Newbie question: Generic easyblock to use for VASP

[Peretti-Pezzi Guilherme]

Andy,


The only way of ensuring reproducibility is to pin the intel version, and
that was our focus when designing these toolchains, rather than having
generic/versionless build recipes that might work or not depending on the
defaults of your system.

Since the intel version can be different from one site to another, the
approach that I would recommend is to also modify the CrayIntel toolchain
version number when you tailor the intel version to match your system (for
example CrayIntel/2015.11-15.0.2).

This will require you to use the --try-toolchain-version=2015.11-15.0.2
when build other stock CrayIntel/2015.11 easyconfig files on your system,
however it will enable your recipes to co-exist with the existing ones
(and will allow you to contribute them back one day).

If you consider that this solution is not acceptable for you, please open
an issue on Github so that we can discuss if we can improve the versioning
scheme.

Cheers,
Guilherme

-Original Message-
From: Kenneth Hoste <kenneth.ho...@ugent.be>
Date: Wednesday 1 June 2016 22:08
To: "easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
Cc: Peretti-Pezzi  Guilherme <pere...@cscs.ch>, Marsella  Luca
<marse...@cscs.ch>, "petar.fo...@imp.ac.at" <petar.fo...@imp.ac.at>
Subject: Re: [easybuild] Newbie question: Generic easyblock to use for VASP

>Hi Andy,
>
>On 01/06/16 14:39, Andy Turner wrote:
>>
>> Just thought I should let you know that this is now working, the final
>> combination of options needed was:
>>
>> patches = [('makefile.include.crayxc_intel', '.')]
>> prebuildopts = 'cp makefile.include.crayxc_intel makefile.include && '
>>
>> Thanks for all your help with this.
>>
>> BTW, in working on this I noticed a bug in your CrayIntel-2015.11.eb
>> toolchain. This toolchain specifies a particular verison of the Intel
>> compiler (15.0.2.164):
>>
>> -- 
>> dependencies = [
>> # PrgEnv version is not pinned, as Cray recommends to use the
>> latest (default) version
>> ('PrgEnv-intel', EXTERNAL_MODULE),
>> ('intel/15.0.1.133', EXTERNAL_MODULE),
>> ('cray-libsci/13.2.0', EXTERNAL_MODULE),
>> ('cray-mpich/7.2.6', EXTERNAL_MODULE),
>> ]
>> -- 
>>
>> In fact, the Intel compiler versions are not part of the Cray PE
>> release process, their version progression is decoupled from the Cray
>> software release schedule.
>>
>> For example, we have the Nov 2015 PE release but do not have
>> 'intel/15.0.1.133' (we have 'intel/15.0.2.164') so I had to modify
>> this EasyConfig file to make the installation work.
>
>The 'intel' version was picked by the EasyBuilders @ CSCS based on
>latest/recommended version info the Cray documentation.
>Guilherme/Luca (CSCS) or Petar (in CC) can provide more details on this...
>
>>
>> Having though about this a bit I think it makes more sense for the
>> 'intel' module to be specified as a dependency in the VASP easyconfig
>> rather than in the toolchain. Could the toolchain just contain:
>>
>> -- 
>> dependencies = [
>> # PrgEnv version is not pinned, as Cray recommends to use the
>> latest (default) version
>> ('PrgEnv-intel', EXTERNAL_MODULE),
>> ('intel', EXTERNAL_MODULE),
>> ('cray-libsci/13.2.0', EXTERNAL_MODULE),
>> ('cray-mpich/7.2.6', EXTERNAL_MODULE),
>> ]
>> -- 
>>
>> and then specify the specific intel version (if needed) in the
>> software easyconfig?
>
>No, that won't work, because this would mean that you let EasyBuild load
>whatever 'intel' version is the default, and then let intel/15.0.1.133
>be loaded as a dependency for VASP. That will most likely not work, and
>result in a version conflict...
>
>If you want to change this, I would recommend slightly tweaking the
>definition of the CrayIntel/2015.11 toolchain to use the 'intel' version
>you have available.
>
>Ideally, these versions should be picked such that they work for anyone
>who has that particular version of Cray PE release, but I'm not sure if
>that's feasible, there seem to be small differences between Cray sites.
>
>Again, this discussion should probably be picked up with the folks at
>CSCS...
>
>
>regards,
>
>Kenneth

Re: [easybuild] caffe/mxnet/tensorflow easyconfigs

[Peretti-Pezzi Guilherme]

Hi

We used EB for all the dependencies (Python, Swig, PCRE, Bazel) when
building a GPU-enabled TF 0.9.0 on our Cray systems.

Building TF itself required a couple of ugly tweaks (see [1]), I was not
convinced that it would work so I didn’t bother to create the .eb for
that. 

So far the users didn’t complain much about this build, so I assume it is
working. You can find the .eb files we used for Python, Swig, PCRE and
Bazel on [2].


I hope it helps.

--
Cheers,
G.

[1] 
https://github.com/eth-cscs/TensorFlow/wiki/TensorFlow-0.9.0-Daint-GPU-buil
d
[2] https://github.com/eth-cscs/tools/tree/master/easybuild/easyconfigs





-Original Message-
From:  on behalf of Erik Smeets

Reply-To: "easybuild@lists.ugent.be" 
Date: Tuesday 18 October 2016 12:50
To: "easybuild@lists.ugent.be" 
Subject: [easybuild] caffe/mxnet/tensorflow easyconfigs

>Hello,
>
>Has anyone got (working) easyconfigs for caffe, mxnet and tensorflow they
>are willing to share? Any work in progress is also appreciated, so we
>could help with these. For Caffe I've already found
>http://www.siliconslick.com/easybuild/ebfiles_repo_cleaned/ada/Caffe/Caffe
>-rc3-intel-2016a-Python-2.7.11.eb and
>https://github.com/hpcugent/easybuild-easyconfigs/pull/3667/files. The
>first is not working (at least for us) and the second is foss toolchain
>based, while we prefer intel toolchain based.
>
>Thanks in advance.
>Kind regards,
>Erik Smeets
>
>-- The information contained in this communication and any attachments is
>confidential and may be privileged, and is for the sole use of the
>intended recipient(s). Any unauthorized review, use, disclosure or
>distribution is prohibited. Unless explicitly stated otherwise in the
>body of this communication or the attachment thereto (if any), the
>information is provided on an AS-IS basis without any express or implied
>warranties or liabilities. To the extent you are relying on this
>information, you are doing so at your own risk. If you are not the
>intended recipient, please notify the sender immediately by replying to
>this message and destroy all copies of this message and any attachments.
>The sender nor the company/group of companies he or she represents shall
>be liable for the proper and complete transmission of the information
>contained in this communication, or for any delay in its receipt.