RE: [easybuild] Updated Lmod RPM package needed for EasyBuild v3.7.0
Hi, while this topic of providing an Lmod RPM is still afresh, let me pledge a few bits relevant to the EB community, since I've spent a good chunk of time during last 2-3 years around it along with Panos from Bright Computing. Also, together with Robert McLay we've debugged plenty of subtle bugs & Settarg related features - good times! https://github.com/Bright-Computing/bic ## .spec file and much paraphernalia included, ignore the filenames If you are running a cluster by Bright Computing with Lmod, I bet you're probably using a fork of that work, already. The driving need for it has been to combine a few requirements together and rationalise the Lmod RPMs: * Deliver Lmod across a dozen+ HPC clusters with varying needs of configuration, but in a consistent manner. * Keep separate what needs to be separate: configuration by upstream Lmod, by Bright & by local HPC sites. * Changes in one of the above separation of domains should not automatically require rebuilding/hacking the RPMs (or, at least, not hack the .spec too often). There's also a pretty neat extendible mechanism to define initialisation variables, check the 2 .yaml files here: https://github.com/Bright-Computing/bic/tree/master/test/etc/profile.definitions/site We went at great length to ensure that 1 upstream provider would do not enforce 1 configuration to downstream sites, but the extended logic has not been fully adopted by Bright - nor is it my/our/anyone's prerogative to dictate that. Anyway, if anyone is into this kind of sport, please find me at G-hangouts to follow it up. (generally, it'd be good if people made an effort to bump their Lmod versions - entertain yourself with: `mkdir posix.lua`) enjoy, F. -- echo "sysadmin know better bash than english" | sed s/min/mins/ \ | sed 's/better bash/bash better/' # Yelling in a CERN forum From: easybuild-requ...@lists.ugent.be [easybuild-requ...@lists.ugent.be] on behalf of Ole Holm Nielsen [ole.h.niel...@fysik.dtu.dk] Sent: Friday, September 28, 2018 09:03 To: easybuild@lists.ugent.be Cc: Kenneth Hoste Subject: Re: [easybuild] Updated Lmod RPM package needed for EasyBuild v3.7.0 The Lmod EPEL package maintainer Orion Poplawski has advised me to file a bug in the RedHat bugzilla requesting a version upgrade of the Lmod RPM, and you can all view it here: https://bugzilla.redhat.com/show_bug.cgi?id=1633929 If your testing of the Lmod 6.6.3 RPM is successful, or if it gives problems for you, or if you have any comments, please add information to the bugzilla page. If you don't have a login there, I can take your comments and add them. When the testing of Lmod has been successful, the updated 6.6.3 RPM package should appear in the official EPEL repository. /Ole On 09/27/2018 09:00 PM, Ole Holm Nielsen wrote: > On 27-09-2018 09:57, Ole Holm Nielsen wrote: >> On 09/26/2018 09:50 PM, Kenneth Hoste wrote:> It seems like there are >> some options for getting an RPM for Lmod 6.6.3, >>> but it turns out it's also reasonable to drop the required version a >>> bit since running the EasyBuild framework tests on top of Lmod 6.5.1 >>> is not revealing any issues with that version (I tried a bunch of >>> versions older than Lmod 6.6.3, but this apparently wasn't one of >>> them...). >> >> Sounds great! I note that Fedora FC28 has a very recent version: >> Lmod-7.7.35-1.fc28.x86_64 >> >> I looked at the page >> https://fedoraproject.org/wiki/Getting_a_Fedora_package_in_EPEL and >> found the Lmod package page https://src.fedoraproject.org/rpms/Lmod. >> Then I wrote to the maintainer Orion Poplawski if he would be willing >> to make his Lmod package available also in EPEL[1]. > > The EPEL maintainer Orion Poplawski has been very kind and responded > quickly to my request. There is now an EPEL build of Lmod 6.6.3 in > https://koji.fedoraproject.org/koji/buildinfo?buildID=1148094 > and an RPM package for testing at > https://kojipkgs.fedoraproject.org//packages/Lmod/6.6.3/1.el7/x86_64/Lmod-6.6.3-1.el7.x86_64.rpm > > > Would some EasyBuild sites running CentOS 7 systems kindly test this > Lmod RPM and report any successes or failures back to me (or directly to > the developer, only I don't know the appropriate feedback channel).
Re: [easybuild] Relaxed hierarchies by relying on MPI ABI compatibility rather than MPI vendor/version.
As long as current behaviour isn't affected I see no problem. On 09/28/2018 09:43 AM, Fotis Georgatos wrote: > Hello Damian, Victor et al, > > I had missed this one, this is a very interesting offer! Indeed, it is very > sensible, too. > > It will come with some footnotes attached, but it can also make some > situations far more manageable. > > It would also make life easier with some software, esp. vendor provided, case > in point: integrating MPI stacks with MATLAB (mpich compatible). > > F. > >> On 22 Aug 2018, at 6:00 PM, Holanda Rusu Victor >> wrote: >> >> Dear Damian, >> >> I think that this is a very good approach. >> In principle, it should work for MPI implementations that are part of the >> MPICH ABI initiative (https://www.mpich.org/abi/) >> >> We do make use of this ABI compatibility for the container usage at Piz >> Daint >> (https://user.cscs.ch/tools/containers/advanced_shifter/#native-mpi-support) >> >> >> Best wishes, >> >> CSCS Swiss National Supercomputing Centre >> Victor Holanda | Computational Scientist >> ETH/CSCS | Via Trevano 131 | 6900 Lugano | Switzerland >> victor.hola...@cscs.ch | Phone +41 91 610 82 65 >> >> On 22.08.18, 16:37, "easybuild-requ...@lists.ugent.be on behalf of Alvarez, >> Damian" > d.alva...@fz-juelich.de> wrote: >> >>Dear EasyBuilders and lmod users, >> >>I have a question for the community. Currently EasyBuild supports to >> deploy its software stack in a hierarchical manner, as intended and >> supported by lmod (ie: load compiler, that expands $MODULEPATH and the MPIs >> become visible, load an MPI, which expands $MODULEPATH again). >> >>There is a very significant number of MPIs that are ABI compatible >> (MPICH, MVAPICH, Intel MPI, ParaStationMPI and possibly 1 or 2 more). I >> don't know if OpenMPI-based MPI runtimes are also ABI compatible, but I >> would guess there is a big chance that they are. >> >>My question is how would you feel about expanding the MPI's $MODULEPATH >> based on ABI compatibility, rather than MPI_vendor/version. That way one >> could offer many MPIs without needing to mirror the whole SW stack in all >> MPI branches. That could simplify SW management significantly. >> >>Caveats I can think of are: >>-One would have to be careful regarding which MPI is used to compile the >> stack. MPI compiler wrappers are different, and might add different compiler >> and/or linker flags. >>-Some MPIs, despite being ABI compatible, might offer different >> capabilities (eg: CUDA-awareness). I guess in these cases it makes sense to >> try to ensure that loading packages that depend in particular MPI >> capabilities, actually load the correct MPI runtime as a dependency, instead >> of making vague assumptions like "the correct MPI is already loaded because >> otherwise the package won't be visible in the environment". >> >>Similarly, one could think of a similar approach for compilers, to allow >> drop-in compiler replacements. Let's say icc 2018.2 is used to compile a >> given branch of the hierarchy. If that compiler has a bug that is fixed in >> 2018.3, right now the whole SW stack needs to be recompiled in a separate >> branch of the hierarchy. However, with a drop-in replacement one could >> install the latest version of the compiler but still reuse the hierarchy >> compiled with 2018.2. Needless to say, this needs to be done carefully. >> However, the possibility seems interesting. >> >>Am I missing something? How do you feel about this? >> >>Best, >>Damian >> >>-- >>Dr. Damian Alvarez >>Juelich Supercomputing Centre >>Forschungszentrum Juelich GmbH >>52425 Juelich, Germany >> >> >> >> >> >> >> >>Forschungszentrum Juelich GmbH >>52425 Juelich >>Sitz der Gesellschaft: Juelich >>Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 >>Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher >>Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), >>Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, >>Prof. Dr. Sebastian M. Schmidt >> >> >> >> >> >> >> -- Ake Sandgren, HPC2N, Umea University, S-90187 Umea, Sweden Internet: a...@hpc2n.umu.se Phone: +46 90 7866134 Fax: +46 90-580 14 Mobile: +46 70 7716134 WWW: http://www.hpc2n.umu.se
Re: [easybuild] Relaxed hierarchies by relying on MPI ABI compatibility rather than MPI vendor/version.
Hello Damian, Victor et al, I had missed this one, this is a very interesting offer! Indeed, it is very sensible, too. It will come with some footnotes attached, but it can also make some situations far more manageable. It would also make life easier with some software, esp. vendor provided, case in point: integrating MPI stacks with MATLAB (mpich compatible). F. > On 22 Aug 2018, at 6:00 PM, Holanda Rusu Victor > wrote: > > Dear Damian, > > I think that this is a very good approach. > In principle, it should work for MPI implementations that are part of the > MPICH ABI initiative (https://www.mpich.org/abi/) > > We do make use of this ABI compatibility for the container usage at Piz Daint > (https://user.cscs.ch/tools/containers/advanced_shifter/#native-mpi-support) > > > Best wishes, > > CSCS Swiss National Supercomputing Centre > Victor Holanda | Computational Scientist > ETH/CSCS | Via Trevano 131 | 6900 Lugano | Switzerland > victor.hola...@cscs.ch | Phone +41 91 610 82 65 > > On 22.08.18, 16:37, "easybuild-requ...@lists.ugent.be on behalf of Alvarez, > Damian" d.alva...@fz-juelich.de> wrote: > >Dear EasyBuilders and lmod users, > >I have a question for the community. Currently EasyBuild supports to > deploy its software stack in a hierarchical manner, as intended and supported > by lmod (ie: load compiler, that expands $MODULEPATH and the MPIs become > visible, load an MPI, which expands $MODULEPATH again). > >There is a very significant number of MPIs that are ABI compatible (MPICH, > MVAPICH, Intel MPI, ParaStationMPI and possibly 1 or 2 more). I don't know if > OpenMPI-based MPI runtimes are also ABI compatible, but I would guess there > is a big chance that they are. > >My question is how would you feel about expanding the MPI's $MODULEPATH > based on ABI compatibility, rather than MPI_vendor/version. That way one > could offer many MPIs without needing to mirror the whole SW stack in all MPI > branches. That could simplify SW management significantly. > >Caveats I can think of are: >-One would have to be careful regarding which MPI is used to compile the > stack. MPI compiler wrappers are different, and might add different compiler > and/or linker flags. >-Some MPIs, despite being ABI compatible, might offer different > capabilities (eg: CUDA-awareness). I guess in these cases it makes sense to > try to ensure that loading packages that depend in particular MPI > capabilities, actually load the correct MPI runtime as a dependency, instead > of making vague assumptions like "the correct MPI is already loaded because > otherwise the package won't be visible in the environment". > >Similarly, one could think of a similar approach for compilers, to allow > drop-in compiler replacements. Let's say icc 2018.2 is used to compile a > given branch of the hierarchy. If that compiler has a bug that is fixed in > 2018.3, right now the whole SW stack needs to be recompiled in a separate > branch of the hierarchy. However, with a drop-in replacement one could > install the latest version of the compiler but still reuse the hierarchy > compiled with 2018.2. Needless to say, this needs to be done carefully. > However, the possibility seems interesting. > >Am I missing something? How do you feel about this? > >Best, >Damian > >-- >Dr. Damian Alvarez >Juelich Supercomputing Centre >Forschungszentrum Juelich GmbH >52425 Juelich, Germany > > > > > > > >Forschungszentrum Juelich GmbH >52425 Juelich >Sitz der Gesellschaft: Juelich >Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 >Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher >Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), >Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, >Prof. Dr. Sebastian M. Schmidt > > > > > > >
Re: [easybuild] Updated Lmod RPM package needed for EasyBuild v3.7.0
The Lmod EPEL package maintainer Orion Poplawski has advised me to file a bug in the RedHat bugzilla requesting a version upgrade of the Lmod RPM, and you can all view it here: https://bugzilla.redhat.com/show_bug.cgi?id=1633929 If your testing of the Lmod 6.6.3 RPM is successful, or if it gives problems for you, or if you have any comments, please add information to the bugzilla page. If you don't have a login there, I can take your comments and add them. When the testing of Lmod has been successful, the updated 6.6.3 RPM package should appear in the official EPEL repository. /Ole On 09/27/2018 09:00 PM, Ole Holm Nielsen wrote: On 27-09-2018 09:57, Ole Holm Nielsen wrote: On 09/26/2018 09:50 PM, Kenneth Hoste wrote:> It seems like there are some options for getting an RPM for Lmod 6.6.3, but it turns out it's also reasonable to drop the required version a bit since running the EasyBuild framework tests on top of Lmod 6.5.1 is not revealing any issues with that version (I tried a bunch of versions older than Lmod 6.6.3, but this apparently wasn't one of them...). Sounds great! I note that Fedora FC28 has a very recent version: Lmod-7.7.35-1.fc28.x86_64 I looked at the page https://fedoraproject.org/wiki/Getting_a_Fedora_package_in_EPEL and found the Lmod package page https://src.fedoraproject.org/rpms/Lmod. Then I wrote to the maintainer Orion Poplawski if he would be willing to make his Lmod package available also in EPEL[1]. The EPEL maintainer Orion Poplawski has been very kind and responded quickly to my request. There is now an EPEL build of Lmod 6.6.3 in https://koji.fedoraproject.org/koji/buildinfo?buildID=1148094 and an RPM package for testing at https://kojipkgs.fedoraproject.org//packages/Lmod/6.6.3/1.el7/x86_64/Lmod-6.6.3-1.el7.x86_64.rpm Would some EasyBuild sites running CentOS 7 systems kindly test this Lmod RPM and report any successes or failures back to me (or directly to the developer, only I don't know the appropriate feedback channel).
