Hello, Specifically for Intel I think for HMNS treat icc and ifort as packages and only toolchain have MODULEPATH
The solution I think could be. Remove from icc and ifort module the modulepath for Compiler/intel Rather have iccifort module have a modulePath for iccifort When loading icc or ifort separately you wouldn’t see the modules built with intel or iccifortcuda, but rather you would see packages when loading the toolchain: iccifort or intel impi module would reside in iccifort or iccifortcuda dependening on toolchain. The module would be in separate trees. When loading either iccifort or iccifortcuda you should see the impi from its toolchain tree. The same goes with imkl, it should be in a tree for iimpi or iimpic since that is the toolchain used. Both intel and intelcuda should be treated as separate module trees. When loading intel it should have MODULEPATH for Compiler/intel and intelcuda should have MODULEPATH for Compiler/intel-CUDA. Intel should load -> iccifort, iimpi, imkl Intel Cuda -> iccifortcuda, iimpic, imkl No need to have explicit load for icc and ifort in the intel module since the iccifort toolchain will take care of this. Keep in mind both iccifort and iccifortcuda will load icc and ifort that will come from the same module. They would reside under Core/ If this is done this way we can see the intelcuda tree while having icc and ifort loaded since they wouldn’t be loading the intel. In my environment, I treat all toolchains as family so that I can’t have an instance where intel and intelcuda are loaded at the same time. From: easybuild-requ...@lists.ugent.be [mailto:easybuild-requ...@lists.ugent.be] On Behalf Of Alan O'Cais Sent: Thursday, March 2, 2017 10:32 AM To: easybuild@lists.ugent.be Subject: Re: [easybuild] MODULEPATH issue when supporting intel and intelcuda toolchain concurrently Dear Shahzeb, I think this is probably the same (or at least related to the) issue that is being discussed in https://github.com/hpcugent/easybuild-framework/pull/2135<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hpcugent_easybuild-2Dframework_pull_2135&d=DwMFaQ&c=UE1eNsedaKncO0Yl_u8bfw&r=RMJdCm7m5fiPWhajwKUnEW5yn4eK2YdUWW-MLVShghg&m=CsWNr_6qdkJE_RoT54a_jmrskakWb47pfZcmQpcT4-Q&s=rtu9QLIWL9XOUXetPh9eal99n64mPjldU22rDdTP9GE&e=> It also exposes one of the problems of a HMNS, the potential non-uniqueness of module names. The problem with not building software with minimal toolchains is that you can have multiple copies at various levels of your toolchain hierarchy. What module you end up loading is then dependent on the order that you load things (perhaps not in Lmod because it is hierarchy-aware but definitely for other module tools). This can clearly lead to issues. Alan On 2 March 2017 at 15:32, Siddiqui, Shahzeb <shahzeb.siddi...@pfizer.com<mailto:shahzeb.siddi...@pfizer.com>> wrote: Hello, I seem to notice an issue when building modules using HierarchicalNamingScheme when building out the intel and intelcuda toolchains. I notice that MODULEPATH is set for icc and ifort for intel directory. This is correct when setting up intel toolchain. hpcswadm@hpcv18$grep -iR MODULEPATH icc/2017.1.132-GCC-5.2.0.lua:prepend_path("MODULEPATH", "/nfs/grid/software/testing/RHEL7/easybuild/modules/all/Compiler/intel/2017.1.132-GCC-5.2.0") ifort/2017.1.132-GCC-5.2.0.lua:prepend_path("MODULEPATH", "/nfs/grid/software/testing/RHEL7/easybuild/modules/all/Compiler/intel/2017.1.132-GCC-5.2.0") impi gets installed in the path for intel as expected. hpcswadm@hpcv18$ls -R /nfs/grid/software/testing/RHEL7/easybuild/modules/all/Compiler/intel/2017.1.132-GCC-5.2.0/ /nfs/grid/software/testing/RHEL7/easybuild/modules/all/Compiler/intel/2017.1.132-GCC-5.2.0/: impi /nfs/grid/software/testing/RHEL7/easybuild/modules/all/Compiler/intel/2017.1.132-GCC-5.2.0/impi: 2017.1.132.lua As for impi built with iccifortcuda it gets installed in hpcswadm@hpcv18$ls -R /nfs/grid/software/testing/RHEL7/easybuild/modules/all/Compiler/intel-CUDA/ /nfs/grid/software/testing/RHEL7/easybuild/modules/all/Compiler/intel-CUDA/: 2017.1.132-GCC-5.2.0-7.5.18 /nfs/grid/software/testing/RHEL7/easybuild/modules/all/Compiler/intel-CUDA/2017.1.132-GCC-5.2.0-7.5.18: impi /nfs/grid/software/testing/RHEL7/easybuild/modules/all/Compiler/intel-CUDA/2017.1.132-GCC-5.2.0-7.5.18/impi: 2017.1.132.lua The problem is when loading iimpic toolchain it loads up icc and ifort modules along with impi that belongs to the module tree from intel and not intel-CUDA. I am not sure if this is a problem, but it seems like the impi is not being picked up correctly. hpcswadm@hpcv18$ml av iimpi ------------------------------------- /nfs/grid/software/testing/RHEL7/easybuild/modules/all/Core ------------------------------------- iimpi/2017.01-GCC-5.2.0 (TC) iimpic/2017.01-GCC-5.2.0 (TC) hpcswadm@hpcv18$ml Currently Loaded Modules: 1) EasyBuild/3.1.0 hpcswadm@hpcv18$ml iimpic Currently Loaded Modules: 1) EasyBuild/3.1.0 3) icc/2017.1.132-GCC-5.2.0 (I) 5) impi/2017.1.132 (I) 7) iimpic/2017.01-GCC-5.2.0 ( TC) 2) GCC/5.2.0 4) ifort/2017.1.132-GCC-5.2.0 (I) 6) CUDA/7.5.18 hpcswadm@hpcv18$which mpicc /nfs/grid/software/testing/RHEL7/easybuild/software/Compiler/intel/2017.1.132-GCC-5.2.0/impi/2017.1.132/bin64/mpicc The one that should be loaded is from /nfs/grid/software/testing/RHEL7/easybuild/software/Compiler/intel-CUDA/2017.1.132-GCC-5.2.0-7.5.18/impi/2017.1.132/bin64/mpicc I think the impi module should not sit inside intel directory or somehow icc and ifort MODULEPATH need to be changed to intel-CUDA when loading iimpic /nfs/grid/software/testing/RHEL7/easybuild/modules/all/Compiler/intel/2017.1.132-GCC-5.2.0 ---------------------- impi/2017.1.132 (I,L) Anyone else come across this issue. Shahzeb Siddiqui HPC Linux Engineer B2220-447.2 Groton, CT -- Dr. Alan O'Cais E-CAM Software Manager Juelich Supercomputing Centre Forschungszentrum Juelich GmbH 52425 Juelich, Germany Phone: +49 2461 61 5213 Fax: +49 2461 61 6656 E-mail: a.oc...@fz-juelich.de<mailto:a.oc...@fz-juelich.de> WWW: http://www.fz-juelich.de/ias/jsc/EN<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.fz-2Djuelich.de_ias_jsc_EN&d=DwMFaQ&c=UE1eNsedaKncO0Yl_u8bfw&r=RMJdCm7m5fiPWhajwKUnEW5yn4eK2YdUWW-MLVShghg&m=CsWNr_6qdkJE_RoT54a_jmrskakWb47pfZcmQpcT4-Q&s=uaDU1IrFmBcfzH691zqnnIIsDG_OoIzF-6l1aLSfHT4&e=> ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------
