Hi Joachim, Whoops I never replied to this. The solution in the end for me was to create my own custom naming scheme. Basically it's a copy of HMNS but I have special cases for icc/ifort/iccifort where iccifort is what extends the path.
You can ignore the is_short_modname_for function, I mess with that because I prefer more descriptive names for the modules for iccifort, psmpi and impi in a HMNS Best, Alan On 19 February 2016 at 10:23, Alan O'Cais <[email protected]> wrote: > Hi Joachim, > > I think that since icc/ifort sit at the core level then they are not > visible to spider when they are invisible modules, which would mean that > the path extension would be seen come from iccifort...but I was only taking > that for a test drive recently and haven't actually done a proper test. > I'll do some further testing today and get back to you. > > The other option I will explore is changing the class of icc/ifort to > something other than compiler (which will bypass the exception code in the > naming scheme and cause the path not to be extended) and change the module > class of iccifort to 'compiler' (which causes the path to be extended). > This would also require a trivial change to the easyblock used for iccifort > (easyblock = 'Bundle') but something similar is already done for GCC in the > latest EasyBuild releases. If that works, we could think about a PR that > explicitly does the same thing for this special case. > > Will get back to you later, > > Alan > > On 18 February 2016 at 21:10, Joachim Hein <[email protected]> > wrote: > >> Hi Alan >> >> I sort of agree that the icc ifort split is somewhat an issue. Though the >> iccifort modules are what I would call "the Intel compiler". Changing >> matters would (might?) cause trouble in other places, such as the try >> options. >> >> If I use the HIDE_DEPS var - what would happen to >> >> module spider scipy >> >> Which compiler would be stated in that output? >> >> Thanks Joachim >> >> >> >> Sent from my iPad >> >> On 18 Feb 2016, at 16:39, Alan O'Cais <[email protected]> wrote: >> >> Dear Joachim, >> >> I consider this a flaw of the current hierarchical implementation, or >> more specifically the module classes of icc/ifort in the easyconfig files. >> Only icc together with ifort should be classed as a compiler (so icc/ifort >> should maybe be classed as tools and iccifort as a compiler not a >> toolchain). It was lmod output like yours that prompted us to use a >> toolchain-based hierarchical scheme. We've learned that this is difficult >> to make robust however and get's messy when you have (3 compilers)x(4 MPI >> implementations). >> >> You can sidestep this issue by adding icc and ifort to the >> EASYBUILD_HIDE_DEPS env var (or your eb configuration, or the command line) >> when building software. This will hide icc/ifort by default and they won't >> show up in your spider output...but doing this means you need to rebuild >> your module tree (you only need new modules, --module-only, the software >> installations themselves are fine) >> >> Alan >> >> On 18 February 2016 at 16:05, Joachim Hein <[email protected]> >> wrote: >> >>> Hi, >>> >>> We use easybuild with an hierarchical naming scheme. I have build >>> scipy-0.16.1-intel-2015b-Python-2.7.10.eb (EB 2.6) and when doing a >>> module spider I get >>> >>> -bash-4.2$ module spider scipy/0.16.1-Python-2.7.10 >>> >>> >>> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- >>> scipy: scipy/0.16.1-Python-2.7.10 >>> >>> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- >>> >>> This module can only be loaded through the following modules: >>> >>> icc/2015.3.187-GNU-4.9.3-2.25 impi/5.0.3.048 >>> ifort/2015.3.187-GNU-4.9.3-2.25 impi/5.0.3.048 >>> >>> >>> Help: >>> SciPy is a collection of mathematical algorithms and convenience >>> functions built on the Numpy extension for Python. - Homepage: >>> http://www.scipy.org >>> >>> When doing >>> >>> module load icc/2015.3.187-GNU-4.9.3-2.25 impi/5.0.3.048 >>> scipy/0.16.1-Python-2.7.10 >>> >>> it will not pull in the ifort. A user reported that: >>> >>> -bash-4.2$ python >>> Python 2.7.10 (default, Jan 25 2016, 13:14:08) >>> [GCC Intel(R) C++ gcc 4.9 mode] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> from scipy.signal import argrelextrema >>> Traceback (most recent call last): >>> File "<stdin>", line 1, in <module> >>> File "/sw/easybuild/software/MPI/intel/2015.3.187-GNU-4.9.3-2.25/impi/ >>> 5.0.3.048/scipy/0.16.1-Python-2.7.10/lib/python2.7/site-packages/scipy/signal/__init__.py", >>> line 280, in <module> >>> from .bsplines import * >>> File "/sw/easybuild/software/MPI/intel/2015.3.187-GNU-4.9.3-2.25/impi/ >>> 5.0.3.048/scipy/0.16.1-Python-2.7.10/lib/python2.7/site-packages/scipy/signal/bsplines.py", >>> line 12, in <module> >>> from scipy.special import comb, gamma >>> File "/sw/easybuild/software/MPI/intel/2015.3.187-GNU-4.9.3-2.25/impi/ >>> 5.0.3.048/scipy/0.16.1-Python-2.7.10/lib/python2.7/site-packages/scipy/special/__init__.py", >>> line 601, in <module> >>> from ._ufuncs import * >>> ImportError: libifport.so.5: cannot open shared object file: No such >>> file or directory >>> >>> >>> >>> The problem goes away if I load the matching intel toolchain. I assume >>> some of the modules involved should load the relevant iccifort to make sure >>> whether loaded via the icc or the ifort route, both modules are present >>> once the scipy is loaded. >>> >>> I am not sure at which level (Python, numpy or scipy) Fortran and C >>> report is required and which module should be modified. I leave that to >>> the experts to decide. >>> >>> Best wishes >>> Joachim >>> >>> >> >> >> -- >> Dr. Alan O'Cais >> Application Support >> Juelich Supercomputing Centre >> Forschungszentrum Juelich GmbH >> 52425 Juelich, Germany >> >> Phone: +49 2461 61 5213 >> Fax: +49 2461 61 6656 >> E-mail: [email protected] >> WWW: http://www.fz-juelich.de/ias/jsc/EN >> >> >> >> ------------------------------------------------------------------------------------------------ >> >> ------------------------------------------------------------------------------------------------ >> Forschungszentrum Juelich GmbH >> 52425 Juelich >> Sitz der Gesellschaft: Juelich >> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 >> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher >> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), >> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, >> Prof. Dr. Sebastian M. Schmidt >> >> ------------------------------------------------------------------------------------------------ >> >> ------------------------------------------------------------------------------------------------ >> >> > > > -- > Dr. Alan O'Cais > Application Support > Juelich Supercomputing Centre > Forschungszentrum Juelich GmbH > 52425 Juelich, Germany > > Phone: +49 2461 61 5213 > Fax: +49 2461 61 6656 > E-mail: [email protected] > WWW: http://www.fz-juelich.de/ias/jsc/EN > -- Dr. Alan O'Cais Application Support Juelich Supercomputing Centre Forschungszentrum Juelich GmbH 52425 Juelich, Germany Phone: +49 2461 61 5213 Fax: +49 2461 61 6656 E-mail: [email protected] WWW: http://www.fz-juelich.de/ias/jsc/EN
custom_mns.py
Description: Binary data
