Hi Kenneth, I will look into Quassel, although it still seem that Quassel Core still needs to be hosted on a system that is always connected too. I will have to arrange something... Anyway, thanks for the suggestion.
Back to our problem. For numpy I used a modified version of
numpy-1.8.2-foss-2016a-Python-2.7.11.eb where the only changes were in the
toolchain (goolf instead of foss) and Python version. Attached you can find the
easyconfig and the resulting lua module. The path to the numpy include
directory does not appear on CPATH once loading the module.
$ ml goolf
$ ml numpy
$ ml
Currently Loaded Modules:
1) GCCcore/.5.4.0 4) numactl/.2.0.11 7) OpenBLAS/0.2.19-LAPACK-3.6.1
10) goolf/2016.10 13) ncurses/.6.0 16) SQLite/3.15.0 19)
libffi/.3.2.1 22) numpy/1.11.2-Python-2.7.12
2) binutils/.2.26 5) hwloc/.1.11.4 8) FFTW/3.3.5
11) bzip2/.1.0.6 14) libreadline/.7.0 17) Tk/8.6.6 20)
OpenSSL/.1.0.2j
3) GCC/5.4.0-2.26 6) OpenMPI/1.10.4 9)
ScaLAPACK/2.0.2-OpenBLAS-0.2.19-LAPACK-3.6.1 12) zlib/.1.2.8 15) Tcl/8.6.6
18) GMP/.6.1.1 21) Python/2.7.12
$ echo $CPATH | tr ':' '\n'
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/OpenSSL/1.0.2j/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/GMP/6.1.1/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Tk/8.6.6/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/SQLite/3.15.0/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Tcl/8.6.6/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/libreadline/7.0/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/ncurses/6.0/include
/opt/easybuild/software/Compiler/GCCcore/5.4.0/zlib/1.2.8/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/bzip2/1.0.6/include
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/ScaLAPACK/2.0.2-OpenBLAS-0.2.19-LAPACK-3.6.1/include
/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/FFTW/3.3.5/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/OpenBLAS/0.2.19-LAPACK-3.6.1/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/OpenMPI/1.10.4/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/hwloc/1.11.4/include
/opt/easybuild/software/Compiler/GCC/5.4.0-2.26/numactl/2.0.11/include
/opt/easybuild/software/Compiler/GCCcore/5.4.0/binutils/2.26/include
/opt/easybuild/software/Core/GCCcore/5.4.0/include
--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu
On Nov 2 2016, at 8:28 am, Kenneth Hoste <[email protected]> wrote:
Hi Davide,
> On 1 Nov 2016, at 18:07, Vanzo, Davide <[email protected]> wrote:
>
> Hello,
> I am moving this discussion over here since it is hard to follow up on the
> IRC channel (I don't have a computer online 24/7 to set up a bouncer and the
> time zone differential from Europe does not help either).
You could use Quassel, then you have a 'server' component that is always
online, and you use a client to connect to it (talk to Pablo on the #easybuild
IRC channel, he can hook you up). That doesn't fix the timezone issue though.
;-)
So, it makes sense to move the discussion here.
> I am trying to build ScientificPython and it fails with the following error:
>
> Include/Scientific/arrayobject.h:2:30: fatal error: numpy/oldnumeric.h: No
> such file or directory
>
> The reason can be found in the fact that in toolchain.py it looks for
> <base_dir>/include, <base_dir>/lib and <base_dir>/lib64 to obtain the paths
> that need to be passed to the compiler.
That's not entirely accurate... The toolchain mechanism does define the build
environment ($CC, $CFLAGS, ...), but $CPATH (which specifies the locations that
the compiler should consider for header files) is defined in the generated
module files.
> Although those are the default paths for the majority of programs, python
> modules have a different path hierarchy and it fails because for numpy the
> headers are in <base_dir>/lib/python2.7/site-packages/numpy/core/include. See
> the following debug log extract for evidence.
>
> So, how would you suggest to proceed? One way would be to have a recursive
> search starting from <base_dir> coupled with a more tight check on the
> lib/include paths based not only on the directory name but also on its
> content (are there .a, .o or .h files?). That would also solve the problem of
> false positive entries like <numpy_base>/lib that does not contain library
> files at all.
I'm actually quite surprised this only pops up now... Doesn't the module file
for numpy correctly define $CPATH? (can't easily check myself atm)
Which easyconfig file are you using exactly? Maybe this is just an issue with a
too recent numpy version being used? I seem to recall that ScientificPython
requires an old version of numpy...
regards,
Kenneth
>
> Davide
>
>
>
> == 2016-11-01 12:05:51,970 toolchain.py:218 DEBUG set_variables: toolchain
> variables. Do nothing.
> == 2016-11-01 12:05:51,970 toolchain.py:647 DEBUG prepare: set additional
> variables onlymod=False
> == 2016-11-01 12:05:51,970 toolchain.py:270 DEBUG get_software_root software
> root /opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12 for Python
> was found in environment
> == 2016-11-01 12:05:51,971 toolchain.py:270 DEBUG get_software_root software
> root
> /opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12
> for numpy was found in environment
> == 2016-11-01 12:05:51,971 variables.py:542 DEBUG Passthrough to LISTCLASS
> element function append_subdirs
> == 2016-11-01 12:05:51,971 variables.py:267 DEBUG _is_protected: False value
> None (None)
> == 2016-11-01 12:05:51,971 variables.py:297 DEBUG nappend: value None
> newvalue [] position None
> == 2016-11-01 12:05:51,971 variables.py:186 DEBUG append_subdirs: base
> /opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12 subdirs
> ['include']
> == 2016-11-01 12:05:51,972 variables.py:198 DEBUG append_subdirs: added
> directory
> /opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12/include
> == 2016-11-01 12:05:51,972 variables.py:542 DEBUG Passthrough to LISTCLASS
> element function append_subdirs
> == 2016-11-01 12:05:51,972 variables.py:267 DEBUG _is_protected: False value
> None (None)
> == 2016-11-01 12:05:51,972 variables.py:297 DEBUG nappend: value None
> newvalue [] position None
> == 2016-11-01 12:05:51,972 variables.py:186 DEBUG append_subdirs: base
> /opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12 subdirs
> ['lib64', 'lib']
> == 2016-11-01 12:05:51,973 variables.py:200 WARNING flags_for_subdirs:
> directory /opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12/lib64
> was not found
> == 2016-11-01 12:05:51,973 variables.py:198 DEBUG append_subdirs: added
> directory /opt/easybuild/software/Compiler/GCC/5.4.0-2.26/Python/2.7.12/lib
> == 2016-11-01 12:05:51,973 variables.py:542 DEBUG Passthrough to LISTCLASS
> element function append_subdirs
> == 2016-11-01 12:05:51,973 variables.py:267 DEBUG _is_protected: False value
> None (None)
> == 2016-11-01 12:05:51,973 variables.py:297 DEBUG nappend: value None
> newvalue [] position None
> == 2016-11-01 12:05:51,974 variables.py:186 DEBUG append_subdirs: base
> /opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12
> subdirs ['include']
> == 2016-11-01 12:05:51,974 variables.py:200 WARNING flags_for_subdirs:
> directory
> /opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12/include
> was not found
> == 2016-11-01 12:05:51,974 variables.py:542 DEBUG Passthrough to LISTCLASS
> element function append_subdirs
> == 2016-11-01 12:05:51,974 variables.py:267 DEBUG _is_protected: False value
> None (None)
> == 2016-11-01 12:05:51,974 variables.py:297 DEBUG nappend: value None
> newvalue [] position None
> == 2016-11-01 12:05:51,975 variables.py:186 DEBUG append_subdirs: base
> /opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12
> subdirs ['lib64', 'lib']
> == 2016-11-01 12:05:51,975 variables.py:200 WARNING flags_for_subdirs:
> directory
> /opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12/lib64
> was not found
> == 2016-11-01 12:05:51,975 variables.py:198 DEBUG append_subdirs: added
> directory
> /opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12/lib
help([[NumPy is the fundamental package for scientific computing with Python. It contains among other things: a powerful N-dimensional array object, sophisticated (broadcasting) functions, tools for integrating C/C++ and Fortran code, useful linear algebra, Fourier transform, and random number capabilities. Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a wide variety of databases. - Homepage: http://www.numpy.org]]) whatis([[Description: NumPy is the fundamental package for scientific computing with Python. It contains among other things: a powerful N-dimensional array object, sophisticated (broadcasting) functions, tools for integrating C/C++ and Fortran code, useful linear algebra, Fourier transform, and random number capabilities. Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a wide variety of databases. - Homepage: http://www.numpy.org]]) local root = "/opt/easybuild/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.4/numpy/1.8.2-Python-2.7.12" conflict("numpy") if not isloaded("OpenBLAS/0.2.19-LAPACK-3.6.1") then load("OpenBLAS/0.2.19-LAPACK-3.6.1") end if not isloaded("FFTW/3.3.5") then load("FFTW/3.3.5") end if not isloaded("ScaLAPACK/2.0.2-OpenBLAS-0.2.19-LAPACK-3.6.1") then load("ScaLAPACK/2.0.2-OpenBLAS-0.2.19-LAPACK-3.6.1") end if not isloaded("Python/2.7.12") then load("Python/2.7.12") end prepend_path("PATH", pathJoin(root, "bin")) setenv("EBROOTNUMPY", root) setenv("EBVERSIONNUMPY", "1.8.2") setenv("EBDEVELNUMPY", pathJoin(root, "easybuild/MPI-GCC-5.4.0-2.26-OpenMPI-1.10.4-numpy-1.8.2-Python-2.7.12-easybuild-devel")) prepend_path("PYTHONPATH", pathJoin(root, "lib/python2.7/site-packages")) -- Built with EasyBuild version 2.9.0
numpy-1.8.2-goolf-2016.10-Python-2.7.12.eb
Description: numpy-1.8.2-goolf-2016.10-Python-2.7.12.eb
