Re: [easybuild] Python-2.7.12-foss-2016b undefined references

2017-01-18 Thread Vanzo, Davide 
Alan,
thanks for pointing me to that PR. That "--with-termlib" is the problem. With 
that option the symbols are stripped on a separate library file (libtinfo.a) 
and the linking fails when building against the ncurses-6.0.eb build.

Since this is not a local problem I have created a dedicated issue here:

https://github.com/hpcugent/easybuild-easyconfigs/issues/4049

DV


On Jan 18 2017, at 3:12 pm, Alan O'Cais  wrote:
This sounds like it might be somehow related to 
https://github.com/hpcugent/easybuild-easyconfigs/pull/3545

On 18 Jan 2017 9:05 pm, "Vanzo, Davide" 
> wrote:
When trying to build Python-2.7.12-foss-2016b it fails with the error below.

=

gcc -L/usr/software/software/Core/GCCcore/5.4.0/lib64 
-L/usr/software/software/Core/GCCcore/5.4.0/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/OpenBLAS/0.2.18-LAPACK-3.6.1/lib
 
-L/usr/software/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/ScaLAPACK/2.0.2-OpenBLAS-0.2.18-LAPACK-3.6.1/lib
 -L/usr/software/software/MPI/GCC/5.4.0-2.26/OpenMPI/1.10.3/FFTW/3.3.4/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/bzip2/1.0.6/lib 
-L/usr/software/software/Core/zlib/1.2.8/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib 
-L/usr/software/software/Core/ncurses/6.0/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/SQLite/3.13.0/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/Tk/8.6.5/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/GMP/6.1.1/lib 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/libffi/3.2.1/lib64 
-L/usr/software/software/Compiler/GCC/5.4.0-2.26/libffi/3.2.1/lib -Xlinker 
-export-dynamic -o python \
Modules/python.o \
-L. -lpython2.7 -ldl -lm -lpthread -lutil 
/usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib/libreadline.a
 /usr/software/software/Core/ncurses/6.0/lib/libncurses.a   -lm
./libpython2.7.so: error: undefined reference to 'tputs'
./libpython2.7.so: error: undefined reference to 'tgoto'
./libpython2.7.so: error: undefined reference to 
'tgetnum'
./libpython2.7.so: error: undefined reference to 'PC'
./libpython2.7.so: error: undefined reference to 'BC'
./libpython2.7.so: error: undefined reference to 'UP'
./libpython2.7.so: error: undefined reference to 
'tgetent'
./libpython2.7.so: error: undefined reference to 
'tgetstr'
./libpython2.7.so: error: undefined reference to 
'tgetflag'
collect2: error: ld returned 1 exit status
make: *** [python] Error 1

==

The weird thing is that libreadline.a contains such symbols when checking with 
readelf. Has anyone had this problem before?

On a side note, while digginig into libreadline and ncurses, I noticed that 
although we want libreadline linked to our custom version of ncurses 6.0, it 
gets actually linked to the system ncurses library. This can be solved by 
modifying the easyconfig to read as follows:

preconfigopts = 'env LDFLAGS="-lncurses -L$EBROOTNCURSES/lib"'

Is it just me having this problem or can be something worth enhancing?


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] EB for Intel Distribution for Python?

2017-01-18 Thread Jack Perdue 


On 01/18/2017 09:09 AM, Kenneth Hoste wrote:
On 18/01/2017 09:51, Ole Holm Nielsen wrote:
I would like to provide an EB module for "Intel Distribution for 
Python", see 
https://software.intel.com/en-us/intel-distribution-for-python. 
However, Intel Python isn't on the EB List of supported software.


Question: Does anyone have EB files for creating modules of Intel 
Python 2.7 and 3.5?


I haven't look into it, but my suspicion/hope is that you should be 
able to use the generic IntelBase easyblock to install Intel Python.


I'm interested to take a look into it, especially how it compares with 
the Python installations on top of Intel compilers & Intel MKL we do 
via EasyBuild.


Seems like https://github.com/IntelPython/ibench could be a good way 
of benchmarking performance there.


Funny story: I was helping out someone on the Intel Python team to get 
scipy to build (correctly) with Intel compilers a couple of months 
ago... ;-)



Howdy Ole,

FWIW, attached is a quick hack that Kenneth would never ever
approve of for _SO_ many reasons (many of them quite right).  :)

But if you just want to slap the Intel Python on your system
using EB, it should work.  Just change the version from '2.7.12'
to '3.5.2' if you want the newer one.

jack (who see no reason for an easyblock [I did try rpm.py], but does
see LOTS of problems with all the libraries the Intel Python 
provides)
# Authors:: Jack Perdue  - TAMU HPRC - http://hprc.tamu.edu

easyblock = 'ConfigureMake'

name = 'Python'
version = '2.7.12'
pysver = ''.join(version.split('.')[:2])
relver = '2017.0.035'
versionsuffix = '-%s' % relver

homepage = 'https://software.intel.com/en-us/intel-distribution-for-python'
description = """
 Intel® Distribution for Python. Powered by Anaconda.
 Accelerating Python* performance on modern architectures from Intel.

 To activate virtual environment (for user modules), run: source $PYACTIVATE

"""

# l_python27_pu_2017.0.035.tgz
sources = ['l_python%s_pu_%s.tgz' % (pysver, relver)]

toolchain = {'name': 'dummy', 'version': 'dummy'}

skipsteps = ['configure', 'build']

preinstallopts = """
cat >> easybuild.cfg << EEOOFF
ACCEPT_EULA=accept
CONTINUE_WITH_OPTIONAL_ERROR=yes
PSET_INSTALL_DIR=%(installdir)s
CONTINUE_WITH_INSTALLDIR_OVERWRITE=yes
COMPONENTS=ALL
PSET_MODE=install
NONRPM_DB_DIR=%(installdir)s
SIGNING_ENABLED=yes
EEOOFF
echo -e "install:\n\t ./install.sh --SHARED_INSTALL -s ./easybuild.cfg" > 
Makefile
"""

# unfortunately, there isn't an obvious way to remove the intelpython27 part of 
the install path

sanity_check_paths = {
'dirs': ['intelpython%s/%s' % (pysver, x) for x in ['bin', 'etc', 
'include', 'lib']],
'files': ['intelpython%s/bin/ipython' % pysver],
}

modextravars = {
'CONDA_DEFAULT_ENV': 'root',
}

modextrapaths = {
'CONDA_PREFIX': 'intelpython%s' % pysver,
'PATH': 'intelpython%s/bin' % pysver,
'LD_LIBRARY_PATH': 'intelpython%s/lib' % pysver,
'PYACTIVATE': 'intelpython%s/bin/activate' % pysver,
}

moduleclass = 'lang'
# EOF

Re: [easybuild] Python-2.7.12-foss-2016b undefined references

2017-01-18 Thread Vanzo, Davide 
Ward,
the symbols exist in the libreadline.a, not in libncurses.a as you can see here:

$ readelf --syms 
/usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib/libreadline.a
 | grep 'tputs\|tgoto\|tgetnum\|PC\|BC\|UP\|tgetent\|tgetstr\|tgetflag'
   97:  0 NOTYPE  GLOBAL DEFAULT  UND tputs
  127:  0 NOTYPE  GLOBAL DEFAULT  UND tgoto
   70:  0 NOTYPE  GLOBAL DEFAULT  UND tgetnum
   97:  0 NOTYPE  GLOBAL DEFAULT  UND PC
   99:  0 NOTYPE  GLOBAL DEFAULT  UND BC
  100:  0 NOTYPE  GLOBAL DEFAULT  UND UP
  101:  0 NOTYPE  GLOBAL DEFAULT  UND tgetent
  102:  0 NOTYPE  GLOBAL DEFAULT  UND tgetstr
  104:  0 NOTYPE  GLOBAL DEFAULT  UND tgetflag
  118:  0 NOTYPE  GLOBAL DEFAULT  UND tputs

$ readelf --syms /usr/software/software/Core/ncurses/6.0/lib/libncurses.a | 
grep 'tputs\|tgoto\|tgetnum\|PC\|BC\|UP\|tgetent\|tgetstr\|tgetflag'
   28:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp
   41:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp
   14:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp
   25:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp

Davide

On Jan 18 2017, at 2:56 pm, Ward Poelmans  wrote:

Hi Davide,

On Wed, Jan 18, 2017 at 9:05 PM, Vanzo, Davide
 wrote:
> When trying to build Python-2.7.12-foss-2016b it fails with the error below.
>
> =
> /usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib/libreadline.a
> /usr/software/software/Core/ncurses/6.0/lib/libncurses.a -lm
> ./libpython2.7.so: error: undefined reference to 'tputs'
> ./libpython2.7.so: error: undefined reference to 'tgoto'
> ./libpython2.7.so: error: undefined reference to 'tgetnum'
> ./libpython2.7.so: error: undefined reference to 'PC'
> ./libpython2.7.so: error: undefined reference to 'BC'
> ./libpython2.7.so: error: undefined reference to 'UP'
> ./libpython2.7.so: error: undefined reference to 'tgetent'
> ./libpython2.7.so: error: undefined reference to 'tgetstr'
> ./libpython2.7.so: error: undefined reference to 'tgetflag'
> collect2: error: ld returned 1 exit status
> make: *** [python] Error 1
>
> ==
>
> The weird thing is that libreadline.a contains such symbols when checking
> with readelf. Has anyone had this problem before?

Yes, I've seen this before. That's why we link libreadline to ncurses.
I'm surprise you get this error when both the static archive for
libreadline and ncurses are include.

So you are sure that those symbols exists in libncurses.a and it is defined?

Can you try what happens if you put the paths for libncurses and
libreadline before -lpython2.7?

> On a side note, while digginig into libreadline and ncurses, I noticed that
> although we want libreadline linked to our custom version of ncurses 6.0, it
> gets actually linked to the system ncurses library. This can be solved by
> modifying the easyconfig to read as follows:
>
> preconfigopts = 'env LDFLAGS="-lncurses -L$EBROOTNCURSES/lib"'
>
> Is it just me having this problem or can be something worth enhancing?

That's definitely a bug. Please open an issue on github (or pull request).

Ward

Re: [easybuild] Python-2.7.12-foss-2016b undefined references

2017-01-18 Thread Ward Poelmans 
Hi Davide,

On Wed, Jan 18, 2017 at 9:05 PM, Vanzo, Davide
 wrote:
> When trying to build Python-2.7.12-foss-2016b it fails with the error below.
>
> =
> /usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib/libreadline.a
> /usr/software/software/Core/ncurses/6.0/lib/libncurses.a   -lm
> ./libpython2.7.so: error: undefined reference to 'tputs'
> ./libpython2.7.so: error: undefined reference to 'tgoto'
> ./libpython2.7.so: error: undefined reference to 'tgetnum'
> ./libpython2.7.so: error: undefined reference to 'PC'
> ./libpython2.7.so: error: undefined reference to 'BC'
> ./libpython2.7.so: error: undefined reference to 'UP'
> ./libpython2.7.so: error: undefined reference to 'tgetent'
> ./libpython2.7.so: error: undefined reference to 'tgetstr'
> ./libpython2.7.so: error: undefined reference to 'tgetflag'
> collect2: error: ld returned 1 exit status
> make: *** [python] Error 1
>
> ==
>
> The weird thing is that libreadline.a contains such symbols when checking
> with readelf. Has anyone had this problem before?

Yes, I've seen this before. That's why we link libreadline to ncurses.
I'm surprise you get this error when both the static archive for
libreadline and ncurses are include.

So you are sure that those symbols exists in libncurses.a and it is defined?

Can you try what happens if you put the paths for libncurses and
libreadline before -lpython2.7?

> On a side note, while digginig into libreadline and ncurses, I noticed that
> although we want libreadline linked to our custom version of ncurses 6.0, it
> gets actually linked to the system ncurses library. This can be solved by
> modifying the easyconfig to read as follows:
>
> preconfigopts = 'env LDFLAGS="-lncurses -L$EBROOTNCURSES/lib"'
>
> Is it just me having this problem or can be something worth enhancing?

That's definitely a bug. Please open an issue on github (or pull request).

Ward

Re: [easybuild] EB for Intel Distribution for Python?

2017-01-18 Thread Kenneth Hoste 

Hi Ole,

On 18/01/2017 09:51, Ole Holm Nielsen wrote:
I would like to provide an EB module for "Intel Distribution for 
Python", see 
https://software.intel.com/en-us/intel-distribution-for-python. 
However, Intel Python isn't on the EB List of supported software.


Question: Does anyone have EB files for creating modules of Intel 
Python 2.7 and 3.5?


I haven't look into it, but my suspicion/hope is that you should be able 
to use the generic IntelBase easyblock to install Intel Python.


I'm interested to take a look into it, especially how it compares with 
the Python installations on top of Intel compilers & Intel MKL we do via 
EasyBuild.


Seems like https://github.com/IntelPython/ibench could be a good way of 
benchmarking performance there.


Funny story: I was helping out someone on the Intel Python team to get 
scipy to build (correctly) with Intel compilers a couple of months 
ago... ;-)



regards,

Kenneth

[easybuild] EB for Intel Distribution for Python?

2017-01-18 Thread Ole Holm Nielsen 
I would like to provide an EB module for "Intel Distribution for 
Python", see 
https://software.intel.com/en-us/intel-distribution-for-python. 
However, Intel Python isn't on the EB List of supported software.


Question: Does anyone have EB files for creating modules of Intel Python 
2.7 and 3.5?


Thanks,
Ole