Hi Ben,
On 09/12/15 00:10, McGough, Benjamin E wrote:
I've tracked the problem down to the numpy easyblock. Specifically line 202-207:
202 cmd = ' '.join([
203 'export PYTHONPATH=%s:$PYTHONPATH &&' %
os.path.join(tmpdir, self.pylibdir),
204 'python -m timeit -n 3 -r 3',
205 '-s "import numpy; x = numpy.random.random((%(size)d,
%(size)d))"' % {'size': size},
206 '"numpy.dot(x, x.T)"',
207 ])
It appears to me that the 'import numpy' here is pulling from
/usr/lib/python2.7/dist-packages/numpy rather than the /tmp/eb- directory where
the recently built numpy seems to reside:
== 2015-12-08 14:35:57,704 main ERROR cmd "export
PYTHONPATH=/tmp/eb-shWoxr/tmpFuf7w4/lib/python2.7/site-packages:$PYTHONPATH && python -m timeit -n 3 -r 3 -s
"import numpy; x = numpy.random.random((1000, 1000))" "numpy.dot(x, x.T)"" exited with
exitcode 1 and output:
Traceback (most recent call last):
File
"/app/easybuild/software/Python/2.7.9-goolf-1.7.20/lib/python2.7/timeit.py",
line 309, in main
r = t.repeat(repeat, number)
File
"/app/easybuild/software/Python/2.7.9-goolf-1.7.20/lib/python2.7/timeit.py",
line 223, in repeat
t = self.timeit(number)
File
"/app/easybuild/software/Python/2.7.9-goolf-1.7.20/lib/python2.7/timeit.py",
line 195, in timeit
timing = self.inner(it, self.timer)
File "<timeit-src>", line 3, in inner
import numpy; x = numpy.random.random((1000, 1000))
File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 153, in
<module>
from . import add_newdocs
File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in
<module>
from numpy.lib import add_newdoc
File "/usr/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 8, in
<module>
from .type_check import *
File "/usr/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, in
<module>
import numpy.core.numeric as _nx
File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 6, in
<module>
from . import multiarray
ImportError: /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so:
undefined symbol: PyUnicodeUCS4_AsUnicodeEscapeString
The undefined symbol seems to be related to pulling the wrong numpy.
I'm going to try uninstalling numpy from the OS at this point, but I can't
figure out how it is importing that numpy. My environment (PYTHONPATH) is clean.
Does https://github.com/hpcugent/easybuild-framework/issues/1409 help?
I'm trying to figure out a solution to this, it's been popping up in way
too many places...
regards,
Kenneth
Ben McGough
System Administrator / Center IT - Scientific Computing / 206.667.7818 /
[email protected] / Fred Hutch / Cures Start Here
----- Original Message -----
From: "Benjamin E McGough" <[email protected]>
To: "easybuild" <[email protected]>
Sent: Monday, November 23, 2015 12:39:26 PM
Subject: [easybuild] Python build problem - pulling in local packages
I am hitting this issue with multiple toolchains.
It appears that during a Python build (I've tried both 2.7.9 and 3.4.3), I am
pulling in numpy from /usr/lib, which is causing undefined symbol errors.
Relevant logs below.
I am building with EasyBuild 2.3.0 and using an EasyConfig from the repo, with
--try-toolchain. This happens with both foss-2015b and intel-2015b.
== 2015-11-23 11:05:37,638 main.run INFO parse_log_for_error msg: Command used:
python setup.py install --prefix=/tmp/eb-NNynmQ/tmpnSaYD4
== 2015-11-23 11:05:37,638 main.run INFO parse_log_for_error (some may be
harmless) regExp (?<![(,-]|\w)(?:error|segmentation
fault|failed)(?![(,-]|\.?\w) found:
_configtest.c:6: error: undefined reference to 'exp'
collect2: error: ld returned 1 exit status
_configtest.c:6: error: undefined reference to 'exp'
collect2: error: ld returned 1 exit status
== 2015-11-23 11:05:37,638 main.run WARNING Found 4 errors in command output
(output: _configtest.c:6: error: undefined reference to 'exp', collect2: error:
ld returned 1 exit status, _configtest.c:6: error: undefined reference to
'exp', collect2: error: ld returned 1 exit status)
== 2015-11-23 11:05:37,835 main ERROR cmd "export
PYTHONPATH=/tmp/eb-NNynmQ/tmpnSaYD4/lib/python2.7/site-packages:$PYTHONPATH && python -m timeit -n 3 -r 3 -s
"import numpy; x = numpy.random.random((1000, 1000))" "numpy.dot(x, x.T)"" exited with
exitcode 1 and output:
Traceback (most recent call last):
File
"/app/easybuild/software/Python/2.7.9-foss-2015b/lib/python2.7/timeit.py", line
309, in main
r = t.repeat(repeat, number)
File
"/app/easybuild/software/Python/2.7.9-foss-2015b/lib/python2.7/timeit.py", line
223, in repeat
t = self.timeit(number)
File
"/app/easybuild/software/Python/2.7.9-foss-2015b/lib/python2.7/timeit.py", line
195, in timeit
timing = self.inner(it, self.timer)
File "<timeit-src>", line 3, in inner
import numpy; x = numpy.random.random((1000, 1000))
File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 153, in
<module>
from . import add_newdocs
File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in
<module>
from numpy.lib import add_newdoc
File "/usr/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 8, in
<module>
from .type_check import *
File "/usr/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, in
<module>
import numpy.core.numeric as _nx
File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 6, in
<module>
from . import multiarray
ImportError: /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so:
undefined symbol: PyUnicodeUCS4_AsUnicodeEscapeString
(at
easybuild/software/EasyBuild/2.3.0/lib/python2.7/site-packages/easybuild_framework-2.3.0-py2.7.egg/easybuild/tools/run.py:360
in parse_cmd_output)
== 2015-11-23 11:05:37,836 main.easyblock WARNING build failed (first 300 chars): cmd "export
PYTHONPATH=/tmp/eb-NNynmQ/tmpnSaYD4/lib/python2.7/site-packages:$PYTHONPATH && python -m timeit -n 3 -r 3 -s
"import numpy; x = numpy.random.random((1000, 1000))" "numpy.dot(x, x.T)"" exited with
exitcode 1 and output:
Traceback (most recent call last):
File "/app/easybuild/software/Py
== 2015-11-23 11:05:37,836 main.EB_Python INFO Closing log for application name
Python version 2.7.9
Ben McGough
System Administrator
Center IT/Scientific Computing
O 206.667.7818
[email protected]
1100 Fairview Ave. N.
P.O. Box 19024
Seattle, WA 98109
Fred Hutch / Cures Start Here
fredhutch.org
