Yes, that got me past the build error (now on the numpy and dotblas issues).
Would using a python virtualenv help to deal with these issues? Ben McGough System Administrator / Center IT - Scientific Computing / 206.667.7818 / [email protected] / Fred Hutch / Cures Start Here ----- Original Message ----- From: "Kenneth Hoste" <[email protected]> To: "easybuild" <[email protected]> Sent: Saturday, December 12, 2015 2:44:54 AM Subject: Re: [easybuild] Python build problem - pulling in local packages Hi Ben, On 09/12/15 00:10, McGough, Benjamin E wrote: > I've tracked the problem down to the numpy easyblock. Specifically line > 202-207: > 202 cmd = ' '.join([ > 203 'export PYTHONPATH=%s:$PYTHONPATH &&' % > os.path.join(tmpdir, self.pylibdir), > 204 'python -m timeit -n 3 -r 3', > 205 '-s "import numpy; x = numpy.random.random((%(size)d, > %(size)d))"' % {'size': size}, > 206 '"numpy.dot(x, x.T)"', > 207 ]) > > It appears to me that the 'import numpy' here is pulling from > /usr/lib/python2.7/dist-packages/numpy rather than the /tmp/eb- directory > where the recently built numpy seems to reside: > > == 2015-12-08 14:35:57,704 main ERROR cmd "export > PYTHONPATH=/tmp/eb-shWoxr/tmpFuf7w4/lib/python2.7/site-packages:$PYTHONPATH > && python -m timeit -n 3 -r 3 -s "import numpy; x = > numpy.random.random((1000, 1000))" "numpy.dot(x, x.T)"" exited with exitcode > 1 and output: > Traceback (most recent call last): > File > "/app/easybuild/software/Python/2.7.9-goolf-1.7.20/lib/python2.7/timeit.py", > line 309, in main > r = t.repeat(repeat, number) > File > "/app/easybuild/software/Python/2.7.9-goolf-1.7.20/lib/python2.7/timeit.py", > line 223, in repeat > t = self.timeit(number) > File > "/app/easybuild/software/Python/2.7.9-goolf-1.7.20/lib/python2.7/timeit.py", > line 195, in timeit > timing = self.inner(it, self.timer) > File "<timeit-src>", line 3, in inner > import numpy; x = numpy.random.random((1000, 1000)) > File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 153, in > <module> > from . import add_newdocs > File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in > <module> > from numpy.lib import add_newdoc > File "/usr/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 8, in > <module> > from .type_check import * > File "/usr/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, > in <module> > import numpy.core.numeric as _nx > File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 6, in > <module> > from . import multiarray > ImportError: /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so: > undefined symbol: PyUnicodeUCS4_AsUnicodeEscapeString > > The undefined symbol seems to be related to pulling the wrong numpy. > > I'm going to try uninstalling numpy from the OS at this point, but I can't > figure out how it is importing that numpy. My environment (PYTHONPATH) is > clean. Does https://github.com/hpcugent/easybuild-framework/issues/1409 help? I'm trying to figure out a solution to this, it's been popping up in way too many places... regards, Kenneth > > Ben McGough > System Administrator / Center IT - Scientific Computing / 206.667.7818 / > [email protected] / Fred Hutch / Cures Start Here > > ----- Original Message ----- > From: "Benjamin E McGough" <[email protected]> > To: "easybuild" <[email protected]> > Sent: Monday, November 23, 2015 12:39:26 PM > Subject: [easybuild] Python build problem - pulling in local packages > > I am hitting this issue with multiple toolchains. > > It appears that during a Python build (I've tried both 2.7.9 and 3.4.3), I am > pulling in numpy from /usr/lib, which is causing undefined symbol errors. > Relevant logs below. > > I am building with EasyBuild 2.3.0 and using an EasyConfig from the repo, > with --try-toolchain. This happens with both foss-2015b and intel-2015b. > > == 2015-11-23 11:05:37,638 main.run INFO parse_log_for_error msg: Command > used: python setup.py install --prefix=/tmp/eb-NNynmQ/tmpnSaYD4 > == 2015-11-23 11:05:37,638 main.run INFO parse_log_for_error (some may be > harmless) regExp (?<![(,-]|\w)(?:error|segmentation > fault|failed)(?![(,-]|\.?\w) found: > _configtest.c:6: error: undefined reference to 'exp' > collect2: error: ld returned 1 exit status > _configtest.c:6: error: undefined reference to 'exp' > collect2: error: ld returned 1 exit status > == 2015-11-23 11:05:37,638 main.run WARNING Found 4 errors in command output > (output: _configtest.c:6: error: undefined reference to 'exp', collect2: > error: ld returned 1 exit status, _configtest.c:6: error: undefined reference > to 'exp', collect2: error: ld returned 1 exit status) > == 2015-11-23 11:05:37,835 main ERROR cmd "export > PYTHONPATH=/tmp/eb-NNynmQ/tmpnSaYD4/lib/python2.7/site-packages:$PYTHONPATH > && python -m timeit -n 3 -r 3 -s "import numpy; x = > numpy.random.random((1000, 1000))" "numpy.dot(x, x.T)"" exited with exitcode > 1 and output: > Traceback (most recent call last): > File > "/app/easybuild/software/Python/2.7.9-foss-2015b/lib/python2.7/timeit.py", > line 309, in main > r = t.repeat(repeat, number) > File > "/app/easybuild/software/Python/2.7.9-foss-2015b/lib/python2.7/timeit.py", > line 223, in repeat > t = self.timeit(number) > File > "/app/easybuild/software/Python/2.7.9-foss-2015b/lib/python2.7/timeit.py", > line 195, in timeit > timing = self.inner(it, self.timer) > File "<timeit-src>", line 3, in inner > import numpy; x = numpy.random.random((1000, 1000)) > File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 153, in > <module> > from . import add_newdocs > File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in > <module> > from numpy.lib import add_newdoc > File "/usr/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 8, in > <module> > from .type_check import * > File "/usr/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, > in <module> > import numpy.core.numeric as _nx > File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 6, in > <module> > from . import multiarray > ImportError: /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so: > undefined symbol: PyUnicodeUCS4_AsUnicodeEscapeString > (at > easybuild/software/EasyBuild/2.3.0/lib/python2.7/site-packages/easybuild_framework-2.3.0-py2.7.egg/easybuild/tools/run.py:360 > in parse_cmd_output) > == 2015-11-23 11:05:37,836 main.easyblock WARNING build failed (first 300 > chars): cmd "export > PYTHONPATH=/tmp/eb-NNynmQ/tmpnSaYD4/lib/python2.7/site-packages:$PYTHONPATH > && python -m timeit -n 3 -r 3 -s "import numpy; x = > numpy.random.random((1000, 1000))" "numpy.dot(x, x.T)"" exited with exitcode > 1 and output: > Traceback (most recent call last): > File "/app/easybuild/software/Py > == 2015-11-23 11:05:37,836 main.EB_Python INFO Closing log for application > name Python version 2.7.9 > > Ben McGough > System Administrator > Center IT/Scientific Computing > O 206.667.7818 > [email protected] > > 1100 Fairview Ave. N. > P.O. Box 19024 > Seattle, WA 98109 > > Fred Hutch / Cures Start Here > fredhutch.org
