First, in reply to Jens, the _ssl module is installed; shows up under “help(‘modules’)”, and:
heywood@bnbmgmt2:~$ python
Python 2.7.2 (default, Jun 14 2012, 09:57:05)
[GCC 4.4.6 20110731 (Red Hat 4.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import _ssl
>>>
And that show the older Python that is part of our “base” install. I am using
easy build for extra, optional, or more up to date installs of specific things
with specific dependencies differing from our base (toolchains are a good
concept in this scenario). Like Python-2.7.5 and 3, and R-3.
I ran this: "eb -fd -r $EBCONFIGS ./Python-2.7.5-goalf-1.5.12-no-OFED.eb “,
where the last 2 extensions are:
('distribute', '0.6.28', {
'source_urls':
['https://pypi.python.org/packages/source/d/distribute/'],
}),
('matplotlib', '1.3.1', {
'source_urls':
['https://downloads.sourceforge.net/project/matplotlib/matplotlib/matplotlib-1.3.1/'],
}),
Got this, running with the debug option:
heywood@bnbmgmt2:/sonas-hs/it/hpc/home/eb/ebfiles/configs/cshl$ eb -fd -r
$EBCONFIGS ./Python-2.7.5-goalf-1.5.12-no-OFED.eb
== temporary log file in case of crash
/tmp/easybuild-SMsBQ5/easybuild-tDl804.log
== resolving dependencies ...
== processing EasyBuild easyconfig
/sonas-hs/it/hpc/home/eb/ebfiles/configs/cshl/Python-2.7.5-goalf-1.5.12-no-OFED.eb
== building and installing Python/2.7.5-goalf-1.5.12-no-OFED...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
ERROR: EasyBuild encountered an exception (at
easybuild/software/EasyBuild/1.10.0/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/main.py:809
in build_and_install_software): autoBuild Failed (last 300 chars):
D/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.py",
line 1247, in unknown_open
raise URLError('unknown url type: %s' % type)
urllib2.URLError: <urlopen error unknown url type: https>
heywood@bnbmgmt2:/sonas-hs/it/hpc/home/eb/ebfiles/configs/cshl$
Log file attached.
Thanks,
Todd
From: Stijn De Weirdt <[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Wednesday, February 12, 2014 at 3:12 AM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: [easybuild] trouble with python modules and R packages built as
extensions
hi todd,
this:
> urllib2.URLError: <urlopen error unknown url type: https>
what python version is EB using to run? (if it is an OS provided one,
can you add the full package version/) and can you produce a debug run
and send us the output?
stijn
On 02/11/2014 06:37 PM, Heywood, Todd wrote:
First, thanks for help on the earlier issue of module unloading, so that I
can give you guys another issue :-).
I am trying to build new versions of Python and R, and have long lists of
required modules and packages. I¹m adding lines under the exts_list
section of .eb files.
I often get crashes due to "unknown url type² errors, which I think come
from modules/packages trying to go find moudles/packages they depend on,
and download them. I have been able to get by some by finding exact URL¹s
and versions for dependencies and putting lines in the exts_list for these
before the module/package in question. However, this is not working for
one Python module I¹m trying to install now, and I figure I should ask
about the real issue here instead of just trying to work around it.
So, trying to install the matplotlib Python module with (eb ‹robot) with
this:
('matplotlib', '1.3.1', {
'source_urls':
['https://downloads.sourceforge.net/project/matplotlib/matplotlib/matplotli
b-1.3.1/'],
}),
The output, and log file excerpt are below. Matplotlib is apparently
trying to download the ³distribute² dependency module. So I put this line
above the matplotlib line:
('distribute', '0.6.28', {
'source_urls': ['https://pypi.python.org/packages/source/d/distribute/'],
}),
But the eaasybuild install still crashes with the same error.
Output from build:
== taking care of extensions...
ERROR: EasyBuild encountered an exception (at
easybuild/software/EasyBuild/1.10.0/lib/python2.7/site-packages/easybuild_f
ramework-1.10.0-py2.7.egg/easybuild/main.py:809 in
build_and_install_software): autoBuild Failed (last 300 chars):
D/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 1247, in unknown_open
raise URLError('unknown url type: %s' % type)
urllib2.URLError: <urlopen error unknown url type: https>
From the log file:
== 2014-02-11 11:35:44,089 main.filetools ERROR EasyBuild crashed with an
error (at
easybuild/software/EasyBuild/1.10.0/lib/python2.7/site-packages/easybuild_f
ramework-1.10.0-py2.7.egg/easybuild/tools/fil\
etools.py:799 in parse_cmd_output): cmd "python setup.py build" exited
with exitcode 1 and output:
Downloading
http://pypi.python.org/packages/source/d/distribute/distribute-0.6.28.tar.g
z
Traceback (most recent call last):
File "setup.py", line 10, in <module>
use_setuptools()
File
"/sonas-hs/it/hpc/home/eb/build/Python/2.7.5/goalf-1.5.12-no-OFED.10/matplo
tlib/matplotlib-1.3.1/distribute_setup.py", line 163, in use_setuptools
return _do_download(version, download_base, to_dir, download_delay)
File
"/sonas-hs/it/hpc/home/eb/build/Python/2.7.5/goalf-1.5.12-no-OFED.10/matplo
tlib/matplotlib-1.3.1/distribute_setup.py", line 132, in _do_download
to_dir, download_delay)
File
"/sonas-hs/it/hpc/home/eb/build/Python/2.7.5/goalf-1.5.12-no-OFED.10/matplo
tlib/matplotlib-1.3.1/distribute_setup.py", line 212, in
download_setuptools
src = urlopen(url)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 127, in urlopen
return _opener.open(url, data, timeout)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 410, in open
response = meth(req, response)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 523, in http_response
'http', request, response, code, msg, hdrs)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 442, in error
result = self._call_chain(*args)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 382, in _call_chain
result = func(*args)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 629, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 404, in open
response = self._open(req, data)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 427, in _open
'unknown_open', req)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 382, in _call_chain
result = func(*args)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 1247, in unknown_open
raise URLError('unknown url type: %s' % type)
urllib2.URLError: <urlopen error unknown url type: https>
== 2014-02-11 11:35:44,090 main ERROR EasyBuild crashed with an error (at
easybuild/software/EasyBuild/1.10.0/lib/python2.7/site-packages/easybuild_f
ramework-1.10.0-py2.7.egg/easybuild/tools/build_log.py:\
96 in exception): EasyBuild encountered an exception (at
easybuild/software/EasyBuild/1.10.0/lib/python2.7/site-packages/easybuild_f
ramework-1.10.0-py2.7.egg/easybuild/main.py:809 in build_and_install_sof\
tware): autoBuild Failed (last 300 chars): D/lib/python2.7/urllib2.py",
line 382, in _call_chain
result = func(*args)
File
"/opt/eb/software/Python/2.7.5-goalf-1.5.12-no-OFED/lib/python2.7/urllib2.p
y", line 1247, in unknown_open
raise URLError('unknown url type: %s' % type)
urllib2.URLError: <urlopen error unknown url type: https>
Any ideas? Both for this specific module, and the fact that ³unknown url
type² is very common?
Thanks,
Todd
easybuild-tDl804.log.bz2
Description: easybuild-tDl804.log.bz2
