Hi Stuart,
On 03/03/15 20:15, Stuart Barkley wrote:
I was running into a build problem with Python similar to the one
described in:
https://github.com/hpcugent/easybuild/issues/23
The cause in my case is that my system does not have any Internet
connectivity and cannot pull anything from the Internet at build time.
I had found this problem report back a few months and for the interim
had just removed dateutil (and paramiko) from the python builds to be
able to progress further.
The patch for dateutil was pretty simple, just including 'six' was
sufficient. A similar problem occurs with paramiko. I had to add
'ecdsa' for it.
This issue has been tackled for (some) recent Python easyconfigs that
are currently in the develop branch of the easyconfigs repository, see
for example
https://github.com/hpcugent/easybuild-easyconfigs/blob/develop/easybuild/easyconfigs/p/Python/Python-2.7.8-intel-2015a.eb
.
But, this change has not been applied across all existing easyconfigs
(yet), however...
Unfortunately, there's no easy way to prevent this auto-downloading and
installing dependencies when installing a Python package with "python
setup.py install".
There are some ways (e.g., doing the build in a jail-like environment),
but they're not trivial.
I'll see what I can do to get six and ecdsa properly included where
needed, as dependencies for dateutil and paramiko, resp., thanks for
bringing this up.
The patch is:<snip>
It looks like this could/should also be applied against all of the
other python-2.7.X easybuild files. There appear to be some other
inconsistencies in the python-2.7.X easybuild files which could be
normalized.
In my environment I have also updated dateutil from 2.1 to 2.4.0 and
zlib from 1.2.7 to 1.2.8. For paramiko I may also need other
libraries for gssapi, but that started to get into a larger set of
additional dependencies.
Just to clarify: the easyconfigs that are part of the EasyBuild
installation are meant to be treated as examples (which does not mean
they should contain inconsistencies like the one you reported).
So, feel free to adjust easyconfigs to your liking for your local
installations.
For now, we stay away from bumping versions of dependencies/extensions
in existing easyconfigs (i.e. ones that have been merged into develop),
to keep things fixed.
This may change once we have support for specifying non-strict
(dependency) versions, but no work has been done on that yet.
I also see a python-dateutil-2.1-goolf-1.4.10-Python-2.7.3.eb
configuration file. It looks like this would be redundant with the
current easybuild files, but could serve as a good example for users
building python addons for individual use.
This one mainly serves as an example, yes.
However, there are other example of similar easyconfigs that do make sense.
See numpy-1.7.1-goolf-1.4.10-Python-2.7.3.eb for example.
The numpy that is included in Python-2.7.3-goolf-1.4.10.eb is older
(v1.6.1), so a seperate easyconfig file for numpy allows to install a
newer version of numpy than the one that comes with the Python module.
These changes just allow building to complete successfully. I have
not started testing any applications beyond the build process.
Some builds include tests, like the numpy easyblock that checks the
performance of numpy.dot in an attempt to ensure that numpy was
correctly linked with BLAS
(see
https://github.com/hpcugent/easybuild-easyblocks/blob/master/easybuild/easyblocks/n/numpy.py#L197).
And that's beside the unit test suite which is also run.
Lately, we've been paying more attention to including (simple) tests in
the build procedure as executed by EasyBuild, but we should do this
more, where possible.
regards,
Kenneth
