Hi Kenneth,
Thanks for explaining. I just have on more question about the installation of
Cython on the UGent clusters. I noticed that Cython is present in the list if
packages that gets installed with Python-2.7.3-ictce-4.1.13. See e.g.
https://github.com/hpcugent/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/p/Python/Python-2.7.3-ictce-4.1.13.eb
However, when loading this module on gengar, cython is apparently not
installed. I think that was one of the reasons I created a separate Cython
config file. Cython is also a package that evolves very rapidly with a
(significant) new release every one or two months. So, I'll include these
configs in a pull request, right?
Best Regards,
Toon
On 08/05/2013 11:47 AM, Kenneth Hoste wrote:
Hi Toon,
Sorry for the rather late reply, I'm picking up on mails after the summer
holidays...
On 12 Jul 2013, at 08:55, Toon Verstraelen wrote:
On 07/03/2013 07:50 PM, Fotis Georgatos wrote:
Hi Toon,
On Jul 3, 2013, at 5:21 PM, Toon Verstraelen wrote:
That worked. Thanks!
Please submit to easybuild git repo with a Pull Request, if you feel ready.
Hello,
I'm still testing things (now with eaasybuild 1.6.0). Once that is ready, I'll
make a pull request. I still have a few other questions.
- I need a newer version of Cython, which is easy enough, i.e. just create a
new eb file. However, this quickly becomes a mess, e.g. when different programs
start to depend on different versions of some library or tool, their modules
can (often) not be loaded together. Is there a way to avoid such mess?
Unfortunately, no. One way to try and avoid problems like that is to bundle
common dependencies together in a module, and then depend on that module.
One example is biodeps, see for example
https://github.com/hpcugent/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/b/biodeps/biodeps-1.6-ictce-5.3.0.eb
.
Note that this is no different from installing tools system-wide (without
modules): if two packages need conflicting versions, you're screwed.
The good news is that this can be resolved with modules: just provide another
module that depends on the other version of the tool you need, and you're good.
The downside is that this quickly leads to an explosion of modules (and
potentially also module names to differentiate between the modules).
The unit test suite in the easyconfigs repository (run it with "python -m
test.easyconfigs,suite) checks for conflicts, so use that if you're not 100% sure
you're not introducing conflicts.
- I noticed that the HDF5 library also comes in a gpfs flavor. How should one
decide whether a package (in my case, a newer h5py) depends on the gpfs version
or not?
Up to you... If you're going to use it on a system that uses GPFS shared
filesystems (e.g. the HPC-UGent infrastructure), then use the GPFS version.
It's probably OK to use the GPFS version in both cases. I think that one
includes a patch for HDF5 to ensure better performance on GPFS (which is
probably harmless on non-GPFS filesystems, I'd guess).
- I had to patch libcx due to a silly bug. Where should one put such patch
files?
Next to the easyconfig files, see for example
https://github.com/hpcugent/easybuild-easyconfigs/tree/master/easybuild/easyconfigs/p/Python
.
EasyBuild will try and find patch files in a number of locations: in the same
directory as the easyconfig file, in the same directory as the sources, etc.
For patch files that are required by easyconfig files included in the
easybuild-easyconfigs repository, they're always included next to the
easyconfig file.
In the unit test suite, we check whether all patch files are present there. If
not, it breaks, and we know about it.
regards,
Kenneth
