as Alan suggests I think python bundle is the way to go for what you want. I would also suggest that if you want to assure reproducibility you should test your bundles in a machine without internet access. Some python libraries download extra dependencies during installation so if you run your easyconfig in a machine with internet access you won't notice which extra deps have been downloaded during the installation and if you run the same easyconfig in the future maybe different versions of the dependencies can be downloaded. If you try the easyconfig without internet access you would get an error in case any extra dependency needs to be download so you can check the error log and add the missing dependency to your bundle. The only procedure I know now to address this issue is by try&error in a machine without internet access. Some development is being done to address this in easybuild in a cleaner way but nothing available in the main easybuild release yet as far as I know.
regards, Pablo. 2016-03-18 14:42 GMT+01:00 [email protected] <[email protected]>: > Thanks, Alan, > > I'll look into this following your numba example. > > I'm just wondering, shouldn't for example the numpy version be part of > the toolchain definition? Libraries like pandas and scipy use the numpy > C stuff, if I remember correctly, so that a pandas build with numpy 1.9 > is different from a pandas build with numpy 1.10. Not sure if numba > from your example should also suffer from this or not. > > Cheers, > Andreas. > > > > Alan O'Cais writes: > > > Dear Andreas, > > > > I was actually going to look into this soon. What you are describing (I > think) is a Python package bundle for the SciPy Stack< > http://www.scipy.org/stackspec.html>. An example of what that might look > like is > > > https://github.com/hpcugent/easybuild-easyconfigs/blob/develop/easybuild/easyconfigs/n/numba/numba-0.22.1-intel-2015b-Python-2.7.11.eb > > The Python dependency is given by the version suffix and the blas > dependency is provided by the toolchain itself (a toolchain is > compiler+mpi+blas/lapack/fftw) > > > > You could work on this yourself based on that example and I'd be happy > to help if you run into issues. > > > > Alan > > > > On 18 March 2016 at 14:20, <[email protected]<mailto: > [email protected]>> wrote: > > I recently stumbled across easybuild, it's an amazing piece of > > software -- thanks for making this available! > > > > I'm wondering if there is any documentation / hints available for > > managing different version of the Python scientific toolstack with > > easybuild. I'm thinking of toolchains like > > "py27-np19-sp016-pd017-openblas" -- you get my point. So basically I'm > > looking at using easybuild as a replacement for virtualenvs / Anaconda > > envs. > > > > Is there anything I need to consider for this use case? > > > > Cheers, > > Andreas. > > > > -- > > Dr. Andreas Hilboll > > > > Center for Marine Environmental Sciences (MARUM) > > - AND - > > Institute of Environmental Physics (IUP) > > > > University of Bremen > > > > U3145 > > Otto-Hahn-Allee 1 > > D-28359 Bremen > > Germany > > > > +49(0)421 218 62133<tel:%2B49%280%29421%20218%2062133> (phone) > > +49(0)421 218 98 62133<tel:%2B49%280%29421%20218%2098%2062133> (fax) > > http://www.iup.uni-bremen.de/~hilboll > > > -- > Dr. Andreas Hilboll > > Center for Marine Environmental Sciences (MARUM) > - AND - > Institute of Environmental Physics (IUP) > > University of Bremen > > U3145 > Otto-Hahn-Allee 1 > D-28359 Bremen > Germany > > +49(0)421 218 62133 (phone) > +49(0)421 218 98 62133 (fax) > http://www.iup.uni-bremen.de/~hilboll > -- Pablo Escobar López HPC systems engineer sciCORE, University of Basel SIB Swiss Institute of Bioinformatics http://scicore.unibas.ch
