Hi Ben, To maintain different subsets of R and Python libraries I suggest you look at the "bundle" easyblock. This is how I am handling this now. Before I was using your approach of adding all the R libraries in the main R easyconfig but adding all the extensions in the main R easyconfig becomes hard to maintain. I had more than 400 R packages in my R easyconfig and upgrading this was a mess. Having bundles you can have smaller subsets of libraries and you can also add new bundles and keep the old ones. Here you have some examples
https://github.com/hpcugent/easybuild-easyconfigs/blob/242a3e19308754796c7dd800d2f9f6f40f6c9e14/easybuild/easyconfigs/r/R-bundle-extra/R-bundle-extra-20150323-intel-2015a-R-3.1.3.eb https://github.com/hpcugent/easybuild-easyconfigs/blob/7787c2c75443ce222443489e4e8ff6b9dffd21d5/easybuild/easyconfigs/i/IPython/IPython-3.2.3-intel-2015b-Python-2.7.10.eb When managing R bundles which contain bioconductor packages there is a problem to distribute them because the bioconductor guys remove old versions from their repository so if you have a working bundle today it will fail downloading the required libraries few days later when the bioconductor mirror publish new versions and remove the old ones. Kenneth (EB release manager) tried to contact them and asked to keep the old versions online but they are not willing to do it. What galaxy developers have done to workaround this problem is building their own bioconductor mirror which also includes old versions ( https://bioarchive.galaxyproject.org/#/pkg/Biobase/). But this is not a complete mirror, it only includes the R packages used in galaxy ( https://galaxyproject.org/) . There has been discussion before in the EasyBuild community about maintaining a mirror with the source tarballs but nothing is in place yet. To be able to distribute R bundles I think something like this would be needed. If you are managing R installation with easybuild maybe you are interested in this https://github.com/fgeorgatos/easybuild.experimental/tree/master/users/pneerincx . It's not merged upstream but it works. The developer of this script ( https://github.com/pneerincx) is in this mailing list. What I do is to use this script to generate an R easyconfig and then I only use the extensions list section to create a R bundle. regards, Pablo. 2016-02-17 23:49 GMT+01:00 McGough, Benjamin E <[email protected]>: > We have begun to use EasyBuild to replace our manual build process for > scientific software. As we have learned more and more about EasyBuild, a > couple of questions have come up. > > First, I am 95% positive I read a 'best practices' for citing > EasyBuild/EasyConfigs... but I cannot find it anywhere now. If such exists > in the documentation, please point me to it. > > I believe we will have need to cite the version of EasyBuild and the > specific easyconfig used to build software involved in published research. > Would it be wise to cite github URLs for EasyBuild and easyconfigs? We may > also have a need to cite easyconfigs that have not been accepted into the > easyconfigs repository, but would like to use a generic URL not associated > with our institution. Is there a common practice for this? > > Second, we have begun to build R and Python with an expanded set of > included extensions (libraries/modules). We intend to keep this up for new > versions of R and Python, including version updates for the extensions and > inclusion of new extensions by request and dependency. My initial feeling > is that this might be useful to others, and I had planned to submit these > easyconfigs with a versionsuffix of something like 'life-sciences' to > indicate the extensions selection is primarily life science-driven. Now I > am not sure that is the right way to proceed. Is there a common practice I > should read? We also have a python script using APIs from CRAN and PyPI to > update the easyconfigs. Is that something that would be a useful tool to > others? > > Thanks! > > Ben McGough > System Administrator > Center IT/Scientific Computing > O 206.667.7818 > [email protected] > > 1100 Fairview Ave. N. > P.O. Box 19024 > Seattle, WA 98109 > > Fred Hutch / Cures Start Here > fredhutch.org > -- Pablo Escobar López HPC systems engineer sciCORE, University of Basel SIB Swiss Institute of Bioinformatics http://scicore.unibas.ch
