Hi Kenneth, On Jun 24, 2015, at 11:10 AM, Kenneth Hoste <[email protected]> wrote:
> Hi Pieter, > > On 24/06/15 10:58, Pieter Neerincx wrote: >> Hi, >> >> On Jun 24, 2015, at 7:34 AM, Kenneth Hoste <[email protected]> wrote: >> >>> Hi Riccardo, >>> >>> On 19/06/15 10:32, Riccardo Murri wrote: >>>> Hi Ken, all, >>>> >>>> (Kenneth Hoste, Thu, Jun 18, 2015 at 04:32:27PM +0200:) >>>>> We deliberately do not enable 'dependency=True', to ensure we can >>>>> reproduce >>>>> the exact same installation later; if you let R automagically resolve >>>>> dependencies and let it pick the most recent version at that time for the >>>>> dependencies, you can't do an exact reproduced install. >>>> Could this be worked around by having EB save the list of installed R >>>> packages + versions (like `pip freeze`) upon first install, then re-use >>>> the saved list if available. >>>> >>>> Something along these lines (pseudo-code): >>>> >>>> package_list_file = easyconfig_file_name.replace('.eb', >>>> '.pkglist.csv') >>>> if not exists(package_list_file): >>>> # 1. install packages by name, let R choose version >>>> # 2. dump list of installed packages to file: (pkgname, version) >>>> else: >>>> # 1. read in the pkglist.csv file >>>> # 2. use R's install_version() to install the exact same pkg >>>> and version >>> I like this idea... >> +1 >> >>> We could have an easyconfig parameter like 'auto_deps = True', and when >>> that is set, EasyBuild lets R resolve dependencies by itself. >>> >>> Once the full list of easyconfigs has been installed, EasyBuild could then >>> see what got installed, and which versions, and adjust the list of >>> extensions included in the easyconfig file. The tricky bit may be the >>> order, but there's probably a solution for that (see Stefano's R script). >>> >>> Now, who's up for implementing this? :-) >> Yesterday I started working on an R-script that generates an *.eb from an >> existing R installation. This is based on Stefano's code (Many thanks for >> sharing that!) and adds some commandline switches, help and code to figure >> out whether a package came from a CRAN repo or from BioConductor. Hence this >> is slightly different compared to the above as it generates a complete *.eb >> from pure R, but I can easily add an option to only generate a pkglist file, >> which may then be used by an existing *.eb. >> >> Is it Ok if I add this with a README to the ..../r/R/ dir in the EasyConfigs >> repo and create a pull request or would you prefer this kind of >> supplementary code to go elsewhere? > > It would be better if you could integrate this in the R easyblock somehow (I > realize that may be a lot to ask for). The small amount of code that generates the *.eb could be in Python and hence that could go in the easyblock, but the majority of the code is R based on Stefano's script to generate the list of R packages their versions & the repos they came from and then re-order the list based on the dependencies. I would not know how to accomplish this without using R's own dependency handling mechanism and AFAIK their is no way to do this in Python. So the best I can imagine for now is a modified R easyblock that can execute an external R script to create a pkglist... and then generate an updated easyconfig. > We currently don't have a good location for this; I don't think we should > start adding 'random' scripts to the easyconfigs repo... The .../r/R/ dir in the EasyConfigs repo would be the first place where I would look for examples and a way to create new updated R easyconfigs, so in that sense it isn't a random location, but I agree that it is random in the sense that it isn't EasyBuild style Python code and R specific. Preferably we would not need a kludge like this :o > Maybe go with https://github.com/fgeorgatos/easybuild.experimental for now to > share it with others? That looks like a good place to start, so we can have a first version of an pure R easyconfig generator by the end of this week... > We should look into a contrib directory in the main EasyBuild repo > (https://github.com/hpcugent/easybuild), where people can get stuff included > without too much reviewing or very strict requirements, we already have a > couple of PRs for stuff like that (see > https://github.com/hpcugent/easybuild-framework/pull/1190, > https://github.com/hpcugent/easybuild-framework/pull/1180). Check, let me know when there is a contrib dir in place. Cheers, Pi > regards, > > Kenneth ------------------------------------------------------------- phone: +31 6 143 66 783 e-mail: [email protected] skype: pieter.online -------------------------------------------------------------
