> I would not know how to accomplish this without using R's own dependency > handling mechanism and AFAIK their is no way to do this in Python.
Could this be done with rpy? Niek On Wed, Jun 24, 2015 at 12:53 PM, Pieter Neerincx <[email protected]> wrote: > Hi Kenneth, > > On Jun 24, 2015, at 11:10 AM, Kenneth Hoste <[email protected]> wrote: > >> Hi Pieter, >> >> On 24/06/15 10:58, Pieter Neerincx wrote: >>> Hi, >>> >>> On Jun 24, 2015, at 7:34 AM, Kenneth Hoste <[email protected]> wrote: >>> >>>> Hi Riccardo, >>>> >>>> On 19/06/15 10:32, Riccardo Murri wrote: >>>>> Hi Ken, all, >>>>> >>>>> (Kenneth Hoste, Thu, Jun 18, 2015 at 04:32:27PM +0200:) >>>>>> We deliberately do not enable 'dependency=True', to ensure we can >>>>>> reproduce >>>>>> the exact same installation later; if you let R automagically resolve >>>>>> dependencies and let it pick the most recent version at that time for the >>>>>> dependencies, you can't do an exact reproduced install. >>>>> Could this be worked around by having EB save the list of installed R >>>>> packages + versions (like `pip freeze`) upon first install, then re-use >>>>> the saved list if available. >>>>> >>>>> Something along these lines (pseudo-code): >>>>> >>>>> package_list_file = easyconfig_file_name.replace('.eb', >>>>> '.pkglist.csv') >>>>> if not exists(package_list_file): >>>>> # 1. install packages by name, let R choose version >>>>> # 2. dump list of installed packages to file: (pkgname, >>>>> version) >>>>> else: >>>>> # 1. read in the pkglist.csv file >>>>> # 2. use R's install_version() to install the exact same pkg >>>>> and version >>>> I like this idea... >>> +1 >>> >>>> We could have an easyconfig parameter like 'auto_deps = True', and when >>>> that is set, EasyBuild lets R resolve dependencies by itself. >>>> >>>> Once the full list of easyconfigs has been installed, EasyBuild could then >>>> see what got installed, and which versions, and adjust the list of >>>> extensions included in the easyconfig file. The tricky bit may be the >>>> order, but there's probably a solution for that (see Stefano's R script). >>>> >>>> Now, who's up for implementing this? :-) >>> Yesterday I started working on an R-script that generates an *.eb from an >>> existing R installation. This is based on Stefano's code (Many thanks for >>> sharing that!) and adds some commandline switches, help and code to figure >>> out whether a package came from a CRAN repo or from BioConductor. Hence >>> this is slightly different compared to the above as it generates a complete >>> *.eb from pure R, but I can easily add an option to only generate a pkglist >>> file, which may then be used by an existing *.eb. >>> >>> Is it Ok if I add this with a README to the ..../r/R/ dir in the >>> EasyConfigs repo and create a pull request or would you prefer this kind of >>> supplementary code to go elsewhere? >> >> It would be better if you could integrate this in the R easyblock somehow (I >> realize that may be a lot to ask for). > > The small amount of code that generates the *.eb could be in Python and hence > that could go in the easyblock, but the majority of the code is R based on > Stefano's script to generate the list of R packages their versions & the > repos they came from and then re-order the list based on the dependencies. I > would not know how to accomplish this without using R's own dependency > handling mechanism and AFAIK their is no way to do this in Python. > > So the best I can imagine for now is a modified R easyblock that can execute > an external R script to create a pkglist... and then generate an updated > easyconfig. > >> We currently don't have a good location for this; I don't think we should >> start adding 'random' scripts to the easyconfigs repo... > > The .../r/R/ dir in the EasyConfigs repo would be the first place where I > would look for examples and a way to create new updated R easyconfigs, so in > that sense it isn't a random location, but I agree that it is random in the > sense that it isn't EasyBuild style Python code and R specific. Preferably we > would not need a kludge like this :o > >> Maybe go with https://github.com/fgeorgatos/easybuild.experimental for now >> to share it with others? > > That looks like a good place to start, so we can have a first version of an > pure R easyconfig generator by the end of this week... > >> We should look into a contrib directory in the main EasyBuild repo >> (https://github.com/hpcugent/easybuild), where people can get stuff included >> without too much reviewing or very strict requirements, we already have a >> couple of PRs for stuff like that (see >> https://github.com/hpcugent/easybuild-framework/pull/1190, >> https://github.com/hpcugent/easybuild-framework/pull/1180). > > Check, let me know when there is a contrib dir in place. > > Cheers, > > Pi > >> regards, >> >> Kenneth > > ------------------------------------------------------------- > phone: +31 6 143 66 783 > e-mail: [email protected] > skype: pieter.online > ------------------------------------------------------------- >
