Hi Kenneth
We do care about the version of R packages we have in the sense that we
prefer to have the latest version. That's why we run update.packages()
once in a while to keep them updated. I think the benefit of having the
latest version of packages outweighs any potential compatibility issues
between package versions ( I assume that's the reason to keep older
versions of packages around). Even when installing a new version it will
still use the same compiler tool chain. I think installing specific
versions of a packages is mostly user specific (i.e. particular user
needs specific version for some reason) and in that case it would make
more sense for the user to install that version in his personal
R_LIBS_USER directory. There are R functions available (e.g.
install_version(...) that will install a specific versions easily).
I am familiar with using EasyBuild to install packages (although Jack is
in charge of that here) but I'm not familiar with installing R and its
packages through EasyBuild, but from Niels' email it seems it's not
trivial wrt all the package dependencies (although there might be an
easy way of doing that he and I are not aware of). I'm not sure how it
works when packages need to be updated in that case. Will it be a
completely different R installation?
Thanks,
Maikel
------------------------
Marinus Pennings
[email protected]
High Performance Research Computing
Texas A&M University
Subject: Re: [easybuild] Easier way of installing R packages
Date: Thu, 18 Jun 2015 19:48:11 +0200
From: Kenneth Hoste <[email protected]>
Reply-To: [email protected]
To: [email protected]
Hi Maikel,
On 18/06/15 19:03, Marinus Pennings wrote:
Hallo Niek,
as Jack mentioned, we keep all our packages in a separate directory
and set the R_LIBS_USER env variable to that directory. For installing
there are actually 3 components
1) a module file
we have a different module (we name it R_tamu), the only thing it does
is to load the matching R module, set R_LIBS_USER, as well as setting
some other env variables (e.g. RPACKAGEROOT that you saw in the the R
install script).
2) a wrapper bash script (we named it install-R-packages.sh), Jack
attached that one. Technically It's not really necessary, but it's
easier to use that to invoke Rscipt manually every time (for lazy
people)
3) the R script (tamu_install.R) , Jack attached that one as well. The
script read the package list and then for every package it checks if
it's already installed, and if not installs it (kind of annoying with
R, but install.packages(...) always will install otherwise), it will
also automatically install dependencies.
Now, when our users request new packages, I just add it to the list
and run "install-R-packages.sh" and all the new packages will be
installed in the correct R_LIBS_USER directory. When we build a new
version of R, the only thing we need to do is create a matching R_tamu
module and then run "install-R-packages.sh". It will install all
packages for the new R version in it's own R_LIBS_USER dir.
There are some caveats, when a packages has extra configure args. If
you look at "tamu_install.R" you see we do Rmpi first. Thats because
it needs specific configure arguments (if you use a different MPI tool
chain it might be easier). Actually Rmpi has some problems anyway for
our toolchain because of some missing libraries so I downloaded the
source, fixed the configure script, and created the tarball again
(that's why in the "tamu_install.R you see it will use a local file
instead of downloading it).
In general, right now our script cannot handle packages with extra
configure args very well (I'm working on a package list that also has
a column for configure args). However, the only packages we have seen
so far that needs extra configure args is Rmpi (and that's for a part
because we use IntelMPI)
Hope that clarifies it a bit,
Thanks for sharing this.
I'd like to see what we can learn from this to improve the way we
currently tackle R libraries using EasyBuild.
Am I right in understanding that you care little about the version of
the R libraries you install? Running install-R-packages.sh will just
install the latest & greatest version of missing libraries + their most
recent dependencies?
We have good support in EasyBuild for specifying custom configure
arguments, so I hope we can maybe convergence to a
best-of-both-solutions approach, where you can do what you want using
EasyBuild as well.
If that means adding support for allowing to specify which R libraries
to install versionless and let R automagically install the required
dependencies, I'm up for that if someone dives in to get it supported (I
just don't feel that should be the norm, but I can imagine some people
feel otherwise).
regards,
Kenneth
