Hi Maikel,
On 18/06/15 19:03, Marinus Pennings wrote:
Hallo Niek,
as Jack mentioned, we keep all our packages in a separate directory
and set the R_LIBS_USER env variable to that directory. For installing
there are actually 3 components
1) a module file
we have a different module (we name it R_tamu), the only thing it does
is to load the matching R module, set R_LIBS_USER, as well as setting
some other env variables (e.g. RPACKAGEROOT that you saw in the the R
install script).
2) a wrapper bash script (we named it install-R-packages.sh), Jack
attached that one. Technically It's not really necessary, but it's
easier to use that to invoke Rscipt manually every time (for lazy people)
3) the R script (tamu_install.R) , Jack attached that one as well. The
script read the package list and then for every package it checks if
it's already installed, and if not installs it (kind of annoying with
R, but install.packages(...) always will install otherwise), it will
also automatically install dependencies.
Now, when our users request new packages, I just add it to the list
and run "install-R-packages.sh" and all the new packages will be
installed in the correct R_LIBS_USER directory. When we build a new
version of R, the only thing we need to do is create a matching R_tamu
module and then run "install-R-packages.sh". It will install all
packages for the new R version in it's own R_LIBS_USER dir.
There are some caveats, when a packages has extra configure args. If
you look at "tamu_install.R" you see we do Rmpi first. Thats because
it needs specific configure arguments (if you use a different MPI tool
chain it might be easier). Actually Rmpi has some problems anyway for
our toolchain because of some missing libraries so I downloaded the
source, fixed the configure script, and created the tarball again
(that's why in the "tamu_install.R you see it will use a local file
instead of downloading it).
In general, right now our script cannot handle packages with extra
configure args very well (I'm working on a package list that also has
a column for configure args). However, the only packages we have seen
so far that needs extra configure args is Rmpi (and that's for a part
because we use IntelMPI)
Hope that clarifies it a bit,
Thanks for sharing this.
I'd like to see what we can learn from this to improve the way we
currently tackle R libraries using EasyBuild.
Am I right in understanding that you care little about the version of
the R libraries you install? Running install-R-packages.sh will just
install the latest & greatest version of missing libraries + their most
recent dependencies?
We have good support in EasyBuild for specifying custom configure
arguments, so I hope we can maybe convergence to a
best-of-both-solutions approach, where you can do what you want using
EasyBuild as well.
If that means adding support for allowing to specify which R libraries
to install versionless and let R automagically install the required
dependencies, I'm up for that if someone dives in to get it supported (I
just don't feel that should be the norm, but I can imagine some people
feel otherwise).
regards,
Kenneth
