Hallo Niek,
as Jack mentioned, we keep all our packages in a separate directory
and set the R_LIBS_USER env variable to that directory. For installing
there are actually 3 components
1) a module file
we have a different module (we name it R_tamu), the only thing it does
is to load the matching R module, set R_LIBS_USER, as well as setting
some other env variables (e.g. RPACKAGEROOT that you saw in the the R
install script).
2) a wrapper bash script (we named it install-R-packages.sh), Jack
attached that one. Technically It's not really necessary, but it's
easier to use that to invoke Rscipt manually every time (for lazy people)
3) the R script (tamu_install.R) , Jack attached that one as well. The
script read the package list and then for every package it checks if
it's already installed, and if not installs it (kind of annoying with R,
but install.packages(...) always will install otherwise), it will also
automatically install dependencies.
Now, when our users request new packages, I just add it to the list and
run "install-R-packages.sh" and all the new packages will be installed
in the correct R_LIBS_USER directory. When we build a new version of R,
the only thing we need to do is create a matching R_tamu module and then
run "install-R-packages.sh". It will install all packages for the new R
version in it's own R_LIBS_USER dir.
There are some caveats, when a packages has extra configure args. If you
look at "tamu_install.R" you see we do Rmpi first. Thats because it
needs specific configure arguments (if you use a different MPI tool
chain it might be easier). Actually Rmpi has some problems anyway for
our toolchain because of some missing libraries so I downloaded the
source, fixed the configure script, and created the tarball again
(that's why in the "tamu_install.R you see it will use a local file
instead of downloading it).
In general, right now our script cannot handle packages with extra
configure args very well (I'm working on a package list that also has a
column for configure args). However, the only packages we have seen so
far that needs extra configure args is Rmpi (and that's for a part
because we use IntelMPI)
Hope that clarifies it a bit,
Groeten,
Maikel.
------------------------
Marinus Pennings
[email protected]
High Performance Research Computing
Texas A&M University
On 06/18/2015 09:38 AM, Jack Perdue wrote:
Howdy Niek,
We have a non-Easybuild module tree. The attachment
from before lives in /software/tamusc/modulefiles/R.
It loads the EB R and then sets the needed env vars
for building extensions.
Note that I don't maintain the R extensions so don't
have details. Here are the scripts written by my colleague
Maikel Pennings for such things. You'd have to ask him to
explain them.
Jack Perdue
Lead Systems Administrator
High Performance Research Computing
TAMU Division of Research
[email protected] http://sc.tamu.edu
SC Helpdesk: [email protected]
On 06/18/2015 09:27 AM, Niek de Klein wrote:
Hi Jack,
Thank you for your answer. I don't understand the attached file. How
do you use that to install extensions, and how do you keep it so that
if you want to migrate to another cluster you can install R with the
exact same R libraries installed? Do you keep a separate file with all
the R packages you installed extra?
Thanks,
Niek
On Thu, Jun 18, 2015 at 4:20 PM, Jack Perdue <[email protected]> wrote:
Howdy Niek,
FWIW, here is our latest build of R (with lots of
dependencies added):
http://www.siliconslick.com/easybuild/ebfiles_repo_cleaned/ada/R/R-3.2.0-intel-2015B-default-mt.eb
For extensions, we have another (non-EB) module (attached)
for installing extensions to the above by hand.
Between the two, we've:
a) tried to make sure we aren't using system libs (e.g. TCL)
b) are able to satisfy user requests for misc. R extensions
(automagically)
Jack Perdue
Lead Systems Administrator
High Performance Research Computing
TAMU Division of Research
[email protected] http://sc.tamu.edu
SC Helpdesk: [email protected]
On 06/18/2015 08:57 AM, Niek de Klein wrote:
Hi mailinglist,
I'm probably missing something here and unnecessarily complicating
things. I am trying to get R installed with just a few packages we
use. One of the packages, ggplot2, has a lot of dependencies. So when
I try to install R with one of the existing EasyBuild files and have
ggplot2 as the dependency it will throw an error due to the
dependencies of ggplot2 not being installed, and I have to add the
dependencies to the .eb file. And then add the dependencies of the
dependencies to the .eb file, etc etc.
When installing libraries from within R you can simply give the option
"dependency=True" and it will automagically install all dependencies
for you. Is there an option like that when installing R and some R
libraries?
Thanks,
Niek
