Hi everyone,
I begun to use Rserve <http://rforge.net/Rserve/> (multi-threaded),
instead of JRIengine (singleton / mono-threaded), in my personal
experimental branch
<https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/>
and really think we could replace JRIengine (which does not take
advantage of parallel tasks processing, as you know), by the former.
I released a version using Rserve (/restricted to the "Baseline
Correction Module"/). If you want to give it a try, here it is:
https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/target/MZmine-2.11-EEE-release-20141128.zip
<https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/target/MZmine-2.11-EEE-release-20141126.zip>
Main points:
-----------
* Requirements:*
* ------------
*
No changes in "startMZmine" script* (we just need to locate "R_HOME",
as usual).
The only requirement is to have the "Rserve" R package installed. Then
it is ran through a simple R command, looking like:
library(Rserve);Rserve(debug=TRUE/FALSE,
args="--RS-enable-control")
# Where '--RS-enable-control' is used and mandatory to enable sending
"SIGKILL/SIGTERM" to the Rserve instances
# (when we need to abort a blocking 'eval' being performed)
(Note: we already had to install "rJava" for using JRI). So this does
not stand for a big additional constraint for the user.
=> The Rserve runnable (server app.) comes with the said R package
*Note:* I believe we could even simplify the*"startMZmine" workflow*,
by first trying to detect the R location automatically.
(the environment variables to be assigned in the script would
be used only as a fallback in case the detection failed)
That would more or less look like (See:RUtilities.java
<https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/src/main/java/net/sf/mzmine/util/RUtilities.java>)
:
public static String getRexecutablePath() {
String osname = System.getProperty("os.name");
if (osname != null && osname.length() >= 7 &&
osname.substring(0,7).equals("Windows")) {
LOG.log(Level.INFO, "Windows: query registry
to find where R is installed ...");
String installPath = null;
try {
Process rp =
Runtime.getRuntime().exec("reg query HKLM\\Software\\R-core\\R");
StreamHog regHog = new
StreamHog(rp.getInputStream(), true);
rp.waitFor();
regHog.join();
installPath =
regHog.getInstallPath();
} catch (Exception rge) {
LOG.log(Level.SEVERE, "ERROR: unable
to run REG to find the location of R: "+rge);
return null;
}
if (installPath == null) {
LOG.log(Level.SEVERE, "ERROR: canot
find path to R. Make sure reg is available and R was installed with registry
settings.");
return null;
}
return installPath + "\\bin\\R.exe";
}
File f = new
File("/Library/Frameworks/R.framework/Resources/bin/R");
if (f.exists()) return f.getPath();
f = new File("/usr/local/lib/R/bin/R");
if (f.exists()) return f.getPath();
f = new File("/usr/lib/R/bin/R");
if (f.exists()) return f.getPath();
f = new File("/sw/bin/R");
if (f.exists()) return f.getPath();
f = new File("/usr/common/bin/R");
if (f.exists()) return f.getPath();
f = new File("/opt/bin/R");
if (f.exists()) return f.getPath();
return null;
}
* The way it works:
----------------
Rserve is a server (implements a communication protocol with R) and, to
keep it simple, it should be run on localhost and default
port 6311 (this is something that should probably be mentioned to the
user, for*security and eventually firewall configuration reasons*).
Each time we create a new "RConnection", the main instance of Rserve
starts a new child process.
- We can store the PID of this instance for later termination.
- We can run asynchronously as many instances as we want (as long as we
take care to close/terminate the ones that are no longer used).
=> I already implemented a basic wrapper (that should be enhanced) for
those operations:
https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/src/main/java/net/sf/mzmine/util/RSession.java
(Simplified version of the class in attachment).
=> The code for starting Rserve main server instance is located in
"RUtilities.java" for now:
https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/src/main/java/net/sf/mzmine/util/RUtilities.java
=>*USAGE* (a sample use case can be found in theBaselineCorrectionTask.java
<https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/src/main/java/net/sf/mzmine/modules/rawdatamethods/filtering/baselinecorrection/BaselineCorrectionTask.java>):
Basically, evaluating one or more R commands is done as follows:
- Create and open a new session:
// (This will automatically start Rserve, if
not already running, and open a new slave instance for this task)
String[] reqPackages = new String[] { "ptw" };
RSession rSession = new RSession(reqPackages);
rSession.open();
- Check the additional packages necessary to perform
the commands for the given session:
// Returns the first failing required package
name or null if successful
String missingPackage =
rSession.loadRequiredPackages();
- Do some R evaluations, such as:
// Set chromatogram.
rSession.assignDoubleArray("chromatogram",
chromatogram);
// Calculate baseline.
rSession.eval("baseline <- asysm(chromatogram," + smoothing +
"," + asymmetry + ")");
baseline =
rSession.collectDoubleArray("baseline");
- Release the session (closes the socket connection for
the session and ends the related Rserve slave process):
rSession.close(false);
*Note:* We should also check, at some point,
if no more tasks are requiring Rserve, and shutdown the server.
(it will be restarted the next time we
use a MZmine feature requiring R anyway...)
* Performance considerations:
--------------------------
Source:http://www.sciencedomain.org/download.php?f=Satman4152014BJMCS10902_1.pdf&aid=4838&type=a
The performances where good enough for what I had to do with the
BaselineCorretors. So, at very first sight, this should fit the other needs
(anyway, the other very few multi-threaded alternatives I gave a test where not
exploitable in term of evaluation speed).
May be we should/will have to consider switching between JRIengine and Rserve
for some particular features).*I hope not*, but in any case,
this is something which seems doable: as far as I tested, there is absolutely
no incompatibilities between Rserve and JRI.
To make your own opinion on all this, the BaselineCorretorModule (again) allows
to choose the R engine among*RCaller (Online), JRIengine and Rserve*.
Just run the MZmine version linked on top of this email, and go to "Raw data methods >
Filtering > Baseline Correction"
Fill free to ask if something is unclear or if you have any further
questionings.
Looking forward to your constructive comments and thoughts...
I'll be glad to help, as much as possible, with migrating to Rserve (if
it turns out that this solution really is viable).
Cheers
Gauthier
-------- Forwarded Message --------
Subject: Re: [Mzmine-devel] Baseline correction
Date: Tue, 2 Dec 2014 05:15:41 +0000
From: Tomas Pluskal <plus...@oist.jp>
To: Gauthier Boaglio <gauthier.boag...@gmail.com>
Hi Gauthier,
Now that we released version 2.12, I think it is a good time to consider
making the switch from JRI to RServe.
I think it is a good idea - I don't really like the current JRI interface.
It would be great if you could send a message to the devel list, where
you can summarize what you found about RServe. Especially, we would like
to know
1) is the initial setup and configuration going to be easier than the
current one (is it necessary to edit the startMZmine script in order to
use RServe?)
2) how does the actual code differ between RServe and JRI?
Thanks a lot!
Cheers,
Tomas
===============================================
Tomas Pluskal
G0 Cell Unit, Okinawa Institute of Science and Technology Graduate
University
1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan
WWW: https://groups.oist.jp/g0
TEL: +81-98-966-8684
Fax: +81-98-966-2890
--
Gauthier BOAGLIO
CEFE - UMR 5175
1919 route de Mende
F-34293 Montpellier cedex 5
Tel: +33/0 4 67 61 32 15
Fax: +33/0 4 67 61 33 36
email: gauthier.boag...@cefe.cnrs.fr
www:
http://www.cefe.cnrs.fr/en/evolutionary-ecology-and-epidemiology/gauthier-boaglio
http://www.evolepid.org/people.php?name=boaglio
package net.sf.mzmine.util;
import org.rosuda.JRI.Rengine;
import org.rosuda.REngine.REXPMismatchException;
import org.rosuda.REngine.REngineException;
import org.rosuda.REngine.Rserve.RConnection;
import org.rosuda.REngine.Rserve.RserveException;
import java.util.logging.Logger;
public class RSession {
private RengineType rEngineType;
private Object rEngine = null;
private String[] reqPackages;
private int rServePid = -1;
public RSession(RengineType type, String[] reqPackages) {
this.rEngineType = type;
this.reqPackages = reqPackages;
}
private void getRengineInstance() {
LOG.info("getRengineInstance() for: " + this.rEngineType.toString());
try {
if (this.rEngine == null) {
if (!RUtilities.checkLocalRserve())
throw new IllegalStateException(
"Could not start Rserve. Please check if R and Rserve are installed and path to the "
+ "libraries is set properly in the startMZmine script.");
// Keep an opened instance and store the related pid
RConnection rconn = new RConnection();
this.rServePid = rconn.eval("Sys.getpid()").asInteger();
this.rEngine = rconn;
LOG.info("Rserve: started instance with pid '" + this.rServePid + "'.");
}
}
catch (Throwable t) {
t.printStackTrace();
throw new IllegalStateException(
"This feature requires R but it couldn't be loaded (" + t.getMessage() + ')');
}
}
public void loadPackage(String packageName) {
String loadCode = "library(" + packageName + ", logical.return = TRUE)";
String errorMsg = "The \"" + packageName + "\" R package couldn't be loaded - is it installed in R?";
int loaded = 0;
try {
loaded = ((RConnection)this.rEngine).eval(loadCode).asInteger(); //("library(" + packageName + ")");
} catch (RserveException | REXPMismatchException e) {
// Remain silent if eval KO ("server down").
loaded = Integer.MIN_VALUE;
}
// Throw loading failure only if eval OK ("server down" case will be handled soon enough).
if (loaded == 0)
throw new IllegalStateException(errorMsg);
}
public String loadRequiredPackages() {
String reqPackage = null;
try {
for (int i=0; i < this.reqPackages.length; ++i) {
reqPackage = this.reqPackages[i];
this.loadPackage(this.reqPackages[i]);
LOG.info("Loaded package: " + reqPackage + "'.");
}
return null;
} catch (Exception e) {
LOG.info("Failed loading package: '" + reqPackage + "'.");
return reqPackage;
}
}
// TODO: templatize: assignDoubleArray<T>(String objName, T obj)
public void assignDoubleArray(String objName, double[] dArray) {
try {
((RConnection)this.rEngine).assign(objName, dArray);
} catch (REngineException e) {
//e.printStackTrace();
throw new IllegalStateException("Rserve error: couldn't assign R object '" + objName + "'.");
}
}
public void eval(String rCode) {
try {
((RConnection)this.rEngine).eval(rCode);
} catch (RserveException e) {
//e.printStackTrace();
throw new IllegalStateException("Rserve error: couldn't eval R code '" + rCode + "'.");
}
}
// TODO: templatize: T collectDoubleArray(String objName)
public double[] collectDoubleArray(String objName) {
try {
return ((RConnection)this.rEngine).eval(objName).asDoubles();
} catch (RserveException | REXPMismatchException e) {
//e.printStackTrace();
throw new IllegalStateException("Rserve error: couldn't collect R object '" + objName + "'.");
}
}
public void open() {
// Load engine
getRengineInstance();
}
/**
* This can be necessary to call 'close()' from a different thread
* than the one which called 'open()', sometimes, with Rserve.
* @param userCanceled
*/
public void close(boolean userCanceled) {
try {
// Terminate.
final RConnection c2 = new RConnection();
// SIGTERM might not be understood everywhere: so using SIGKILL signal, as well.
c2.eval("tools::pskill("+ this.rServePid + ")"); //win
c2.eval("tools::pskill("+ this.rServePid + ", tools::SIGKILL)"); //nix
c2.close();
LOG.info("Rserve: terminated instance with pid '" + this.rServePid + "'.");
} catch (RserveException e) {
// Adapt message accordingly to if the termination was provoked by user or
// unexpected...
if (userCanceled) {
//e.printStackTrace();
throw new IllegalStateException("Rserve error: couldn't terminate instance with pid '" + this.rServePid + "'.");
} else {
throw new IllegalStateException("Rserve error: something when wrong with instance of pid '" + this.rServePid + "'.");
}
}
}
}
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Mzmine-devel mailing list
Mzmine-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mzmine-devel
