Hi Gauthier,
I think another option to consider seriously is Renjin. Although Renjin in its
current development state does not really support installation of packages, we
could simply copy & paste the relevant functions from the existing R packages.
If that works, no configuration or platform-specific hacks would be necessary,
as Renjin is pure java.
Cheers,
Tomas
On Dec 3, 2014, at 12:44, Gauthier Boaglio
<gauthier.boag...@gmail.com<mailto:gauthier.boag...@gmail.com>> wrote:
I just went through "rSession" code again and a bit further, and I might have
spoken too fast.
Mainly classes "Rsession" and "RserverConf" seems to speak about things like
"windows multi-session emulation", playing with ports, etc. (Sounds like
exactly what I was trying to describe previously, isn't it?).
I'll have a new fresh eye on it tomorrow (that could avoid to reinvent the
wheel). 5AM over here...
Cheers
Gauthier
-------- Forwarded Message --------
Subject: Re: [Mzmine-devel] R usage in MZmine - What about replacing
JRIengine? - Not working well under Windows.
Date: Wed, 03 Dec 2014 04:29:34 +0100
From: Gauthier Boaglio
<gauthier.boag...@gmail.com><mailto:gauthier.boag...@gmail.com>
To:
mzmine-devel@lists.sourceforge.net<mailto:mzmine-devel@lists.sourceforge.net>
Hi Tomas,
Yes I have, and it really doesn't help that much, that is why I preferred
implementing my own minimalistic class. I might change my mind at some point,
but I don't believe adding this dependency corresponds to an actual need.
Otherwise, I have some news about Rserve (quite bad ones). It relies on unix
features and is not able, under windows, to 'fork' instances like unix systems
do. This is quite deceiving. But, I am implementing a workaround: windows
specific solution to mimic processes forking. We'll see if it is convincing
once I have it working.
Since we still can run several instances of Rserve main server under windows
(despite we cannot use the great UNIX features), the idea is:
Creating a pool of a finite number of warm instances of Rserve - 8 is a good
number I suppose, ready for use (and started at the application startup, or at
the very first time R is required). Those instances would have to be mapped on
dedicated/respective ports (one for each worker of the pool). Those port should
be chosen among "IANA Registered but marked as UNASSIGNED ports, if I'm
correct. As long as we don't need more workers than the default warm ones, we
manage the pool, reserving and releasing them on demand. If the amount of
concurrent R tasks required increases, we still can scale the size (number of
active/alive workers) of the pool. When no more R task is running, we can then
scale back to normal, and put the default number of warm workers to sleep.
I am a bit tired, so my apologies if my English is getting odd, but I hope that
what I am telling here is clear enough.
So, to make it short: Rserve is not ideal (well, it is, but windows isn't). At
least, I found nothing better anywhere. A workaround can be concidered, which I
am going to implement for my own needs anyway, so you'll have the occasion to
test it quite soon.
I'll check again if "Rsession" doesn't provides the features I am working on
already, but I'm pretty sure it doesn't.
So, what do you say? To my opinion, with a good handy wrapper, all this could
be viable...
If you have any idea or suggestion about all this. Please tell me such that I
can integrate into my reflections.
Cheers
Gauthier
On 12/03/2014 01:54 AM, Tomas Pluskal wrote:
Hi Gauthier,
Just wondering - have you checked the Rsession project?
(https://code.google.com/p/rsession/)
I don't have any personal experience with it, but it seems to contain the code
to run the Rserve from java.
Cheers,
Tomas
On Dec 3, 2014, at 3:43, Gauthier Boaglio
<gauthier.boag...@gmail.com<mailto:gauthier.boag...@gmail.com>> wrote:
Hi,
My apologies to those who tried to test (hope none of you actually) the release
linked above under Windows. This is working under Linux, but not (completely)
under Windows.
Sorry about that. I try to fix it ASAP and keep you informed.
Cheers
Gauthier
On 12/02/2014 11:47 AM, Gauthier Boaglio wrote:
Text formatted message (for non HTML mailers)
-------- Forwarded Message --------
Subject: R usage in MZmine - What about replacing JRIengine to get
multi-threading support?
Date: Tue, 02 Dec 2014 11:36:55 +0100
From: Gauthier Boaglio
<gauthier.boag...@gmail.com><mailto:gauthier.boag...@gmail.com>
To:
mzmine-devel@lists.sourceforge.net<mailto:mzmine-devel@lists.sourceforge.net>
Hi everyone,
I begun to use Rserve (multi-threaded), instead of JRIengine (singleton /
mono-threaded), in my personal experimental branch and really think we could
replace JRIengine (which does not take advantage of parallel tasks processing,
as you know), by the former.
I released a version using Rserve (restricted to the "Baseline Correction
Module"). If you want to give it a try, here it is:
https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/target/MZmine-2.11-EEE-release-20141128.zip
Main points:
-----------
* Requirements:
------------
No changes in "startMZmine" script (we just need to locate "R_HOME", as
usual).
The only requirement is to have the "Rserve" R package installed. Then it
is ran through a simple R command, looking like:
library(Rserve);Rserve(debug=TRUE/FALSE, args="--RS-enable-control")
# Where '--RS-enable-control' is used and mandatory to enable sending
"SIGKILL/SIGTERM" to the Rserve instances
# (when we need to abort a blocking 'eval' being performed)
(Note: we already had to install "rJava" for using JRI). So this does not
stand for a big additional constraint for the user.
=> The Rserve runnable (server app.) comes with the said R package
Note: I believe we could even simplify the "startMZmine" workflow, by first
trying to detect the R location automatically.
(the environment variables to be assigned in the script would be used
only as a fallback in case the detection failed)
That would more or less look like (See: RUtilities.java) :
public static String getRexecutablePath() {
String osname = System.getProperty("os.name");
if (osname != null && osname.length() >= 7 &&
osname.substring(0,7).equals("Windows")) {
LOG.log(Level.INFO, "Windows: query registry to find where
R is installed ...");
String installPath = null;
try {
Process rp = Runtime.getRuntime().exec("reg query
HKLM\\Software\\R-core\\R");
StreamHog regHog = new StreamHog(rp.getInputStream(),
true);
rp.waitFor();
regHog.join();
installPath = regHog.getInstallPath();
} catch (Exception rge) {
LOG.log(Level.SEVERE, "ERROR: unable to run REG to find
the location of R: "+rge);
return null;
}
if (installPath == null) {
LOG.log(Level.SEVERE, "ERROR: canot find path to R.
Make sure reg is available and R was installed with registry settings.");
return null;
}
return installPath + "\\bin\\R.exe<smb://bin//R.exe>";
}
File f = new
File("/Library/Frameworks/R.framework/Resources/bin/R");
if (f.exists()) return f.getPath();
f = new File("/usr/local/lib/R/bin/R");
if (f.exists()) return f.getPath();
f = new File("/usr/lib/R/bin/R");
if (f.exists()) return f.getPath();
f = new File("/sw/bin/R");
if (f.exists()) return f.getPath();
f = new File("/usr/common/bin/R");
if (f.exists()) return f.getPath();
f = new File("/opt/bin/R");
if (f.exists()) return f.getPath();
return null;
}
* The way it works:
----------------
Rserve is a server (implements a communication protocol with R) and, to
keep it simple, it should be run on localhost and default
port 6311 (this is something that should probably be mentioned to the user,
for security and eventually firewall configuration reasons).
Each time we create a new "RConnection", the main instance of Rserve starts
a new child process.
- We can store the PID of this instance for later termination.
- We can run asynchronously as many instances as we want (as long as we
take care to close/terminate the ones that are no longer used).
=> I already implemented a basic wrapper (that should be enhanced) for
those operations:
https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/src/main/java/net/sf/mzmine/util/RSession.java
(Simplified version of the class in attachment).
=> The code for starting Rserve main server instance is located in
"RUtilities.java" for now:
https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/src/main/java/net/sf/mzmine/util/RUtilities.java
=> USAGE (a sample use case can be found in the
BaselineCorrectionTask.java):
Basically, evaluating one or more R commands is done as follows:
- Create and open a new session:
// (This will automatically start Rserve, if not already
running, and open a new slave instance for this task)
String[] reqPackages = new String[] { "ptw" };
RSession rSession = new RSession(reqPackages);
rSession.open();
- Check the additional packages necessary to perform the commands
for the given session:
// Returns the first failing required package name or null if
successful
String missingPackage = rSession.loadRequiredPackages();
- Do some R evaluations, such as:
// Set chromatogram.
rSession.assignDoubleArray("chromatogram", chromatogram);
// Calculate baseline.
rSession.eval("baseline <- asysm(chromatogram," + smoothing +
"," + asymmetry + ")");
baseline = rSession.collectDoubleArray("baseline");
- Release the session (closes the socket connection for the session
and ends the related Rserve slave process):
rSession.close(false);
Note: We should also check, at some point, if no more tasks are
requiring Rserve, and shutdown the server.
(it will be restarted the next time we use a MZmine feature
requiring R anyway...)
* Performance considerations:
--------------------------
Source:
http://www.sciencedomain.org/download.php?f=Satman4152014BJMCS10902_1.pdf&aid=4838&type=a
The performances where good enough for what I had to do with the
BaselineCorretors. So, at very first sight, this should fit the other needs
(anyway, the other very few multi-threaded alternatives I gave a test where not
exploitable in term of evaluation speed).
May be we should/will have to consider switching between JRIengine and Rserve
for some particular features). I hope not, but in any case,
this is something which seems doable: as far as I tested, there is absolutely
no incompatibilities between Rserve and JRI.
To make your own opinion on all this, the BaselineCorretorModule (again) allows
to choose the R engine among RCaller (Online), JRIengine and Rserve.
Just run the MZmine version linked on top of this email, and go to "Raw data
methods > Filtering > Baseline Correction"
Fill free to ask if something is unclear or if you have any further
questionings.
Looking forward to your constructive comments and thoughts...
I'll be glad to help, as much as possible, with migrating to Rserve (if it
turns out that this solution really is viable).
Cheers
Gauthier
-------- Forwarded Message --------
Subject: Re: [Mzmine-devel] Baseline correction
Date: Tue, 2 Dec 2014 05:15:41 +0000
From: Tomas Pluskal <plus...@oist.jp><mailto:plus...@oist.jp>
To: Gauthier Boaglio
<gauthier.boag...@gmail.com><mailto:gauthier.boag...@gmail.com>
Hi Gauthier,
Now that we released version 2.12, I think it is a good time to consider making
the switch from JRI to RServe.
I think it is a good idea - I don't really like the current JRI interface.
It would be great if you could send a message to the devel list, where you can
summarize what you found about RServe. Especially, we would like to know
1) is the initial setup and configuration going to be easier than the current
one (is it necessary to edit the startMZmine script in order to use RServe?)
2) how does the actual code differ between RServe and JRI?
Thanks a lot!
Cheers,
Tomas
===============================================
Tomas Pluskal
G0 Cell Unit, Okinawa Institute of Science and Technology Graduate University
1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan
WWW: https://groups.oist.jp/g0
TEL: +81-98-966-8684
Fax: +81-98-966-2890
--
Gauthier BOAGLIO
CEFE - UMR 5175
1919 route de Mende
F-34293 Montpellier cedex 5
Tel: +33/0 4 67 61 32 15
Fax: +33/0 4 67 61 33 36
email: gauthier.boag...@cefe.cnrs.fr<mailto:gauthier.boag...@cefe.cnrs.fr>
www:
http://www.cefe.cnrs.fr/en/evolutionary-ecology-and-epidemiology/gauthier-boaglio
http://www.evolepid.org/people.php?name=boaglio
-------- Forwarded Message --------
Subject: R usage in MZmine - What about replacing JRIengine to get
multi-threading support?
Date: Tue, 02 Dec 2014 11:36:55 +0100
From: Gauthier Boaglio
<gauthier.boag...@gmail.com><mailto:gauthier.boag...@gmail.com>
To:
mzmine-devel@lists.sourceforge.net<mailto:mzmine-devel@lists.sourceforge.net>
Hi everyone,
I begun to use Rserve<http://rforge.net/Rserve/> (multi-threaded), instead of
JRIengine (singleton / mono-threaded), in my personal experimental
branch<https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/>
and really think we could replace JRIengine (which does not take advantage of
parallel tasks processing, as you know), by the former.
I released a version using Rserve (restricted to the "Baseline Correction
Module"). If you want to give it a try, here it is:
https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/target/MZmine-2.11-EEE-release-20141128.zip<https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/target/MZmine-2.11-EEE-release-20141126.zip>
Main points:
-----------
* Requirements:
------------
No changes in "startMZmine" script (we just need to locate "R_HOME", as
usual).
The only requirement is to have the "Rserve" R package installed. Then
it is ran through a simple R command, looking like:
library(Rserve);Rserve(debug=TRUE/FALSE,
args="--RS-enable-control")
# Where '--RS-enable-control' is used and mandatory to enable
sending "SIGKILL/SIGTERM" to the Rserve instances
# (when we need to abort a blocking 'eval' being performed)
(Note: we already had to install "rJava" for using JRI). So this does
not stand for a big additional constraint for the user.
=> The Rserve runnable (server app.) comes with the said R package
Note: I believe we could even simplify the "startMZmine" workflow, by
first trying to detect the R location automatically.
(the environment variables to be assigned in the script would
be used only as a fallback in case the detection failed)
That would more or less look like (See:
RUtilities.java<https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/src/main/java/net/sf/mzmine/util/RUtilities.java>)
:
public static String getRexecutablePath() {
String osname = System.getProperty("os.name");
if (osname != null && osname.length() >= 7 &&
osname.substring(0,7).equals("Windows")) {
LOG.log(Level.INFO, "Windows: query
registry to find where R is installed ...");
String installPath = null;
try {
Process rp =
Runtime.getRuntime().exec("reg query HKLM\\Software\\R-core\\R");
StreamHog regHog = new
StreamHog(rp.getInputStream(), true);
rp.waitFor();
regHog.join();
installPath =
regHog.getInstallPath();
} catch (Exception rge) {
LOG.log(Level.SEVERE, "ERROR:
unable to run REG to find the location of R: "+rge);
return null;
}
if (installPath == null) {
LOG.log(Level.SEVERE, "ERROR:
canot find path to R. Make sure reg is available and R was installed with
registry settings.");
return null;
}
return installPath +
"\\bin\\R.exe<smb://bin//R.exe>";
}
File f = new
File("/Library/Frameworks/R.framework/Resources/bin/R");
if (f.exists()) return f.getPath();
f = new File("/usr/local/lib/R/bin/R");
if (f.exists()) return f.getPath();
f = new File("/usr/lib/R/bin/R");
if (f.exists()) return f.getPath();
f = new File("/sw/bin/R");
if (f.exists()) return f.getPath();
f = new File("/usr/common/bin/R");
if (f.exists()) return f.getPath();
f = new File("/opt/bin/R");
if (f.exists()) return f.getPath();
return null;
}
* The way it works:
----------------
Rserve is a server (implements a communication protocol with R) and, to
keep it simple, it should be run on localhost and default
port 6311 (this is something that should probably be mentioned to the
user, for security and eventually firewall configuration reasons).
Each time we create a new "RConnection", the main instance of Rserve
starts a new child process.
- We can store the PID of this instance for later termination.
- We can run asynchronously as many instances as we want (as long as we
take care to close/terminate the ones that are no longer used).
=> I already implemented a basic wrapper (that should be enhanced) for
those operations:
https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/src/main/java/net/sf/mzmine/util/RSession.java
(Simplified version of the class in attachment).
=> The code for starting Rserve main server instance is located in
"RUtilities.java" for now:
https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/src/main/java/net/sf/mzmine/util/RUtilities.java
=> USAGE (a sample use case can be found in the
BaselineCorrectionTask.java<https://sourceforge.net/p/mzmine/code/HEAD/tree/branches/gboaglio-experimental/src/main/java/net/sf/mzmine/modules/rawdatamethods/filtering/baselinecorrection/BaselineCorrectionTask.java>):
Basically, evaluating one or more R commands is done as follows:
- Create and open a new session:
// (This will automatically start Rserve, if
not already running, and open a new slave instance for this task)
String[] reqPackages = new String[] { "ptw" };
RSession rSession = new RSession(reqPackages);
rSession.open();
- Check the additional packages necessary to perform
the commands for the given session:
// Returns the first failing required package
name or null if successful
String missingPackage =
rSession.loadRequiredPackages();
- Do some R evaluations, such as:
// Set chromatogram.
rSession.assignDoubleArray("chromatogram",
chromatogram);
// Calculate baseline.
rSession.eval("baseline <- asysm(chromatogram,"
+ smoothing + "," + asymmetry + ")");
baseline =
rSession.collectDoubleArray("baseline");
- Release the session (closes the socket connection for
the session and ends the related Rserve slave process):
rSession.close(false);
Note: We should also check, at some point, if
no more tasks are requiring Rserve, and shutdown the server.
(it will be restarted the next time we
use a MZmine feature requiring R anyway...)
* Performance considerations:
--------------------------
<Mail Attachment.png>
Source:
http://www.sciencedomain.org/download.php?f=Satman4152014BJMCS10902_1.pdf&aid=4838&type=a
The performances where good enough for what I had to do with the
BaselineCorretors. So, at very first sight, this should fit the other needs
(anyway, the other very few multi-threaded alternatives I gave a test where not
exploitable in term of evaluation speed).
May be we should/will have to consider switching between JRIengine and Rserve
for some particular features). I hope not, but in any case,
this is something which seems doable: as far as I tested, there is absolutely
no incompatibilities between Rserve and JRI.
To make your own opinion on all this, the BaselineCorretorModule (again) allows
to choose the R engine among RCaller (Online), JRIengine and Rserve.
Just run the MZmine version linked on top of this email, and go to "Raw data
methods > Filtering > Baseline Correction"
<Mail Attachment.png>
Fill free to ask if something is unclear or if you have any further
questionings.
Looking forward to your constructive comments and thoughts...
I'll be glad to help, as much as possible, with migrating to Rserve (if it
turns out that this solution really is viable).
Cheers
Gauthier
-------- Forwarded Message --------
Subject: Re: [Mzmine-devel] Baseline correction
Date: Tue, 2 Dec 2014 05:15:41 +0000
From: Tomas Pluskal <plus...@oist.jp><mailto:plus...@oist.jp>
To: Gauthier Boaglio
<gauthier.boag...@gmail.com><mailto:gauthier.boag...@gmail.com>
Hi Gauthier,
Now that we released version 2.12, I think it is a good time to consider making
the switch from JRI to RServe.
I think it is a good idea - I don't really like the current JRI interface.
It would be great if you could send a message to the devel list, where you can
summarize what you found about RServe. Especially, we would like to know
1) is the initial setup and configuration going to be easier than the current
one (is it necessary to edit the startMZmine script in order to use RServe?)
2) how does the actual code differ between RServe and JRI?
Thanks a lot!
Cheers,
Tomas
===============================================
Tomas Pluskal
G0 Cell Unit, Okinawa Institute of Science and Technology Graduate University
1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan
WWW: https://groups.oist.jp/g0
TEL: +81-98-966-8684
Fax: +81-98-966-2890
--
Gauthier BOAGLIO
CEFE - UMR 5175
1919 route de Mende
F-34293 Montpellier cedex 5
Tel: +33/0 4 67 61 32 15
Fax: +33/0 4 67 61 33 36
email: gauthier.boag...@cefe.cnrs.fr<mailto:gauthier.boag...@cefe.cnrs.fr>
www:
http://www.cefe.cnrs.fr/en/evolutionary-ecology-and-epidemiology/gauthier-boaglio
http://www.evolepid.org/people.php?name=boaglio
--
Gauthier BOAGLIO
CEFE - UMR 5175
1919 route de Mende
F-34293 Montpellier cedex 5
Tel: +33/0 4 67 61 32 15
Fax: +33/0 4 67 61 33 36
email: gauthier.boag...@cefe.cnrs.fr<mailto:gauthier.boag...@cefe.cnrs.fr>
www:
http://www.cefe.cnrs.fr/en/evolutionary-ecology-and-epidemiology/gauthier-boaglio
http://www.evolepid.org/people.php?name=boaglio
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk_______________________________________________
Mzmine-devel mailing list
Mzmine-devel@lists.sourceforge.net<mailto:Mzmine-devel@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/mzmine-devel
===============================================
Tomas Pluskal
G0 Cell Unit, Okinawa Institute of Science and Technology Graduate University
1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan
WWW: https://groups.oist.jp/g0
TEL: +81-98-966-8684
Fax: +81-98-966-2890
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
Mzmine-devel mailing list
Mzmine-devel@lists.sourceforge.net<mailto:Mzmine-devel@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/mzmine-devel
--
Gauthier BOAGLIO
CEFE - UMR 5175
1919 route de Mende
F-34293 Montpellier cedex 5
Tel: +33/0 4 67 61 32 15
Fax: +33/0 4 67 61 33 36
email: gauthier.boag...@cefe.cnrs.fr<mailto:gauthier.boag...@cefe.cnrs.fr>
www:
http://www.cefe.cnrs.fr/en/evolutionary-ecology-and-epidemiology/gauthier-boaglio
http://www.evolepid.org/people.php?name=boaglio
===============================================
Tomas Pluskal
G0 Cell Unit, Okinawa Institute of Science and Technology Graduate University
1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan
WWW: https://groups.oist.jp/g0
TEL: +81-98-966-8684
Fax: +81-98-966-2890
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
Mzmine-devel mailing list
Mzmine-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mzmine-devel
