Michael,
On 26 July 2011 at 23:01, Michael Braun wrote:
| Dirk:
|
| I am sorry I was not clearer about what I am trying to accomplish. The issue
is with operating on elements of an Rcpp::List object in parallel.
R is very famously single-threaded. So when I experimented with OpenMP, I
tried a very basic and careful approach which, in pseudo code, does:
function foo() {
inits: get data from R, stick it into C++ data structures
do work, possibly with OpenMP
synchronise
pass data back into R data structure, return
}
| Before posting to the list, I had looked at the example to which you refer.
| But I am missing where the example does anything with an Rcpp object in
| parallel at all.
See above, maybe we shouldn't try to work in parallel with an R object. I'd
be happy to see an 'existence proof' example that does it.
| Nor do I see any instance of thread synchronization.
Point taken. I played with some in my own experimentations trying to get
RcppDE working in parallel (and I still have unsolved issues there
w.r.t. RNGs under OpenMP).
| So I am not sure how the example helps.
Well it does show how to use OpenMP and it uses a clever class and interrupt
handler to even allow you to interrupt out. That is neat.
| I have already tried the ordered
| directive (segfaults every time), and the critical directives around the
| call to func (no change). I don't know what else I could do with OpenMP;
| this is such a simple example. Perhaps you have something specific in
| mind. Is there any way you (or someone else on the list) could give me
| another hint?
Have you tried adding
#pragma omp barrier
and the end of your OpenMP to force thread synchronisation?
Otherwise, try maybe developing the core of your parallel algo in a
standalone program, outside of R and Rcpp, to "make it work" and then worry
about connecting it to R via Rcpp. One battle at a time...
Cheers, Dirk
| Thanks,
|
| Michael
|
|
|
|
| On Jul 26, 2011, at 8:26 PM, Dirk Eddelbuettel wrote:
|
| >
| > Michael,
| >
| > On 26 July 2011 at 17:08, Michael Braun wrote:
| > | I have an R list that I want to pass to C++ code through Rcpp, and then
process each of the list elements with another function, in parallel through
OpenMP. Here is a simplified example in which I compute the square root of a
number in each list element:
| > |
| > | // file omp.cpp
| > |
| > | #include <Rcpp.h>
| > | #include <omp.h>
| > |
| > | double func(SEXP Y_) {
| > | using namespace Rcpp;
| > | double Y = as<double>(Y_);
| > | double res = sqrt(Y);
| > | return(res);
| > | }
| > |
| > | RcppExport SEXP omp3 (SEXP R_) {
| > |
| > | BEGIN_RCPP
| > |
| > | using namespace Rcpp;
| > |
| > | List R = R_;
| > | int n = R.size();
| > | int i;
| > |
| > | NumericVector X(n);
| > |
| > | #pragma omp parallel shared(R, n, X) private(i)
| > | {
| > | #pragma omp for
| > | for (i=0; i<n; i++) {
| > | X(i) = func(R(i));
| > | }
| > | }
| > |
| > | return(wrap(X));
| > |
| > | END_RCPP
| > |
| > | }
| > |
| > |
| > | The R code to call the omp3 C++ function is:
| > |
| > | dyn.load("omp.so")
| > |
| > | R <- vector("list",5)
| > | for(i in 1:5) R[[i]] = i
| > |
| > | res <- .Call("omp3",R)
| > |
| > | dyn.unload("omp.so")
| > |
| > |
| > | When I run this program, I almost always get stack imbalance warnings
like:
| > |
| > | Warning: stack imbalance in '.Call', 33 then 32
| > | Warning: stack imbalance in '<-', 31 then 30
| > | Warning: stack imbalance in 'eval.with.vis', 27 then 26
| > | Warning: stack imbalance in '.Internal', 26 then 25
| > | Warning: stack imbalance in '<-', 20 then 19
| > | Warning: stack imbalance in '{', 18 then 17
| > | Warning: stack imbalance in 'if', 16 then 15
| > | Warning: stack imbalance in '{', 14 then 13
| > | Warning: stack imbalance in 'for', 8 then 7
| > |
| > | I say almost because it does not happen all the time, but if I run the
same script a few more times, I will invariably get a stack imbalance warning.
(Also, the numbers are not always the same).
| > |
| > | But there's more. If I run the program a bunch of times, eventually I
get:
| > |
| > | Lost warning messages
| > | > Error in eval.with.vis(expr, envir, enclos) :
| > | REAL() can only be applied to a 'numeric', not a 'NULL'
| > | Error during wrapup: C stack usage is too close to the limit
| > |
| > |
| > | Once I get this error, I will no longer get stack imbalance warnings, but
I cannot predict how many attempts it would take before the C stack usage error
comes up.
| > |
| > | And if I am really lucky, I'll get memory not mapped segfaults, but not
every time.
| > |
| > | I have run this on my Mac Pro with both g++ 4.6 and Intel C++ compiler
12.0.4. I have also run it on a Red Hat Linux machine with g++ 4.4 (the
segfaults happen much more frequently under Linux than under Mac OSX). The
stack imbalance warnings come up on both machines, with all compilers.
| > |
| > | I also wonder if this is related to another problem with Rcpp and OpenMP
that I posted back in late March. In that case, the problem was using an old
compiler; updating the compiler solved the compilation problem. The
difference between that case and this example is calling the second function
with a SEXP as an argument. This example compiles, but does not run.
| > |
| > | I know that Rcpp is rigorously tested, so the problem is probably
something that I am doing wrong. In particular, I wonder if each element of
the list should be passed as a SEXP, or if there is a better way to do it. In
any case, I greatly appreciate your assistance.
| >
| > I would have to agree with your conjecture here. You are simply not being
| > careful enough about OpenMP usage. And R tells you that there are still
| > dynamically created objects floating around.
| >
| > I would recommend stronger synchronisation before and particularly after
your
| > OpenMP block. For a working example of OpenMP with Rcpp, see the directory
| > examples/OpenMP/ which was added in the 0.9.5 release.
| >
| > | (Also, I deeply apologize for not using the inline package with the
example. I could not figure out how to include the two different functions.)
| >
| > See help(cxxfunction) and particularly the "includes=" argument. That said,
| > for OpenMP you need to pass -fopenmp and we currently do not have an option
| > for that in cxxfunction.
| >
| > Hope this helps, Dirk
| >
| >
| > | Thanks,
| > |
| > | Michael
| > |
| > |
| > |
| > |
| > |
|
|
|
|
| xapplication/pkcs7-signatu [Click mouse-2 to save to a file]
--
Gauss once played himself in a zero-sum game and won $50.
-- #11 at http://www.gaussfacts.com
_______________________________________________
Rcpp-devel mailing list
Rcpp-devel@lists.r-forge.r-project.org
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
