On 12 May 2016 at 09:18, Dirk Eddelbuettel wrote: | | On 12 May 2016 at 13:11, Mark van der Loo wrote: | | Charles, | | | | 1. Perhaps this question is better directed at the R-help or | | R-pacakge-devel mailinglist. | | | | 2. It basically means that R itself can only evaluate one R expression at | | the time. | | | | The parallel package circumvents this by starting multiple R-sessions and | | dividing workload. | | | | Compiled code called by R (such as C++ code through RCpp or C-code through | | base R's interface) can execute multi-threaded code for internal purposes, | | using e.g. openMP. A limitation is that compiled code cannot call R's C API | | from multiple threads (in many cases). For example, it is not thread-safe | | to create R-variables from multiple threads running in C. (R's variable | | administration is such that the order of (un)making them from compiled code | | matters). | | Well put. | | | I am not very savvy on Rcpp or XPtr objects, but it appears that Dirk | | provided answers about that in your SO-question. | | Charles seems to hang himself up completely about a small detail, failing to | see the forest for the trees. | | There are (many) working examples of parallel (compiled) code with R. All of | them stress (and I simplify here) that can you touch R objects, or call back
An import 'not' missing here (and a reordering); "that you CANNOT touch R objects" Sorry, Dirk | into R, for fear of any assignment or allocation triggering an R event. R | being single-threaded it cannot do this. | | My answer to this problem is to only use non-R data structures. That is what | RcpParallel does in the actual parallel code portions in all examples -- | types RVector and RMatrix do NOT connect back to R. There are several working | examples. That is also what the OpenMP examples at the Rcpp Gallery do. | | Charles seems to be replying 'but I use XPtr' or 'I use XPtr on arma::mat or | Eigen::Matrixxd' and seems to forget that these are proxy objects to SEXPs. | XPtr just wrap the SEXP for external pointers; Arma's and Eigen's matrices | are performant via RcppArmadillo and RcppEigen because we use R memory via | proxies. All of that is 'too close to R' for comfort. | | So the short answer is: enter compiled code from R, set a mutex (either | conceptually or explicitly), _copy_ your data in to plain C++ data structures | and go to town in parallel via OpenMP and other multithreaded approaches. | Then collect the result, release the mutex and move back up. | | I hope this help. | | Dirk | | | | | Best, | | Mark | | | | | | | | | | | | | | | | | | | | | | Op do 12 mei 2016 om 14:46 schreef Charles Determan <cdeterma...@gmail.com>: | | | | > R Developers, | | > | | > Could someone help explain what it means that R is single threaded? I am | | > trying to understand what is actually going on inside R when users want to | | > parallelize code. For example, using mclapply or foreach (with some | | > backend) somehow allows users to benefit from multiple CPUs. | | > | | > Similarly there is the RcppParallel package for RMatrix/RVector objects. | | > But none of these address the general XPtr objects in Rcpp. Some readers | | > here may recognize my question on SO ( | | > | | > http://stackoverflow.com/questions/37167479/rcpp-parallelize-functions-that-return-xptr | | > ) | | > where I was curious about parallel calls to C++/Rcpp functions that return | | > XPtr objects. I am being a little more persistent here as this limitation | | > provides a very hard stop on the development on one of my packages that | | > heavily uses XPtr objects. It's not meant to be a criticism or intended to | | > be rude, I just want to fully understand. | | > | | > I am willing to accept that it may be impossible currently but I want to at | | > least understand why it is impossible so I can explain to future users why | | > parallel functionality is not available. Which just echos my original | | > question, what does it mean that R is single threaded? | | > | | > Kind Regards, | | > Charles | | > | | > [[alternative HTML version deleted]] | | > | | > ______________________________________________ | | > R-devel@r-project.org mailing list | | > https://stat.ethz.ch/mailman/listinfo/r-devel | | > | | | | [[alternative HTML version deleted]] | | | | ______________________________________________ | | R-devel@r-project.org mailing list | | https://stat.ethz.ch/mailman/listinfo/r-devel | | -- | http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org | | ______________________________________________ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
