Hello,
I have a piece of code that needs parallelization and it used to work just fine
before (about 6 months ago). However, I had to rerun it yesterday and found out
that my code is now behaving in a weird way - not all worker processes are
charged with the computation. I created a little code that allows reproducing
the issue. Here is a snapshot of the top command:
PID COMMAND %CPU TIME
872 R 97.7 00:02.45
871 R 0.0 00:00.03
870 R 98.2 00:02.93
869 R 0.0 00:00.03
868 R 97.7 00:02.46
867 R 0.0 00:00.03
866 R 94.4 00:02.36
862 R 1.0 00:04.64
Interestingly, the mclapply command seems to correctly charge all workers
instantly.
So is this an intended behavior?
Example that reproduces the issue on my machine:
#
rm(list = ls())
library(parallel)
set.seed(123)
#
expr <- expression(pnorm(b0 + b1*x1 + b2*x2 + b3*x3))
G <- list(
D(D(expr, "x2"), "b0"),
D(D(expr, "x2"), "b1"),
D(D(expr, "x2"), "b2"),
D(D(expr, "x2"), "b3"))
#
b0 <- 1.2
b1 <- 0.4
b2 <- 0.2
b3 <- -0.6
x1 <- rnorm(10^7)
x2 <- rnorm(10^7)
x3 <- rnorm(10^7)
#
nc <- detectCores() - 1
cl <- makeCluster(nc, type = "FORK")
grad_g <- parLapply(cl, G, function(Z) lapply(Z, function(x) mean(eval(x))))
stopCluster(cl)
#
sessionInfo()
R version 3.2.3 (2015-12-10)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.2 (El Capitan)
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base
----
Ce message electronique et tous les fichiers attaches qu'il contient sont
confidentiels et destines exclusivement à l'usage de la personne à laquelle ils
sont adresses. Si vous avez reçu ce message par erreur, merci de le retourner à
son metteur. Les idees et opinions presentees dans ce message sont celles de
son auteur, et ne representent pas necessairement celles de l'institution ou
entite affiliee dont l'auteur est l'employe. La publication, l'usage, la
distribution, l'impression ou la copie non autorisee de ce message et des
attachements qu'il contient sont strictement interdits.
This email and any files transmitted with it are confide...{{dropped:10}}
_______________________________________________
R-SIG-Mac mailing list
[email protected]
https://stat.ethz.ch/mailman/listinfo/r-sig-mac
