Hello Yan, I think parLapply is just assigning a core for every item in the list, G, you supplied. Because you have more cores than items in the list, some of the cores won't receive any work.
John On Fri, Dec 11, 2015 at 4:00 AM, ALPEROVYCH Yan <[email protected]> wrote: > Hello, > > I have a piece of code that needs parallelization and it used to work just > fine before (about 6 months ago). However, I had to rerun it yesterday and > found out that my code is now behaving in a weird way - not all worker > processes are charged with the computation. I created a little code that > allows reproducing the issue. Here is a snapshot of the top command: > > PID COMMAND %CPU TIME > 872 R 97.7 00:02.45 > 871 R 0.0 00:00.03 > 870 R 98.2 00:02.93 > 869 R 0.0 00:00.03 > 868 R 97.7 00:02.46 > 867 R 0.0 00:00.03 > 866 R 94.4 00:02.36 > 862 R 1.0 00:04.64 > > Interestingly, the mclapply command seems to correctly charge all workers > instantly. > > So is this an intended behavior? > > Example that reproduces the issue on my machine: > # > rm(list = ls()) > library(parallel) > set.seed(123) > # > expr <- expression(pnorm(b0 + b1*x1 + b2*x2 + b3*x3)) > G <- list( > D(D(expr, "x2"), "b0"), > D(D(expr, "x2"), "b1"), > D(D(expr, "x2"), "b2"), > D(D(expr, "x2"), "b3")) > # > b0 <- 1.2 > b1 <- 0.4 > b2 <- 0.2 > b3 <- -0.6 > x1 <- rnorm(10^7) > x2 <- rnorm(10^7) > x3 <- rnorm(10^7) > # > nc <- detectCores() - 1 > cl <- makeCluster(nc, type = "FORK") > grad_g <- parLapply(cl, G, function(Z) lapply(Z, function(x) > mean(eval(x)))) > stopCluster(cl) > # > sessionInfo() > R version 3.2.3 (2015-12-10) > Platform: x86_64-apple-darwin13.4.0 (64-bit) > Running under: OS X 10.11.2 (El Capitan) > > locale: > [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 > > attached base packages: > [1] parallel stats graphics grDevices utils datasets methods > base > > > > > ---- > Ce message electronique et tous les fichiers attaches qu'il contient sont > confidentiels et destines exclusivement à l'usage de la personne à laquelle > ils sont adresses. Si vous avez reçu ce message par erreur, merci de le > retourner à son metteur. Les idees et opinions presentees dans ce message > sont celles de son auteur, et ne representent pas necessairement celles de > l'institution ou entite affiliee dont l'auteur est l'employe. La > publication, l'usage, la distribution, l'impression ou la copie non > autorisee de ce message et des attachements qu'il contient sont strictement > interdits. > > This email and any files transmitted with it are confide...{{dropped:10}} > > _______________________________________________ > R-SIG-Mac mailing list > [email protected] > https://stat.ethz.ch/mailman/listinfo/r-sig-mac > [[alternative HTML version deleted]] _______________________________________________ R-SIG-Mac mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-mac
