Hi Dario -- This didn't cause problems for me. One thing you might do is to
re-use the MulticoreParam() like this
param = bpstart(MulticoreParam(25))
lapply(1:20, function(variety) bplapply(1:100, function() {}, BPPARAM =
param))
bpstop(param)
Also there have been a number of changes in devel and if possible it would be
good to know whether the problem still appears there.
It might be interesting to see the number of connections open when the process
hangs, from the command line while R is still running
~$ lsof -i |grep -c R
Martin
________________________________________
From: Bioc-devel [[email protected]] on behalf of Dario Strbenac
[[email protected]]
Sent: Sunday, December 27, 2015 8:00 PM
To: [email protected]
Subject: [Bioc-devel] bplapply Processes Sometimes Stall
Good day,
I have a minimal example of bplapply stalling.
results <- lapply(1:20, function(variety)
{
message("Variety", variety)
bplapply(1:100, function(index) {res <- list(sample(20000),
sample(c("Healthy", "Disease"), 20000, replace = TRUE)); res}, BPPARAM =
MulticoreParam(workers = 25))
})
It sometimes stalls on no particular iteration, but other times it runs all 20
iterations and returns to the R command prompt. It's not reproducible when the
stall happens. I am trying to find the cause of a cross-validation loop that
progresses for a few hours, then stalls. When the stall happens, two or three
of the R processes appear to be using 100% CPU whereas the others are finished,
according to the output of top. The server was previously running R 3.1.2 and
Debian 7 and this didn't ever happen. The server has 48 processors.
If I set workers to 5, it always completes the loop and returns to the prompt.
Using mclapply with mc.cores set to 25 also always works, so the problem is
with bplapply.
R version 3.2.3 (2015-12-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 8 (jessie)
locale:
[1] LC_CTYPE=C.UTF-8 LC_NUMERIC=C LC_TIME=C.UTF-8
LC_COLLATE=C.UTF-8
[5] LC_MONETARY=C.UTF-8 LC_MESSAGES=C.UTF-8 LC_PAPER=C.UTF-8
LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=C.UTF-8
LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base
other attached packages:
[1] BiocParallel_1.4.3
loaded via a namespace (and not attached):
[1] futile.logger_1.4.1 lambda.r_1.1.7 futile.options_1.0.0
--------------------------------------
Dario Strbenac
PhD Student
University of Sydney
Camperdown NSW 2050
Australia
_______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
This email message may contain legally privileged and/or confidential
information. If you are not the intended recipient(s), or the employee or
agent responsible for the delivery of this message to the intended
recipient(s), you are hereby notified that any disclosure, copying,
distribution, or use of this email message is prohibited. If you have received
this message in error, please notify the sender immediately by e-mail and
delete this email message from your computer. Thank you.
_______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
