I appreciate the welcome! Also - I believe that replying to an email is the way to respond here, but please let me know if that's not the case.
In any event - passing in a cluster context is an interesting idea. I will think about that. Also, it seems that despite me telling myself to write bug-free code, you have correctly identified that I don't actually make use of the passed cores parameter - oops! This is where it would have really helped me to have a peer reviewer. Thanks! On Fri, Feb 14, 2025 at 3:48 PM Ivan Krylov <ikry...@disroot.org> wrote: > Dear Stephen Abrams, > > Welcome to R-package-devel! > > В Thu, 13 Feb 2025 22:20:50 -0500 > Stephen Abrams <stephen.abr...@gmail.com> пишет: > > > A secondary worry is that even if I resolve this, there might be > > something else causing threads to spin up. > > Instead of using detectCores() [*] and creating cluster objects > yourself, how about letting the user provide a cluster object for you > as a function argument? Yes, it takes slightly more typing for the user, > but on the other hand it lets the user: > > - choose the number of cores for themselves (currently the code seems > to be ignoring the 'cores' argument) > - distribute the computation over the network by connecting to the > machines they know about > - provide a completely custom, non-PSOCK cluster object that > 'parallel' will nevertheless will work with > > Since you're already using doParallel, maybe the right choice is to let > the user call registerDoParallel()? > > Determining the right amount of parallelism in your code is a > surprisingly hard problem. Especially on shared computers, a program > naively deciding to use all (or 3/4 of all, or 1/2 of all) processors > may end up working much worse than a purely sequential one [**]. > > While rendering the vignette in a CRAN package, create a two-process > cluster or set use_parallel = FALSE: CRAN needs the rest of the > processors to check other packages in parallel with yours [***]. > > Good luck! > > -- > Best regards, > Ivan > > [*] > > https://github.com/dawdawdo/spect/blob/d48b002332f1a1c2d302afb28e08e7998f416200/R/spect.R#L388 > > [**] > https://mastodon.social/@henrikbengtsson/113835651303559942 > > [***] > > http://contributor.r-project.org/cran-cookbook/code_issues.html#using-more-than-2-cores > -- Stephen Abrams Divergent Blue <http://www.divergentblue.com/> [[alternative HTML version deleted]] ______________________________________________ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
