Re: [Rd] [parallel] fixes load balancing of parLapplyLB

2018-03-13 Thread Tomas Kalibera
Chunk size support has been added in R-devel 74353. Please let me know if you find any problem. Thanks, Tomas On 03/01/2018 09:19 AM, Christian Krause wrote: Dear Tomas, Thanks for your commitment to fix this issue and also to add the chunk size as an argument. If you want our input, let

Re: [Rd] [parallel] fixes load balancing of parLapplyLB

2018-03-01 Thread Christian Krause
Dear Tomas, Thanks for your commitment to fix this issue and also to add the chunk size as an argument. If you want our input, let us know ;) Best Regards On 02/26/2018 04:01 PM, Tomas Kalibera wrote: > Dear Christian and Henrik, > > thank you for spotting the problem and suggestions for a

Re: [Rd] [parallel] fixes load balancing of parLapplyLB

2018-02-26 Thread Tomas Kalibera
Dear Christian and Henrik, thank you for spotting the problem and suggestions for a fix. We'll probably add a chunk.size argument to parLapplyLB and parLapply to follow OpenMP terminology, which has already been an inspiration for the present code (parLapply already implements static

Re: [Rd] [parallel] fixes load balancing of parLapplyLB

2018-02-20 Thread Christian Krause
Dear Henrik, The rationale is just that it is within these extremes and that it is really simple to calculate, without making any assumptions and knowing that it won't be perfect. The extremes A and B you are mentioning are special cases based on assumptions. Case A is based on the assumption

Re: [Rd] [parallel] fixes load balancing of parLapplyLB

2018-02-19 Thread Henrik Bengtsson
Hi, I'm trying to understand the rationale for your proposed amount of splitting and more precisely why that one is THE one. If I put labels on your example numbers in one of your previous post: nbrOfElements <- 97 nbrOfWorkers <- 5 With these, there are two extremes in how you can split up

Re: [Rd] [parallel] fixes load balancing of parLapplyLB

2018-02-19 Thread Christian Krause
Dear R-Devel List, I have installed R 3.4.3 with the patch applied on our cluster and ran a *real-world* job of one of our users to confirm that the patch works to my satisfaction. Here are the results. The original was a series of jobs, all essentially doing the same stuff using bootstrapped

[Rd] [parallel] fixes load balancing of parLapplyLB

2018-02-12 Thread Christian Krause
Dear R-Devel List, **TL;DR:** The function **parLapplyLB** of the parallel package has [reportedly][1] (see also attached RRD output) not been doing its job, i.e. not actually balancing the load. My colleague Dirk Sarpe and I found the cause of the problem and we also have a patch to fix it