Re: [R] Rsge: recursive parallelization

2010-04-12 Thread Peter Danenberg
Quoth Martin Morgan on Boomtime, the 29th of Discord: It seems like it would be hard to think about the tasks that are being executed, how many processes there are, how load balancing works, etc. A flat representation of our nested data is, alas, too large to contain in memory; and dynamically

Re: [R] Rsge: recursive parallelization

2010-04-10 Thread Martin Morgan
On 04/09/2010 08:52 AM, Peter Danenberg wrote: In principle, I'd like to be able to do something like this: sge.parLapply(seq(10), function(x) parLapply(seq(x), function(x) x^2)) I'm not sure that's such a good principle! It seems like it would be hard to think about the tasks that are

[R] Rsge: recursive parallelization

2010-04-09 Thread Peter Danenberg
In principle, I'd like to be able to do something like this: sge.parLapply(seq(10), function(x) parLapply(seq(x), function(x) x^2)) In practice, however, I have to resort to acrobatics like this: sge.options(sge.remove.files=FALSE) sge.options(sge.qsub.options='-cwd -V')