-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/29/2011 02:34 PM, Vincent Aubanel wrote: > Hi all, > > I'm using mclapply() of the multicore package for processing chunks > of data in parallel --and it works great. > > But when I want to collect all processed elements of the returned > list into one big data frame it takes ages. > > The elements are all data frames having identical column names, and > I'm using a simple rbind() inside a loop to do that. But I guess it > makes some expensive checking computations at each iteration as it > gets slower and slower as it goes. Writing out to disk individual > files, concatenating with the system and reading back from disk the > resulting file is actually faster...
Why do you have to write to disk? Can you collect the results as a list L and then do.call(rbind,L) in one go? > > Is there a magic argument to rbind() that I'm missing, or is there > any other solution to collect the results of parallel processing > efficiently? > > Thanks, Vincent > > _______________________________________________ R-SIG-Mac mailing > list [email protected] > https://stat.ethz.ch/mailman/listinfo/r-sig-mac -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4Lc1sACgkQc5UpGjwzenMofgCdFEzN6qO8e8owV9GCJ6Mwafvn n0oAoIZt5txoRS9Ma73XefnEzBDW19lE =BDfd -----END PGP SIGNATURE----- _______________________________________________ R-SIG-Mac mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-mac
