All, Does anyone have experience using data.table in parallel using doMC/foreach?
I have a data.table with approximately 3.5 million rows and am calculating different summaries (e.g., medians and counts) on some of the variables across approximately 500 distinct groupings (same j variable with 500 different by groups). My thought was to run the different analyses in parallel on my multi-core machine and get a good performance boost but thus far it doesn't work any faster. The data.table performance is great for each by grouping (about 7.5 seconds), but doing this 500 times take a while. Anyone have experience along these lines. Best, Damian
_______________________________________________ datatable-help mailing list [email protected] https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
