The requirement to set use.names to TRUE if fill is TRUE seems ugly. I suggest that fill be the default for use.names.
On Tue, May 20, 2014 at 3:27 PM, Arunkumar Srinivasan <[email protected]> wrote: > Hello everyone, > > With the latest commit #1266, the extra functionality offered via rbind > (use.names and fill) is also now available to rbindlist. In addition, the > implementation is completely moved to C, and is therefore tremendously fast, > especially for cases where one has to bind using with use.names=TRUE and/or > with fill=TRUE. I’ll try to put out a benchmark comparing speed differences > with the older implementation ASAP. > > Note that this change comes with a very low cost to the default speed to > rbindlist - with use.names=FALSE and fill=FALSE. As an example, binding > 10,000 data.tables with 20 columns each, resulted in the new version running > in 0.107 seconds, where as the older version ran in 0.095 seconds. > > In addition the documentation for ?rbindlist also has been improved (#5158 > from Alexander). Here’s the change log from NEWS: > > o 'rbindlist' gains 'use.names' and 'fill' arguments and is now > implemented entirely in C. Closes #5249 > -> use.names by default is FALSE for backwards compatibility > (doesn't bind by names by default) > -> rbind(...) now just calls rbindlist() internally, except that > 'use.names' is TRUE by default, > for compatibility with base (and backwards compatibility). > -> fill by default is FALSE. If fill is TRUE, use.names has to be > TRUE. > -> At least one item of the input list has to have non-null column > names. > -> Duplicate columns are bound in the order of occurrence, like > base. > -> Attributes that might exist in individual items would be lost in > the bound result. > -> Columns are coerced to the highest SEXPTYPE, if they are > different, if/when possible. > -> And incredibly fast ;). > -> Documentation updated in much detail. Closes DR #5158. > Eddi's (excellent) work on finding factor levels, type coercion of > columns etc. are all retained. > > Please try it and write back if things aren’t working as it was before. The > tests that had to be fixed are extremely rare cases. I suspect there should > be minimal issue, if at all, in this version. However, I do find the changes > here bring consistency to the function. > > One (very rare) feature that is not available due to this implementation is > the ability to recycle. > > dt1 <- data.table(x=1:3, y=4:6, z=list(1:2, 1:3, 1:4)) > lst1 <- list(x=4, y=5, z=as.list(1:3)) > > rbind(dt1, lst1) > # x y z > # 1: 1 4 1,2 > # 2: 2 5 1,2,3 > # 3: 3 6 1,2,3,4 > # 4: 4 5 1 > # 5: 4 5 2 > # 6: 4 5 3 > > The 4,5 are recycled very nicely here.. This is not possible at the moment. > This is because the earlier rbind implementation used as.data.table to > convert to data.table, however it takes a copy (very inefficient on huge / > many tables). I’d love to add this feature in C as well, as it would help > incredibly for use within [.data.table (now that we can fill columns and > bind by names faster). Will add a FR. > > In summary, I think there should be minimal issues, if any and should be > much faster (for rbind cases). Please write back what you think, if you > happen to try out. > > > > Arun > > > _______________________________________________ > datatable-help mailing list > [email protected] > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help -- Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com _______________________________________________ datatable-help mailing list [email protected] https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
