paleolimbot commented on issue #15271: URL: https://github.com/apache/arrow/issues/15271#issuecomment-1377370938
Thanks for opening this! I'm not surprised that very small objects have this property...Arrow's columnar format exploits that there are frequently many more rows than columns, and there are some places in the R package where we loop over columns in R. Mostly that is fine, although looping in R for 500 columns, as you've seen, can result in some overhead. If you look at the absolute times (instead of the relative times), I imagine that what you're seeing is still very small (maybe 0.1s) overhead...R can just do that much faster. Is there a workflow where you're seeing this impact analysis time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
