thisisnic commented on issue #43404: URL: https://github.com/apache/arrow/issues/43404#issuecomment-2254122929
I think this is a great idea @ajinkya-k though I'm not sure how feasible this is as this has been discussed in relation to another piece of functionality and we concluded it would be tricky as it'd require non-trivial updates to the Arrow C++ library. Out of interest, once you've loaded the file, are you performing further dplyr manipulations? It might be that you get better performance calling `open_dataset()` on the file so it's not pulled into your R session, running whatever manipulations, and then only calling `collect()` to pull the relevant bits into memory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
