amoeba commented on issue #45300: URL: https://github.com/apache/arrow/issues/45300#issuecomment-2605208539
Thanks @nealrichardson. @nicki-dese: I think the workflow you're after isn't something the arrow R package can support unfortunately. If data.table were to elevate that behavior to some sort of standard interface it might be possible to reconsider. To re-gain the speed you had before, you may have to explore more of the functionality of the arrow R package or packages like duckdb. Generally speaking, writing a table out to Parquet and reading it back in is expensive and something the these packages can help you avoid. Happy to chat more here about this or the original report. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
