lidavidm commented on issue #10899: URL: https://github.com/apache/arrow/issues/10899#issuecomment-896009602
So to be clear: Parquet, CSV, and Feather are all different file formats, and memory-mapping is just one way to read a file. In the case of uncompressed Feather files *only*, you can get zero-copy reads. Again, this is because the format on disk is in this case the same as the format in memory. So if memory is your bottleneck, it sounds like this is likely your best choice. Otherwise, memory-mapping may be faster (or slower!) than just reading the file. You cannot and will not get zero-copy from a Parquet file. However, Parquet files can be read incrementally, so you may still be able to make this work - it will just take more effort. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
