pitrou edited a comment on pull request #10485: URL: https://github.com/apache/arrow/pull/10485#issuecomment-857684749
Hmm, what is the context where this need stems from? `fadvise` is advisory. Furthermore, if you're reading the file in streaming fashion, the kernel is probably smart enough to recognize that and readahead a bit on its own. Using `fadvise`/`madvise` is mostly useful for random access, AFAIU (e.g. for Parquet or IPC file reads). More generally, there may be arcane ways to optimize local reads, but baseline performance should generally be quite good already, since that's an extremely common use case that the kernel is in a good position to optimize for. Our main challenge is for remote filesystems with significant latency and/or throughput constraints. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
