marsupialtail commented on PR #13640: URL: https://github.com/apache/arrow/pull/13640#issuecomment-1221236445
Let's recap here -- we started from this PR, then we went to https://github.com/apache/arrow/pull/13662 because we think O_DIRECT is not needed. Turns out fadvise performance is bad, and we should revert to O_DIRECT. We will be doing O_DIRECT without O_SYNC, because we do not need to persist the writes. The direct IO here is not for persistence/fault tolerance but rather to relieve memory pressure, so you don't care if the data is persisted on disk or in "limbo" (most likely SSD cache) as long as it's not in page cache. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
