niyue commented on pull request #11588: URL: https://github.com/apache/arrow/pull/11588#issuecomment-961117611
@pitrou Sorry, probably I didn't make it clear enough. This is not what I concern. My use case is pretty similar with the elastic's discussion @westonpace mentioned above. In my case, I stored a list of sorted terms in a string array in arrow IPC file, and do mmap and binary search on this sorted string array. Each term is a short string (usually less than 10 bytes), when the program reads the term, only 1 page (the minimum unit of IO) is needed to be loaded since the program requests only less than 10 bytes from OS for each access. However, from what I currently observe, macOS 11.6 will prefetch 24 pages (even only 1 page is requested by the program), and Linux (Alpine Linux running in a Docker container) will prefetch 16 pages (even only 1 page is requested). The array in my case is not very small (typically 0.5~1 million words in the array), two binary search accesses usually won't fall into the prefetched pages range, which makes the OS prefetching useless and disk doing lots of unnecessary IO. We have thousands of such IPC files to binary search for each request, and the amount of wasted IO is non trivial. Additionally, some of the storage we test (from some big cloud vendor), have limited bandwidth, which makes this behavior worse. I submitted this PR because after reading some documentation, I realized I need to advise OS not doing the prefetching for data that is NOT requested by user program since I know up front the prefetching is useless. So far I find this will not matter too much if the storage has good bandwidth, but it could help a lot if the storage bandwidth is limited. But I don't perform test for multiple concurrent programs yet, and I feel even for fast storage, this could help if there are multiple programs running since system page cache can be utilized better. More generally, I think people may run into such issue when doing random access for an mmaped array with small size element (e.g. small string, integer, date time, most of the primitive types) in an IPC file if they read these files from bandwidth limited storage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
