[ 
https://issues.apache.org/jira/browse/KUDU-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18147525#comment-18147525
 ] 

ASF subversion and git services commented on KUDU-3830:
-------------------------------------------------------

Commit 15c4979b8c08b928e35f695f4501d71fd925619d in kudu's branch 
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=15c4979b8 ]

KUDU-3830: Add RWFileOptions::random_access to disable readahead

If set, PosixEnv::NewRWFile() advises the kernel with
posix_fadvise(POSIX_FADV_RANDOM) that the file is read at random
offsets, so the kernel doesn't read ahead when reading from it. The
advice applies only to the newly opened file description; writes and
hole punching are not affected. A failure of posix_fadvise() is logged
as a warning since it affects only performance, not correctness.
On platforms other than Linux, the option is ignored and a warning is
logged once.

The default is false, so the behavior of existing callers doesn't
change.

The motivation comes from Impala's remote data cache, which uses
kudu::RWFile for its backing files and reads mainly random offsets.
On tpcds 10 TB benchmark with NVMe drive + RHEL 8.6 this reduced
IO wait time ~40%.

Change-Id: I394ef038c5dc21752fbadf9eda48dc933cc56aa9
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Reviewed-on: https://gerrit.cloudera.org/25464
Reviewed-by: Alexey Serbin <[email protected]>
Reviewed-by: Ashwani Raina <[email protected]>
Tested-by: Ashwani Raina <[email protected]>


> Allow fadvise POSIX_FADV_RANDOM on RWFile to avoid readahead
> ------------------------------------------------------------
>
>                 Key: KUDU-3830
>                 URL: https://issues.apache.org/jira/browse/KUDU-3830
>             Project: Kudu
>          Issue Type: Improvement
>            Reporter: Csaba Ringhofer
>            Assignee: Csaba Ringhofer
>            Priority: Major
>
> Impala uses RWFile as remote data cache (e.g. cache S3 reads to nvme). 
> Turning off readahead almost double IO speed on RHEL 8 +m6id.4xl.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
  • [jira] [Commented] (KUDU... Csaba Ringhofer (Jira) via issues
    • [jira] [Commented] ... ASF subversion and git services (Jira) via issues

Reply via email to