[
https://issues.apache.org/jira/browse/HDFS-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632920#comment-13632920
]
Kihwal Lee commented on HDFS-4697:
----------------------------------
The OS-level readahead will happen even without this for most sequential reads.
The fadvise done by manageOsCache() helps triggering OS readahead, which may
not happen for slow small reads. The gap between reads are worse for remote
reads and that's where manageOsCache() would help most. It does not, however,
prolong the lifetime of cached data, so if the reader is slow and the memory
pressure is high, the data may get thrown away before the reader gets to it. In
such cases, it may actually lower the overall system throughput by causing
extra reads.
I agree that this needs to be fixed, but am also curious how much performance
improvement can be obtained for short-circuit reads. If we have important use
cases for the precise control of caching and disk activities, aio + direct i/o
can be used. What are the common and performance-critical access patterns of
hbase? All I know from old days is that it does a lot of random reads of about
64KB.
> short-circuit reads do not honor readahead settings
> ---------------------------------------------------
>
> Key: HDFS-4697
> URL: https://issues.apache.org/jira/browse/HDFS-4697
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
>
> Neither the new nor the legacy short-circuit read implementations honor
> {{dfs.datanode.readahead.bytes}}. This can result in scenarios where
> non-short-circuit reads are faster for long reads, simply because they are
> doing more readahead, and SCR is not. We should do readahead in both cases
> if it is configured.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
