[
https://issues.apache.org/jira/browse/HDFS-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137667#comment-13137667
]
Todd Lipcon commented on HDFS-2465:
-----------------------------------
bq. There isn't any config that enables syncBehindWrites in BlockerRecevier
Woops! good catch. Lost that somewhere along the line.
bq. Would it be more clear to express LONG_READ_THRESHOLD in terms of a
multiple of the packet size or the IO file buffer size? Seems like the host
readahead should match the implicit HDFS readahead.
I don't think so. HDFS doesn't really do readahead except for filling up the
TCP window. But submitting very large (multi-MB)readahead to the OS lets it
schedule big sequential reads. So, I don't think there's any real correlation.
To set this tunable, I'd probably do some mental math like: "well, I probably
have 20 files being read at once, so 8M of readahead on each of those will end
up being 160M of RAM used." Having to multiply by other factors is harder to
follow.
Fixed the other nits. New patch momentarily.
> Add HDFS support for fadvise readahead and drop-behind
> ------------------------------------------------------
>
> Key: HDFS-2465
> URL: https://issues.apache.org/jira/browse/HDFS-2465
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hdfs-2465.txt
>
>
> This is the HDFS side of HADOOP-7714. The initial implementation is heuristic
> based and should be considered experimental, as discussed in the parent JIRA.
> It should be off by default until better heuristics, APIs, and tuning
> experience is developed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
