[
https://issues.apache.org/jira/browse/HDFS-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
binlijin updated HDFS-4184:
---------------------------
Description:
HDFS now has the ability to use posix_fadvise and sync_data_range syscalls to
manage the OS buffer cache.
{code}
When hbase read hlog the data we can set dfs.datanode.drop.cache.behind.reads
to true to drop data out of the buffer cache when performing sequential reads.
When hbase write hlog we can set dfs.datanode.drop.cache.behind.writes to true
to drop data out of the buffer cache after writing
When hbase read hfile during compaction we can set dfs.datanode.readahead.bytes
to a non-zero value to trigger readahead for sequential reads， and also set
dfs.datanode.drop.cache.behind.reads to true to drop data out of the buffer
cache when performing sequential reads.
and so on...
{code}
Current we can only set these feature global in datanode，we should set these
feature per session.
was:
HDFS now has the ability to use posix_fadvise and sync_data_range syscalls to
manage the OS buffer cache.
{code}
When hbase read hlog the data we can set dfs.datanode.drop.cache.behind.reads
to true to drop data out of the buffer cache when performing sequential reads.
When hbase write hlog we can set dfs.datanode.drop.cache.behind.writes to true
to drop data out of the buffer cache after writing
When hbase read hfile during compaction we can set dfs.datanode.readahead.bytes
to a non-zero value to trigger readahead for sequential reads， and also set
dfs.datanode.drop.cache.behind.reads to true to drop data out of the buffer
cache when performing sequential reads.
and so on...
{code}
> Add the ability for Client to provide more hint information for DataNode to
> manage the OS buffer cache more accurate
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4184
> URL: https://issues.apache.org/jira/browse/HDFS-4184
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: binlijin
>
> HDFS now has the ability to use posix_fadvise and sync_data_range syscalls to
> manage the OS buffer cache.
> {code}
> When hbase read hlog the data we can set dfs.datanode.drop.cache.behind.reads
> to true to drop data out of the buffer cache when performing sequential reads.
> When hbase write hlog we can set dfs.datanode.drop.cache.behind.writes to
> true to drop data out of the buffer cache after writing
> When hbase read hfile during compaction we can set
> dfs.datanode.readahead.bytes to a non-zero value to trigger readahead for
> sequential reads， and also set dfs.datanode.drop.cache.behind.reads to true
> to drop data out of the buffer cache when performing sequential reads.
> and so on...
> {code}
> Current we can only set these feature global in datanode，we should set these
> feature per session.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
