[
https://issues.apache.org/jira/browse/HDFS-6988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157687#comment-14157687
]
Arpit Agarwal commented on HDFS-6988:
-------------------------------------
bq. Why is dfs.datanode.ram.disk.low.watermark.replicas specified in terms of
number of replicas? Block size is a per-replica property--
Ok, we can make them byte counts.
bq. Block size is a per-replica property-- I could easily have a client that
writes 256 MB or 1 GB replicas, while the DataNode is configured with
dfs.blocksize at 64MB. It's pretty common for formats like ORCFile and Apache
Parquet to use large blocks and seek around within them.
This also favors a limit of at least 1GB + some delta so we can keep space
around at least 1 replica and its meta file.
If we are going with a single value can you suggest a default.
> Add configurable limit for percentage-based eviction threshold
> --------------------------------------------------------------
>
> Key: HDFS-6988
> URL: https://issues.apache.org/jira/browse/HDFS-6988
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: HDFS-6581
> Reporter: Arpit Agarwal
> Fix For: HDFS-6581
>
> Attachments: HDFS-6988.01.patch, HDFS-6988.02.patch
>
>
> Per feedback from [~cmccabe] on HDFS-6930, we can make the eviction
> thresholds configurable. The hard-coded thresholds may not be appropriate for
> very large RAM disks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
