Jim Brennan created YARN-9914:
---------------------------------
Summary: Use separate configs for free disk space checking for
full and not-full disks
Key: YARN-9914
URL: https://issues.apache.org/jira/browse/YARN-9914
Project: Hadoop YARN
Issue Type: Improvement
Components: yarn
Reporter: Jim Brennan
Assignee: Jim Brennan
[YARN-3943] added separate configurations for the nodemanager health check disk
utilization full disk check:
{{max-disk-utilization-per-disk-percentage}} - threshold for marking a good
disk full
{{disk-utilization-watermark-low-per-disk-percentage}} - threshold for marking
a full disk as not full.
On our clusters, we do not use these configs. We instead use
{{min-free-space-per-disk-mb}} so we can specify the limit in mb instead of
percent of utilization. We have observed the same oscillation behavior as
described in [YARN-3943] with this parameter. I would like to add an optional
config to specify a separate threshold for marking a full disk as not full:
{{min-free-space-per-disk-mb}} - threshold at which a good disk is marked full
{{disk-free-space-per-disk-high-watermark-mb}} - threshold at which a full disk
is marked good.
So for example, we could set {{min-free-space-per-disk-mb = 5GB}}, which would
cause a disk to be marked full when free space goes below 5GB, and
{{disk-free-space-per-disk-high-watermark-mb = 10GB}} to keep the disk in the
full state until free space goes above 10GB.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
