[
https://issues.apache.org/jira/browse/YARN-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923760#comment-13923760
]
Varun Vasudev commented on YARN-1781:
-------------------------------------
{quote}
Styling
Some of the patch has non-standard formatting. Please fix them. For e.g.,
we say
if (condition) {
//
} else {
//
}
{quote}
My apologies for this. Fixed.
{quote}
YarnConfiguration
Create a prefix for .disk-health-checker. and use it for all Disk-check
related configs.
max-space-utilization-perc -> max-disk-utilization-per-disk-percentage?
min-free-space-mb -> min-free-space-per-disk-percentage?
{quote}
Fixed.(I'm presuming "min-free-space-per-disk-percentage" meant
"min-free-space-per-disk-mb")
{quote}
DirectoryCollection
checkDiskPercentageLimit() -> isDiskUsageUnderPercentageLimit() and
checkDiskSpaceLimit() -> isDiskFreeSpaceWithinLimit()
Please replace 'perc/Perc' etc with the more verbose
'percentage/Percentage' everywhere
{quote}
Fixed
{quote}
LocalDirsHandlerService
maxUsedSpacePerc -> maxUsableSpacePercentagePerDisk and minFreeSpaceMB ->
minFreeSpacePerDiskMB
{quote}
Fixed
{quote}
TestDirectoryCollection
The test using testDir.getTotalSpace() can be flaky depending on what other
things are running on the testing machine. Can we just mock the disk-usage here?
testCutoffSetters -> testDiskLimitsCutoffSetters,
{quote}
getTotalSpace() should be consistent no? According to the Java docs, it returns
the size of the partition which should be the same every run. The idea of the
test is to simulate a case where we don't allow any of the disk to be used.
> NM should allow users to specify max disk utilization for local disks
> ---------------------------------------------------------------------
>
> Key: YARN-1781
> URL: https://issues.apache.org/jira/browse/YARN-1781
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
> Reporter: Varun Vasudev
> Assignee: Varun Vasudev
> Attachments: apache-yarn-1781.0.patch, apache-yarn-1781.1.patch,
> apache-yarn-1781.2.patch, apache-yarn-1781.3.patch
>
>
> This is related to YARN-257(it's probably a sub task?). Currently, the NM
> does not detect full disks and allows full disks to be used by containers
> leading to repeated failures. YARN-257 deals with graceful handling of full
> disks. This ticket is only about detection of full disks by the disk health
> checkers.
> The NM should allow users to set a maximum disk utilization for local disks
> and mark disks as bad once they exceed that utilization. At the very least,
> the NM should at least detect full disks.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
