[jira] [Updated] (YARN-1781) NM should allow users to specify max disk utilization for local disks
[ https://issues.apache.org/jira/browse/YARN-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-1781: Attachment: apache-yarn-1781.4.patch Patch updated with yarn-default.xml additions and fix to YarnConfiguration. > NM should allow users to specify max disk utilization for local disks > - > > Key: YARN-1781 > URL: https://issues.apache.org/jira/browse/YARN-1781 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: apache-yarn-1781.0.patch, apache-yarn-1781.1.patch, > apache-yarn-1781.2.patch, apache-yarn-1781.3.patch, apache-yarn-1781.4.patch > > > This is related to YARN-257(it's probably a sub task?). Currently, the NM > does not detect full disks and allows full disks to be used by containers > leading to repeated failures. YARN-257 deals with graceful handling of full > disks. This ticket is only about detection of full disks by the disk health > checkers. > The NM should allow users to set a maximum disk utilization for local disks > and mark disks as bad once they exceed that utilization. At the very least, > the NM should at least detect full disks. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-1781) NM should allow users to specify max disk utilization for local disks
[ https://issues.apache.org/jira/browse/YARN-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-1781: Attachment: apache-yarn-1781.3.patch > NM should allow users to specify max disk utilization for local disks > - > > Key: YARN-1781 > URL: https://issues.apache.org/jira/browse/YARN-1781 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: apache-yarn-1781.0.patch, apache-yarn-1781.1.patch, > apache-yarn-1781.2.patch, apache-yarn-1781.3.patch > > > This is related to YARN-257(it's probably a sub task?). Currently, the NM > does not detect full disks and allows full disks to be used by containers > leading to repeated failures. YARN-257 deals with graceful handling of full > disks. This ticket is only about detection of full disks by the disk health > checkers. > The NM should allow users to set a maximum disk utilization for local disks > and mark disks as bad once they exceed that utilization. At the very least, > the NM should at least detect full disks. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-1781) NM should allow users to specify max disk utilization for local disks
[ https://issues.apache.org/jira/browse/YARN-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-1781: Attachment: apache-yarn-1781.2.patch Patch with code review comments incorporated. > NM should allow users to specify max disk utilization for local disks > - > > Key: YARN-1781 > URL: https://issues.apache.org/jira/browse/YARN-1781 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: apache-yarn-1781.0.patch, apache-yarn-1781.1.patch, > apache-yarn-1781.2.patch > > > This is related to YARN-257(it's probably a sub task?). Currently, the NM > does not detect full disks and allows full disks to be used by containers > leading to repeated failures. YARN-257 deals with graceful handling of full > disks. This ticket is only about detection of full disks by the disk health > checkers. > The NM should allow users to set a maximum disk utilization for local disks > and mark disks as bad once they exceed that utilization. At the very least, > the NM should at least detect full disks. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-1781) NM should allow users to specify max disk utilization for local disks
[ https://issues.apache.org/jira/browse/YARN-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-1781: Attachment: apache-yarn-1781.1.patch This patch adds support to allows users to specify a minimum amount of disk space that must be available for the disk to be marked as good. If the disk fails either the percentage check or the minimum free disk space check, the disk is marked as bad. > NM should allow users to specify max disk utilization for local disks > - > > Key: YARN-1781 > URL: https://issues.apache.org/jira/browse/YARN-1781 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: apache-yarn-1781.0.patch, apache-yarn-1781.1.patch > > > This is related to YARN-257(it's probably a sub task?). Currently, the NM > does not detect full disks and allows full disks to be used by containers > leading to repeated failures. YARN-257 deals with graceful handling of full > disks. This ticket is only about detection of full disks by the disk health > checkers. > The NM should allow users to set a maximum disk utilization for local disks > and mark disks as bad once they exceed that utilization. At the very least, > the NM should at least detect full disks. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-1781) NM should allow users to specify max disk utilization for local disks
[ https://issues.apache.org/jira/browse/YARN-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-1781: Attachment: apache-yarn-1781.0.patch > NM should allow users to specify max disk utilization for local disks > - > > Key: YARN-1781 > URL: https://issues.apache.org/jira/browse/YARN-1781 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev > Attachments: apache-yarn-1781.0.patch > > > This is related to YARN-257(it's probably a sub task?). Currently, the NM > does not detect full disks and allows full disks to be used by containers > leading to repeated failures. YARN-257 deals with graceful handling of full > disks. This ticket is only about detection of full disks by the disk health > checkers. > The NM should allow users to set a maximum disk utilization for local disks > and mark disks as bad once they exceed that utilization. At the very least, > the NM should at least detect full disks. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-1781) NM should allow users to specify max disk utilization for local disks
[ https://issues.apache.org/jira/browse/YARN-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-1781: -- Issue Type: Sub-task (was: Bug) Parent: YARN-257 > NM should allow users to specify max disk utilization for local disks > - > > Key: YARN-1781 > URL: https://issues.apache.org/jira/browse/YARN-1781 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Varun Vasudev >Assignee: Varun Vasudev > > This is related to YARN-257(it's probably a sub task?). Currently, the NM > does not detect full disks and allows full disks to be used by containers > leading to repeated failures. YARN-257 deals with graceful handling of full > disks. This ticket is only about detection of full disks by the disk health > checkers. > The NM should allow users to set a maximum disk utilization for local disks > and mark disks as bad once they exceed that utilization. At the very least, > the NM should at least detect full disks. -- This message was sent by Atlassian JIRA (v6.2#6252)
