[ https://issues.apache.org/jira/browse/YARN-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050539#comment-14050539 ]
Karthik Kambatla commented on YARN-2175: ---------------------------------------- I ll let Anubhav provide details on the particular instance we ran into this. bq. We should try to address the right individual problem with its solution before we put a band-aid that may still be useful for issues that we cannot just address directly if any. Having worked on several MR1 production issues, I see your point. I agree we should look into and address individual problems. That said, I also believe in failsafes to avoid bringing down a production cluster or failing a critical job altogether in the face of hardware issues. That gives us time to fix the individual issues correctly when we encounter them, instead of hurrying for a hot fix. > Container localization has no timeouts and tasks can be stuck there for a > long time > ----------------------------------------------------------------------------------- > > Key: YARN-2175 > URL: https://issues.apache.org/jira/browse/YARN-2175 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager > Affects Versions: 2.4.0 > Reporter: Anubhav Dhoot > Assignee: Anubhav Dhoot > > There are no timeouts that can be used to limit the time taken by various > container startup operations. Localization for example could take a long time > and there is no automated way to kill an task if its stuck in these states. > These may have nothing to do with the task itself and could be an issue > within the platform. > Ideally there should be configurable limits for various states within the > NodeManager to limit various states. The RM does not care about most of these > and its only between AM and the NM. We can start by making these global > configurable defaults and in future we can make it fancier by letting AM > override them in the start container request. > This jira will be used to limit localization time and we can open others if > we feel we need to limit other operations. -- This message was sent by Atlassian JIRA (v6.2#6252)