[ https://issues.apache.org/jira/browse/YARN-3415 ]
mumu deleted comment on YARN-3415: ---------------------------- was (Author: JIRAUSER285934): [~zxu] Hello, I am using version 2.7.2 and after merging this patch, I found that the issue did not occur. However, I found that AM Used Resources is greater than AM Max Resources and tasks can still be submitted. > Non-AM containers can be counted towards amResourceUsage of a Fair Scheduler > queue > ---------------------------------------------------------------------------------- > > Key: YARN-3415 > URL: https://issues.apache.org/jira/browse/YARN-3415 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler > Affects Versions: 2.6.0 > Reporter: Rohit Agarwal > Assignee: Zhihai Xu > Priority: Critical > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: YARN-3415.000.patch, YARN-3415.001.patch, > YARN-3415.002.patch > > > We encountered this problem while running a spark cluster. The > amResourceUsage for a queue became artificially high and then the cluster got > deadlocked because the maxAMShare constrain kicked in and no new AM got > admitted to the cluster. > I have described the problem in detail here: > https://github.com/apache/spark/pull/5233#issuecomment-87160289 > In summary - the condition for adding the container's memory towards > amResourceUsage is fragile. It depends on the number of live containers > belonging to the app. We saw that the spark AM went down without explicitly > releasing its requested containers and then one of those containers memory > was counted towards amResource. > cc - [~sandyr] -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org