[jira] [Commented] (YARN-10617) Fifo and Fair intra-queue preemption goes on indefinitely when apps are in pending state due to max AM limit reached
[
https://issues.apache.org/jira/browse/YARN-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286248#comment-17286248
]
Hadoop QA commented on YARN-10617:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
15s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} yetus {color} | {color:red} 0m 7s{color}
| {color:red}{color} | {color:red} Unprocessed flag(s):
--findbugs-strict-precheck {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/629/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10617 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13020461/YARN-10617.0001.patch
|
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/629/console |
| versions | git=2.25.1 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Fifo and Fair intra-queue preemption goes on indefinitely when apps are in
> pending state due to max AM limit reached
>
>
> Key: YARN-10617
> URL: https://issues.apache.org/jira/browse/YARN-10617
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
>Affects Versions: 3.1.1
>Reporter: VADAGA ANANYO RAO
>Assignee: VADAGA ANANYO RAO
>Priority: Major
> Attachments: YARN-10617.0001.patch
>
>
> This case occurs when:
> 1. an application gets submitted in a cluster running at max-AM limit.
> 2. The new job requests AM resource. So it has 1 pending request.
> 3. To fulfil this request, the preemption logic preempts 1 resource from a
> running app.
> 4. Because the cluster is at max-AM limit, the scheduler re-assigns the
> preempted container back to the running app.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10617) Fifo and Fair intra-queue preemption goes on indefinitely when apps are in pending state due to max AM limit reached
[
https://issues.apache.org/jira/browse/YARN-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285658#comment-17285658
]
Hadoop QA commented on YARN-10617:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 30m
12s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} yetus {color} | {color:red} 0m 7s{color}
| {color:red}{color} | {color:red} Unprocessed flag(s):
--findbugs-strict-precheck {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/624/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10617 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13020461/YARN-10617.0001.patch
|
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/624/console |
| versions | git=2.25.1 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Fifo and Fair intra-queue preemption goes on indefinitely when apps are in
> pending state due to max AM limit reached
>
>
> Key: YARN-10617
> URL: https://issues.apache.org/jira/browse/YARN-10617
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
>Affects Versions: 3.1.1
>Reporter: VADAGA ANANYO RAO
>Assignee: VADAGA ANANYO RAO
>Priority: Major
> Attachments: YARN-10617.0001.patch
>
>
> This case occurs when:
> 1. an application gets submitted in a cluster running at max-AM limit.
> 2. The new job requests AM resource. So it has 1 pending request.
> 3. To fulfil this request, the preemption logic preempts 1 resource from a
> running app.
> 4. Because the cluster is at max-AM limit, the scheduler re-assigns the
> preempted container back to the running app.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10617) Fifo and Fair intra-queue preemption goes on indefinitely when apps are in pending state due to max AM limit reached
[ https://issues.apache.org/jira/browse/YARN-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281535#comment-17281535 ] Sunil G commented on YARN-10617: Hi [~ananyo_rao] Yes. In the preemption module, we get all apps from the scheduler. Hence some of the apps may be in a pending state which cant be scheduled (due to AM limit etc). So I think this is a quick fix. > Fifo and Fair intra-queue preemption goes on indefinitely when apps are in > pending state due to max AM limit reached > > > Key: YARN-10617 > URL: https://issues.apache.org/jira/browse/YARN-10617 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Affects Versions: 3.1.1 >Reporter: VADAGA ANANYO RAO >Assignee: VADAGA ANANYO RAO >Priority: Major > Attachments: YARN-10617.patch > > > This case occurs when: > 1. an application gets submitted in a cluster running at max-AM limit. > 2. The new job requests AM resource. So it has 1 pending request. > 3. To fulfil this request, the preemption logic preempts 1 resource from a > running app. > 4. Because the cluster is at max-AM limit, the scheduler re-assigns the > preempted container back to the running app. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10617) Fifo and Fair intra-queue preemption goes on indefinitely when apps are in pending state due to max AM limit reached
[ https://issues.apache.org/jira/browse/YARN-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17281513#comment-17281513 ] VADAGA ANANYO RAO commented on YARN-10617: -- Hi [~leftnoteasy] [~sunilg] , could you please review this jira? The fix is in proportional capacity preemption logic. Basically, instead of considering all apps for preemption, we only consider apps which are schedulable by the scheduling logic we are using. cc: [~epayne] Thanks. > Fifo and Fair intra-queue preemption goes on indefinitely when apps are in > pending state due to max AM limit reached > > > Key: YARN-10617 > URL: https://issues.apache.org/jira/browse/YARN-10617 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Affects Versions: 3.1.1 >Reporter: VADAGA ANANYO RAO >Assignee: VADAGA ANANYO RAO >Priority: Major > Attachments: YARN-10617.patch > > > This case occurs when: > 1. an application gets submitted in a cluster running at max-AM limit. > 2. The new job requests AM resource. So it has 1 pending request. > 3. To fulfil this request, the preemption logic preempts 1 resource from a > running app. > 4. Because the cluster is at max-AM limit, the scheduler re-assigns the > preempted container back to the running app. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
