[
https://issues.apache.org/jira/browse/YARN-9944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963909#comment-16963909
]
Xianghao Lu edited comment on YARN-9944 at 10/31/19 11:51 AM:
--------------------------------------------------------------
log before fix
{quote}
2019-10-31 16:50:28,103 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSPreemptionThread:
Preempting container container_e01_1572511557192_0003_01_000007 from queue root
.yarn
2019-10-31 16:50:34,944 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:{color:red}
container_e01_1572511557192_0003_01_000007 Container Transitioned from RUNNING
to COM{color}
PLETED
2019-10-31 16:50:34,944 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=yarn
OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS
APPID=applica
tion_1572511557192_0003
CONTAINERID=container_e01_1572511557192_0003_01_000007 RESOURCE=<memory:2048,
vCores:1>
...
...
2019-10-31 16:50:43,103 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSPreemptionThread:
Killing container container_e01_1572511557192_0003_01_000007
2019-10-31 16:50:43,103 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt:
{color:red}Additional complete request on completed container
container_e01_1572511557192_0003_0{color}
1_000007
{quote}
log after fix
{quote}
2019-10-31 19:21:27,291 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSPreemptionThread:
Preempting container container_e01_1572520847877_0001_01_000003 from queue root
.yarn
2019-10-31 19:21:37,656 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
{color:red}container_e01_1572520847877_0001_01_000003 Container Transitioned
from RUNNING to COMPLETED{color}
2019-10-31 19:21:37,656 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=yarn
OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS
APPID=application_1572520847877_0001
CONTAINERID=container_e01_1572520847877_0001_01_000003 RESOURCE=<memory:2048,
vCores:1>
...
...
2019-10-31 19:21:42,292 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSPreemptionThread:
{color:red}Skipping killing container
container_e01_1572520847877_0001_01_000003 for preemption{color}
{quote}
was (Author: luxianghao):
log before fix
{code:java}
2019-10-31 16:50:28,103 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSPreemptionThread:
Preempting container container_e01_1572511557192_0003_01_000007 from queue root
.yarn
2019-10-31 16:50:34,944 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:{color:red}
container_e01_1572511557192_0003_01_000007 Container Transitioned from RUNNING
to COM{color}
PLETED
2019-10-31 16:50:34,944 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=yarn
OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS
APPID=applica
tion_1572511557192_0003
CONTAINERID=container_e01_1572511557192_0003_01_000007 RESOURCE=<memory:2048,
vCores:1>
...
...
2019-10-31 16:50:43,103 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSPreemptionThread:
Killing container container_e01_1572511557192_0003_01_000007
2019-10-31 16:50:43,103 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt:
{color:red}Additional complete request on completed container
container_e01_1572511557192_0003_0{color}
1_000007
{code}
log after fix
{code:java}
2019-10-31 19:21:27,291 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSPreemptionThread:
Preempting container container_e01_1572520847877_0001_01_000003 from queue root
.yarn
2019-10-31 19:21:37,656 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
{color:red}container_e01_1572520847877_0001_01_000003 Container Transitioned
from RUNNING to COMPLETED{color}
2019-10-31 19:21:37,656 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=yarn
OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS
APPID=application_1572520847877_0001
CONTAINERID=container_e01_1572520847877_0001_01_000003 RESOURCE=<memory:2048,
vCores:1>
...
...
2019-10-31 19:21:42,292 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSPreemptionThread:
{color:red}Skipping killing container
container_e01_1572520847877_0001_01_000003 for preemption{color}
{code}
> Don't need to kill container which already released in FSPreemtion thread
> -------------------------------------------------------------------------
>
> Key: YARN-9944
> URL: https://issues.apache.org/jira/browse/YARN-9944
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Affects Versions: 2.9.1
> Reporter: Xianghao Lu
> Assignee: Xianghao Lu
> Priority: Major
> Fix For: 2.9.1
>
> Attachments: YARN-9944-branch-2.9.1.001.patch
>
>
> Killing container for preemption is a timed task, container may become
> completed during the waiting period.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
