[
https://issues.apache.org/jira/browse/MAPREDUCE-5833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967356#comment-13967356
]
Zhijie Shen commented on MAPREDUCE-5833:
----------------------------------------
The second race condition happens at
{code}
amDispatcher.await();
// Wait till all map-attempts request for containers
for (Task t : job.getTasks().values()) {
if (t.getType() == TaskType.MAP) {
mrApp.waitForInternalState((TaskAttemptImpl) t.getAttempts().values()
.iterator().next(), TaskAttemptStateInternal.UNASSIGNED);
}
}
amDispatcher.await();
allocator.schedule();
rmDispatcher.await();
amNodeManager.nodeHeartbeat(true);
rmDispatcher.await();
allocator.schedule();
rmDispatcher.await();
{code}
Though all the attempts have reached UNASSIGNED, CONTAINER_REQ may still not be
handled by RMContainerAllocator, because the events *again* are handled on a
separate thread of RMContainerAllocator. Therefore, it's possible not all the
resource requests have been added before doing heartbeat with RM, such that the
task cannot move on to RUNNING later. See the following log of a failure case:
{code}
2014-04-10 02:44:51,439 DEBUG [Thread-230] rm.RMContainerRequestor
(RMContainerRequestor.java:addResourceRequest(360)) - addResourceRequest:
applicationId=1 priority=20 resourceName=* numContainers=1 #asks=1
2014-04-10 02:44:51,439 DEBUG [AsyncDispatcher event handler]
event.AsyncDispatcher (AsyncDispatcher.java:dispatch(164)) - Dispatching the
event org.apache.hadoop.mapreduce.jobhistory.JobHistoryEvent.EventType:
NORMALIZED_RESOURCE
TaskAttempt Internal State is : UNASSIGNED
TaskAttempt Internal State is : UNASSIGNED
TaskAttempt Internal State is : UNASSIGNED
TaskAttempt Internal State is : UNASSIGNED
TaskAttempt Internal State is : UNASSIGNED
2014-04-10 02:44:51,439 DEBUG [Thread-230] rm.RMContainerRequestor
(RMContainerRequestor.java:addResourceRequest(360)) - addResourceRequest:
applicationId=1 priority=20 resourceName=* numContainers=2 #asks=1
TaskAttempt Internal State is : UNASSIGNED
TaskAttempt Internal State is : UNASSIGNED
TaskAttempt Internal State is : UNASSIGNED
TaskAttempt Internal State is : UNASSIGNED
2014-04-10 02:44:51,440 DEBUG [Thread-230] rm.RMContainerRequestor
(RMContainerRequestor.java:addResourceRequest(360)) - addResourceRequest:
applicationId=1 priority=20 resourceName=* numContainers=3 #asks=1
TaskAttempt Internal State is : UNASSIGNED
2014-04-10 02:44:51,441 DEBUG [Thread-230] rm.RMContainerRequestor
(RMContainerRequestor.java:addResourceRequest(360)) - addResourceRequest:
applicationId=1 priority=20 resourceName=* numContainers=4 #asks=1
2014-04-10 02:44:51,441 INFO [main] rm.RMContainerAllocator
(RMContainerAllocator.java:log(1223)) - Before Scheduling: PendingReds:0
ScheduledMaps:4 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0
CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2014-04-10 02:44:51,442 DEBUG [AsyncDispatcher event handler]
event.AsyncDispatcher (AsyncDispatcher.java:dispatch(164)) - Dispatching the
event
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptStatusupdateEvent.EventType:
STATUS_UPDATE
2014-04-10 02:44:51,442 DEBUG [AsyncDispatcher event handler]
attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(647)) - Processing event
for appattempt_1397123055648_0001_000001 of type STATUS_UPDATE
2014-04-10 02:44:51,442 DEBUG [main] fifo.FifoScheduler
(FifoScheduler.java:allocate(308)) - allocate: pre-update
applicationId=appattempt_1397123055648_0001_000001
application=org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp@7544a229
2014-04-10 02:44:51,442 DEBUG [main] scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(328)) - showRequests:
application=application_1397123055648_0001 headRoom=<memory:10240, vCores:10>
currentConsumption=1024
2014-04-10 02:44:51,442 DEBUG [main] scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(332)) - showRequests:
application=application_1397123055648_0001 request={Priority: 0, Capability:
<memory:1024, vCores:1>, # Containers: 0, Location: *, Relax Locality: true}
2014-04-10 02:44:51,442 DEBUG [main] scheduler.AppSchedulingInfo
(AppSchedulingInfo.java:updateResourceRequests(135)) - update:
application=application_1397123055648_0001 request={Priority: 20, Capability:
<memory:1024, vCores:1>, # Containers: 4, Location: *, Relax Locality: true}
2014-04-10 02:44:51,444 DEBUG [main] scheduler.ActiveUsersManager
(ActiveUsersManager.java:activateApplication(68)) - User
appattempt_1397123055648_0001_000001 added to activeUsers, currently: 1
2014-04-10 02:44:51,444 DEBUG [main] fifo.FifoScheduler
(FifoScheduler.java:allocate(316)) - allocate: post-update
applicationId=appattempt_1397123055648_0001_000001
application=org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp@7544a229
2014-04-10 02:44:51,444 DEBUG [main] scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(328)) - showRequests:
application=application_1397123055648_0001 headRoom=<memory:10240, vCores:10>
currentConsumption=1024
2014-04-10 02:44:51,444 DEBUG [main] scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(332)) - showRequests:
application=application_1397123055648_0001 request={Priority: 0, Capability:
<memory:1024, vCores:1>, # Containers: 0, Location: *, Relax Locality: true}
2014-04-10 02:44:51,444 DEBUG [main] scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(328)) - showRequests:
application=application_1397123055648_0001 headRoom=<memory:10240, vCores:10>
currentConsumption=1024
2014-04-10 02:44:51,445 DEBUG [main] scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(332)) - showRequests:
application=application_1397123055648_0001 request={Priority: 20, Capability:
<memory:1024, vCores:1>, # Containers: 4, Location: *, Relax Locality: true}
2014-04-10 02:44:51,445 DEBUG [main] fifo.FifoScheduler
(FifoScheduler.java:allocate(321)) - allocate:
applicationId=appattempt_1397123055648_0001_000001 #ask=1
2014-04-10 02:44:51,445 INFO [main] rm.RMContainerRequestor
(RMContainerRequestor.java:makeRemoteRequest(172)) - getResources() for
application_1397123055648_0001: ask=1 release= 0 newContainers=0
finishedContainers=0 resourcelimit=<memory:10240, vCores:10> knownNMs=1
2014-04-10 02:44:51,445 DEBUG [main] rm.RMContainerAllocator
(RMContainerAllocator.java:getResources(603)) - headroom=10240
2014-04-10 02:44:51,446 DEBUG [AsyncDispatcher event handler]
event.AsyncDispatcher (AsyncDispatcher.java:dispatch(164)) - Dispatching the
event
org.apache.hadoop.mapreduce.v2.app.job.event.JobUpdatedNodesEvent.EventType:
JOB_UPDATED_NODES
2014-04-10 02:44:51,446 DEBUG [AsyncDispatcher event handler] impl.JobImpl
(JobImpl.java:handle(969)) - Processing job_1397123055648_0001 of type
JOB_UPDATED_NODES
2014-04-10 02:44:51,446 DEBUG [Thread-230] rm.RMContainerRequestor
(RMContainerRequestor.java:addResourceRequest(360)) - addResourceRequest:
applicationId=1 priority=20 resourceName=* numContainers=5 #asks=1
2014-04-10 02:44:51,447 DEBUG [AsyncDispatcher event handler]
event.AsyncDispatcher (AsyncDispatcher.java:dispatch(164)) - Dispatching the
event
org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeStatusEvent.EventType:
STATUS_UPDATE
2014-04-10 02:44:51,447 DEBUG [AsyncDispatcher event handler] rmnode.RMNodeImpl
(RMNodeImpl.java:handle(373)) - Processing amNM:1234 of type STATUS_UPDATE
2014-04-10 02:44:51,447 DEBUG [AsyncDispatcher event handler]
event.AsyncDispatcher (AsyncDispatcher.java:dispatch(164)) - Dispatching the
event
org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeUpdateSchedulerEvent.EventType:
NODE_UPDATE
2014-04-10 02:44:51,448 DEBUG [AsyncDispatcher event handler]
fifo.FifoScheduler (FifoScheduler.java:nodeUpdate(711)) - Node heartbeat
amNM:1234 available resource = <memory:10240, vCores:10>
2014-04-10 02:44:51,448 DEBUG [AsyncDispatcher event handler]
fifo.FifoScheduler (FifoScheduler.java:assignContainers(452)) -
assignContainers: node=amNM:1234 #applications=1
2014-04-10 02:44:51,448 DEBUG [AsyncDispatcher event handler]
fifo.FifoScheduler (FifoScheduler.java:assignContainers(461)) -
pre-assignContainers
2014-04-10 02:44:51,448 DEBUG [AsyncDispatcher event handler]
scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(328)) - showRequests:
application=application_1397123055648_0001 headRoom=<memory:10240, vCores:10>
currentConsumption=1024
2014-04-10 02:44:51,448 DEBUG [AsyncDispatcher event handler]
scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(332)) - showRequests:
application=application_1397123055648_0001 request={Priority: 0, Capability:
<memory:1024, vCores:1>, # Containers: 0, Location: *, Relax Locality: true}
2014-04-10 02:44:51,448 DEBUG [AsyncDispatcher event handler]
scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(328)) - showRequests:
application=application_1397123055648_0001 headRoom=<memory:10240, vCores:10>
currentConsumption=1024
2014-04-10 02:44:51,449 DEBUG [AsyncDispatcher event handler]
scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(332)) - showRequests:
application=application_1397123055648_0001 request={Priority: 20, Capability:
<memory:1024, vCores:1>, # Containers: 4, Location: *, Relax Locality: true}
2014-04-10 02:44:51,449 DEBUG [AsyncDispatcher event handler]
fifo.FifoScheduler (FifoScheduler.java:assignContainer(634)) -
assignContainers: node=amNM:1234 application=1 priority=20
assignableContainers=4 request={Priority: 20, Capability: <memory:1024,
vCores:1>, # Containers: 4, Location: *, Relax Locality: true} type=OFF_SWITCH
2014-04-10 02:44:51,449 DEBUG [AsyncDispatcher event handler]
scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(266)) - allocate:
applicationId=application_1397123055648_0001
container=container_1397123055648_0001_01_000002 host=amNM:1234
user=appattempt_1397123055648_0001_000001 resource=<memory:1024, vCores:1>
2014-04-10 02:44:51,449 DEBUG [AsyncDispatcher event handler]
rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(309)) - Processing
container_1397123055648_0001_01_000002 of type START
2014-04-10 02:44:51,450 INFO [AsyncDispatcher event handler]
rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(321)) -
container_1397123055648_0001_01_000002 Container Transitioned from NEW to
ALLOCATED
2014-04-10 02:44:51,450 DEBUG [AsyncDispatcher event handler]
fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(140)) - allocate:
applicationAttemptId=appattempt_1397123055648_0001_000001
container=container_1397123055648_0001_01_000002 host=amNM type=OFF_SWITCH
2014-04-10 02:44:51,450 INFO [AsyncDispatcher event handler]
resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(100)) -
USER=appattempt_1397123055648_0001_000001 OPERATION=AM Allocated Container
TARGET=SchedulerApp RESULT=SUCCESS
APPID=application_1397123055648_0001
CONTAINERID=container_1397123055648_0001_01_000002
2014-04-10 02:44:51,451 INFO [AsyncDispatcher event handler]
fica.FiCaSchedulerNode (FiCaSchedulerNode.java:allocateContainer(116)) -
Assigned container container_1397123055648_0001_01_000002 of capacity
<memory:1024, vCores:1> on host amNM:1234, which currently has 2 containers,
<memory:2048, vCores:2> used and <memory:9216, vCores:9> available
2014-04-10 02:44:51,451 DEBUG [AsyncDispatcher event handler]
scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(266)) - allocate:
applicationId=application_1397123055648_0001
container=container_1397123055648_0001_01_000003 host=amNM:1234
user=appattempt_1397123055648_0001_000001 resource=<memory:1024, vCores:1>
2014-04-10 02:44:51,452 DEBUG [AsyncDispatcher event handler]
rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(309)) - Processing
container_1397123055648_0001_01_000003 of type START
2014-04-10 02:44:51,452 INFO [AsyncDispatcher event handler]
rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(321)) -
container_1397123055648_0001_01_000003 Container Transitioned from NEW to
ALLOCATED
2014-04-10 02:44:51,452 DEBUG [AsyncDispatcher event handler]
fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(140)) - allocate:
applicationAttemptId=appattempt_1397123055648_0001_000001
container=container_1397123055648_0001_01_000003 host=amNM type=OFF_SWITCH
2014-04-10 02:44:51,452 INFO [AsyncDispatcher event handler]
resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(100)) -
USER=appattempt_1397123055648_0001_000001 OPERATION=AM Allocated Container
TARGET=SchedulerApp RESULT=SUCCESS
APPID=application_1397123055648_0001
CONTAINERID=container_1397123055648_0001_01_000003
2014-04-10 02:44:51,452 INFO [AsyncDispatcher event handler]
fica.FiCaSchedulerNode (FiCaSchedulerNode.java:allocateContainer(116)) -
Assigned container container_1397123055648_0001_01_000003 of capacity
<memory:1024, vCores:1> on host amNM:1234, which currently has 3 containers,
<memory:3072, vCores:3> used and <memory:8192, vCores:8> available
2014-04-10 02:44:51,453 DEBUG [AsyncDispatcher event handler]
scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(266)) - allocate:
applicationId=application_1397123055648_0001
container=container_1397123055648_0001_01_000004 host=amNM:1234
user=appattempt_1397123055648_0001_000001 resource=<memory:1024, vCores:1>
2014-04-10 02:44:51,453 DEBUG [AsyncDispatcher event handler]
rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(309)) - Processing
container_1397123055648_0001_01_000004 of type START
2014-04-10 02:44:51,453 INFO [AsyncDispatcher event handler]
rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(321)) -
container_1397123055648_0001_01_000004 Container Transitioned from NEW to
ALLOCATED
2014-04-10 02:44:51,453 DEBUG [AsyncDispatcher event handler]
fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(140)) - allocate:
applicationAttemptId=appattempt_1397123055648_0001_000001
container=container_1397123055648_0001_01_000004 host=amNM type=OFF_SWITCH
2014-04-10 02:44:51,453 INFO [AsyncDispatcher event handler]
resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(100)) -
USER=appattempt_1397123055648_0001_000001 OPERATION=AM Allocated Container
TARGET=SchedulerApp RESULT=SUCCESS
APPID=application_1397123055648_0001
CONTAINERID=container_1397123055648_0001_01_000004
2014-04-10 02:44:51,453 INFO [AsyncDispatcher event handler]
fica.FiCaSchedulerNode (FiCaSchedulerNode.java:allocateContainer(116)) -
Assigned container container_1397123055648_0001_01_000004 of capacity
<memory:1024, vCores:1> on host amNM:1234, which currently has 4 containers,
<memory:4096, vCores:4> used and <memory:7168, vCores:7> available
2014-04-10 02:44:51,453 DEBUG [AsyncDispatcher event handler]
scheduler.ActiveUsersManager
(ActiveUsersManager.java:deactivateApplication(94)) - User
appattempt_1397123055648_0001_000001 removed from activeUsers, currently: 0
2014-04-10 02:44:51,453 DEBUG [AsyncDispatcher event handler]
scheduler.AppSchedulingInfo (AppSchedulingInfo.java:allocate(266)) - allocate:
applicationId=application_1397123055648_0001
container=container_1397123055648_0001_01_000005 host=amNM:1234
user=appattempt_1397123055648_0001_000001 resource=<memory:1024, vCores:1>
2014-04-10 02:44:51,453 DEBUG [AsyncDispatcher event handler]
rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(309)) - Processing
container_1397123055648_0001_01_000005 of type START
2014-04-10 02:44:51,453 INFO [AsyncDispatcher event handler]
rmcontainer.RMContainerImpl (RMContainerImpl.java:handle(321)) -
container_1397123055648_0001_01_000005 Container Transitioned from NEW to
ALLOCATED
2014-04-10 02:44:51,455 DEBUG [AsyncDispatcher event handler]
fica.FiCaSchedulerApp (FiCaSchedulerApp.java:allocate(140)) - allocate:
applicationAttemptId=appattempt_1397123055648_0001_000001
container=container_1397123055648_0001_01_000005 host=amNM type=OFF_SWITCH
2014-04-10 02:44:51,455 INFO [AsyncDispatcher event handler]
resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(100)) -
USER=appattempt_1397123055648_0001_000001 OPERATION=AM Allocated Container
TARGET=SchedulerApp RESULT=SUCCESS
APPID=application_1397123055648_0001
CONTAINERID=container_1397123055648_0001_01_000005
2014-04-10 02:44:51,455 INFO [AsyncDispatcher event handler]
fica.FiCaSchedulerNode (FiCaSchedulerNode.java:allocateContainer(116)) -
Assigned container container_1397123055648_0001_01_000005 of capacity
<memory:1024, vCores:1> on host amNM:1234, which currently has 5 containers,
<memory:5120, vCores:5> used and <memory:6144, vCores:6> available
2014-04-10 02:44:51,455 DEBUG [AsyncDispatcher event handler]
fifo.FifoScheduler (FifoScheduler.java:assignContainersOnNode(556)) -
assignContainersOnNode: node=amNM:1234 application=1 priority=20 #assigned=4
2014-04-10 02:44:51,455 DEBUG [AsyncDispatcher event handler]
fifo.FifoScheduler (FifoScheduler.java:assignContainers(485)) -
post-assignContainers
2014-04-10 02:44:51,456 DEBUG [AsyncDispatcher event handler]
scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(328)) - showRequests:
application=application_1397123055648_0001 headRoom=<memory:10240, vCores:10>
currentConsumption=5120
2014-04-10 02:44:51,456 DEBUG [AsyncDispatcher event handler]
scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(332)) - showRequests:
application=application_1397123055648_0001 request={Priority: 0, Capability:
<memory:1024, vCores:1>, # Containers: 0, Location: *, Relax Locality: true}
2014-04-10 02:44:51,456 DEBUG [AsyncDispatcher event handler]
scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(328)) - showRequests:
application=application_1397123055648_0001 headRoom=<memory:10240, vCores:10>
currentConsumption=5120
2014-04-10 02:44:51,456 DEBUG [AsyncDispatcher event handler]
scheduler.SchedulerApplicationAttempt
(SchedulerApplicationAttempt.java:showRequests(332)) - showRequests:
application=application_1397123055648_0001 request={Priority: 20, Capability:
<memory:1024, vCores:1>, # Containers: 0, Location: *, Relax Locality: true}
2014-04-10 02:44:51,456 DEBUG [AsyncDispatcher event handler]
fifo.FifoScheduler (FifoScheduler.java:nodeUpdate(716)) - Node after allocation
amNM:1234 resource = <memory:6144, vCores:6>
2014-04-10 02:44:51,457 DEBUG [AsyncDispatcher event handler]
event.AsyncDispatcher (AsyncDispatcher.java:dispatch(164)) - Dispatching the
event
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerAllocatedEvent.EventType:
CONTAINER_ALLOCATED
2014-04-10 02:44:51,457 DEBUG [AsyncDispatcher event handler]
attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(647)) - Processing event
for appattempt_1397123055648_0001_000001 of type CONTAINER_ALLOCATED
2014-04-10 02:44:51,457 DEBUG [AsyncDispatcher event handler]
event.AsyncDispatcher (AsyncDispatcher.java:dispatch(164)) - Dispatching the
event
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerAllocatedEvent.EventType:
CONTAINER_ALLOCATED
2014-04-10 02:44:51,457 DEBUG [AsyncDispatcher event handler]
attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(647)) - Processing event
for appattempt_1397123055648_0001_000001 of type CONTAINER_ALLOCATED
2014-04-10 02:44:51,457 DEBUG [AsyncDispatcher event handler]
event.AsyncDispatcher (AsyncDispatcher.java:dispatch(164)) - Dispatching the
event
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerAllocatedEvent.EventType:
CONTAINER_ALLOCATED
2014-04-10 02:44:51,457 DEBUG [AsyncDispatcher event handler]
attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(647)) - Processing event
for appattempt_1397123055648_0001_000001 of type CONTAINER_ALLOCATED
2014-04-10 02:44:51,458 DEBUG [AsyncDispatcher event handler]
event.AsyncDispatcher (AsyncDispatcher.java:dispatch(164)) - Dispatching the
event
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.event.RMAppAttemptContainerAllocatedEvent.EventType:
CONTAINER_ALLOCATED
2014-04-10 02:44:51,458 DEBUG [AsyncDispatcher event handler]
attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(647)) - Processing event
for appattempt_1397123055648_0001_000001 of type CONTAINER_ALLOCATED
2014-04-10 02:44:51,458 DEBUG [Thread-230] rm.RMContainerRequestor
(RMContainerRequestor.java:addResourceRequest(360)) - addResourceRequest:
applicationId=1 priority=20 resourceName=* numContainers=6 #asks=1
2014-04-10 02:44:51,459 DEBUG [Thread-230] rm.RMContainerRequestor
(RMContainerRequestor.java:addResourceRequest(360)) - addResourceRequest:
applicationId=1 priority=20 resourceName=* numContainers=7 #asks=1
2014-04-10 02:44:51,459 DEBUG [Thread-230] rm.RMContainerRequestor
(RMContainerRequestor.java:addResourceRequest(360)) - addResourceRequest:
applicationId=1 priority=20 resourceName=* numContainers=8 #asks=1
2014-04-10 02:44:51,459 DEBUG [Thread-230] rm.RMContainerRequestor
(RMContainerRequestor.java:addResourceRequest(360)) - addResourceRequest:
applicationId=1 priority=20 resourceName=* numContainers=9 #asks=1
2014-04-10 02:44:51,460 DEBUG [Thread-230] rm.RMContainerRequestor
(RMContainerRequestor.java:addResourceRequest(360)) - addResourceRequest:
applicationId=1 priority=20 resourceName=* numContainers=10 #asks=1
{code}
Heartbeat happened between adding resource requests.
> TestRMContainerAllocator fails ocassionally
> -------------------------------------------
>
> Key: MAPREDUCE-5833
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5833
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Zhijie Shen
> Assignee: Zhijie Shen
>
> testReportedAppProgress and testReportedAppProgressWithOnlyMaps have race
> conditions.
> {code}
> Stacktrace
> java.util.NoSuchElementException: null
> at java.util.Collections$EmptyIterator.next(Collections.java:2998)
> at
> org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator.testReportedAppProgress(TestRMContainerAllocator.java:535)
> {code}
> {code}
> Error Message
> Task state is not correct (timedout) expected:<RUNNING> but was:<SCHEDULED>
> Stacktrace
> junit.framework.AssertionFailedError: Task state is not correct (timedout)
> expected:<RUNNING> but was:<SCHEDULED>
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at org.apache.hadoop.mapreduce.v2.app.MRApp.waitForState(MRApp.java:393)
> at
> org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator.testReportedAppProgressWithOnlyMaps(TestRMContainerAllocator.java:700)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
