[
https://issues.apache.org/jira/browse/MAPREDUCE-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820992#comment-13820992
]
Devaraj K commented on MAPREDUCE-5622:
--------------------------------------
RM Log:
{code:xml}
yarn-hdpcrm-resourcemanager-ocean24.log.1:2013-11-11 13:58:36,738 DEBUG
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp:
allocate: applicationAttemptId=appattempt_1383828930654_0030_000001
container=container_1383828930654_0030_01_000073 host=ocean21 type=NODE_LOCAL
yarn-hdpcrm-resourcemanager-ocean24.log.1:2013-11-11 13:58:36,743 DEBUG
org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp:
allocate: applicationAttemptId=appattempt_1383828930654_0030_000001
container=container_1383828930654_0030_01_000074 host=ocean21 type=NODE_LOCAL
{code}
\\
NM Log:
{code:xml}
2013-11-11 13:57:38,904 INFO [RMCommunicator Allocator]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container
(Container: [ContainerId: container_1383828930654_0030_01_000073, NodeId:
ocean21:23240, NodeHttpAddress: ocean21:23239, Resource: <memory:6144,
vCores:1>, Priority: 20, State: NEW, Token: null, Status: container_id {,
app_attempt_id {, application_id {, id: 30, cluster_timestamp: 1383828930654,
}, attemptId: 1, }, id: 73, }, state: C_NEW, ]) to task
attempt_1383828930654_0030_m_000061_0 on node ocean21:23240
2013-11-11 13:57:38,904 DEBUG [RMCommunicator Allocator]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned based on
rack match /default-rack
2013-11-11 13:57:38,905 INFO [RMCommunicator Allocator]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container
(Container: [ContainerId: container_1383828930654_0030_01_000074, NodeId:
ocean21:23240, NodeHttpAddress: ocean21:23239, Resource: <memory:6144,
vCores:1>, Priority: 20, State: NEW, Token: null, Status: container_id {,
app_attempt_id {, application_id {, id: 30, cluster_timestamp: 1383828930654,
}, attemptId: 1, }, id: 74, }, state: C_NEW, ]) to task
attempt_1383828930654_0030_m_000063_0 on node ocean21:23240
2013-11-11 13:57:38,905 DEBUG [RMCommunicator Allocator]
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned based on
rack match /default-rack
{code}
\\
Here RM says allocating the container as NODE_LOCAL but AM is assigning based
on rack match.
> MRAppMaster doesn't assign all allocated NODE_LOCAL containers to node-local
> maps
> ---------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5622
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5622
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 2.2.0
> Reporter: Devaraj K
>
> MRAppMaster will request containers for all the splits to launch map tasks,
> RM will give Node Local containers for all these if available. When the RM
> gives all containers as Node Local, MR AM may assign these NODE-LOCAL
> containers to non-local maps.
> \\
> \\
> |node1|split1|split2| |split4|
> |node2| |split2|split3| |
> |node3|split1|split2|split3|split4|
> |node4|split1| |split3|split4|
> \\
> Consider this instance, assume RM has given one NODE LOCAL container on each
> node to process all the splits as local maps. While assigning, if the AM
> gives node1-container for split1, node2-container for split3, node3-container
> for split3 and node4-container can be given to only split2 which is not
> local.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
