[
https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14515853#comment-14515853
]
Wangda Tan commented on MAPREDUCE-6304:
---------------------------------------
bq. Well my view should be to support them as i feel in some cases we might
feel like mapper can run in any node but may be reducer requires more mem so
may be clients might require it to be run in high mem nodes (may be some
constraint on mem for the nodes) or some other node constraints.
That's correct, so you agree to add mapper/reducer node-label-expression?
bq. Well, IIUC then ApplicationSubmissionContext.getNodeLabelExpression() is
for mapper/reducer and AM's node label expression is set in
ApplicationSubmissionContext.getAMContainerResourceRequest. I could cross
verify the same with code in ApplicationMasterService (line number 495) &
RMAppManager (line number 378) and so did not get what you meant by the above
statement, correct me if my understanding is wrong.
What I meant is:
When job.label = x, and don't set am/mapper/reducer label, all containers
should get allocated on x
When job.label = x and am.label = y, don't set mapper/reducer label, am will
get allocated on y (overwritten x) but mapper/reducer should get allocated on x.
When job.label = x, am.label=y, mapper.label=z, am will get allocated on y,
mapper will get allocated on z (overwrite x), and reducer get allocated on x.
This should be existing behavior. And mapper/reducer's label should be added to
ResourceRequest (may need modify {{RMContainerRequestor}}) instead of
AplicationSubmissionContext.
> Specifying node labels when submitting MR jobs
> ----------------------------------------------
>
> Key: MAPREDUCE-6304
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6304
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Jian Fang
> Assignee: Naganarasimha G R
> Fix For: 2.8.0
>
> Attachments: MAPREDUCE-6304.20150410-1.patch,
> MAPREDUCE-6304.20150411-1.patch
>
>
> Per the discussion on YARN-796, we need a mechanism in MAPREDUCE to specify
> node labels when submitting MR jobs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
