[
https://issues.apache.org/jira/browse/MAPREDUCE-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092874#comment-13092874
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2652:
----------------------------------------------------
hadoop-mr-client modules seems to have been renamed to hadoop-mapreduce-client,
the patch needs regeneration only w.r.t that.
Proceeding with review after patching the patch. It mostly looks clean, great
job, I have only a few comments:
TaskAttemptEvent.java
- "This is only valid if the type of the event is TA_CONTAINER_LAUNCHED": This
clearly seems to tell us (:)) that we need a subclass of TaskAttemptEvent -
TaskAttemptContainerLaunchedEvent. We can put the shuffle-port as a payload of
that event. Once you do that, the rest of the changes to TaskAttempt.java can
be dropped.
TaskAttemptImpl.java
- TODO in public TaskAttemptReport getReport(): We don't need it here,
instead we need this information in TaskCompletionEvents returned to client.
Can you get this returned via {{MRClientService.getTaskCompletionEvents()}}?
ContainersLauncherImpl:
- We should probably not have the default value to be 8080 here, if the
response to ContainerLaunch doesn't have the port, we should fail the task then
and there itself.
Integration test:
- To test this patch's integration, you should probably just set this port to
a non-default value say 8081 in TestMRJobs before starting MiniMRYarnCluster.
Either that or change MiniMRYarnCluster itself.
General: Please wrap around lines longer than 80 characters.
> MR-279: Cannot run multiple NMs on a single node
> -------------------------------------------------
>
> Key: MAPREDUCE-2652
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2652
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2652-trunk-v3.patch, MR-2652-v1.txt, MR-2652-v2.txt
>
>
> Currently in MR-279 the Auxiliary services, like ShuffleHandler, have no way
> to communicate information back to the applications. Because of this the Map
> Reduce Application Master has hardcoded in a port of 8080 for shuffle. This
> prevents the configuration "mapreduce.shuffle.port" form ever being set to
> anything but 8080. The code should be updated to allow this information to
> be returned to the application master. Also the data needs to be persisted
> to the task log so that on restart the data is not lost.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
