[
https://issues.apache.org/jira/browse/MAPREDUCE-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104377#comment-13104377
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2983:
----------------------------------------------------
bq. Could we make this an easier experience for new users by having the MR AM
double check that the ShuffleService is configured at application submission
time?
This is a hack irrespective of whether we do this in the AM or the client.
Unfortunately even that is insufficient. Shuffle service is a configured
auxiliary service on the NodeManagers. Auxiliary services are designed to be,
well, auxiliary. For e.g. you can imagine only a subset of the cluster to have
mapreduce ShuffleService and the rest not to have it. So, the correct way to do
this is make the AM request containers on nodes which have the specified
auxiliary service configured, basically this should be part of
container-requests. Will file a ticket for this.
> All tasks are failing due to invalid shuffle port number
> --------------------------------------------------------
>
> Key: MAPREDUCE-2983
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2983
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 0.24.0
> Reporter: Devaraj K
>
> {code:xml}
> 2011-09-12 18:43:10,361 INFO mapreduce.Job (Job.java:printTaskEvents(1227))
> - Task Id : attempt_1315831998314_0007_r_000000_0, Status : FAILED
> Container launch failed for container_1315831998314_0007_01_000003 :
> java.lang.IllegalStateException: Invalid shuffle port number -1 returned for
> attempt_1315831998314_0007_r_000000_0
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:226)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
