[
https://issues.apache.org/jira/browse/MAPREDUCE-6338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098116#comment-15098116
]
Frank Nguyen commented on MAPREDUCE-6338:
-----------------------------------------
The patch only requires a small change in:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/TaskAttemptListenerImpl.java
@@ -127,7 +127,7 @@ protected void startRpcServer() {
server =
new RPC.Builder(conf).setProtocol(TaskUmbilicalProtocol.class)
.setInstance(this).setBindAddress("0.0.0.0")
- .setPort(0).setNumHandlers(
+
.setPortRangeConfig(MRJobConfig.MR_AM_JOB_CLIENT_PORT_RANGE).setNumHandlers(
conf.getInt(MRJobConfig.MR_AM_TASK_LISTENER_THREAD_COUNT,
MRJobConfig.DEFAULT_MR_AM_TASK_LISTENER_THREAD_COUNT))
.setVerbose(false).setSecretManager(jobTokenSecretManager)
Other files were not modified. Not sure why it is included in the patch file.
For some reason, this small change never passed the Hadoop QA process since
last year. For those familiar with this process, feel free to take over and
submit a proper patch file. Thanks
> MR AppMaster does not honor ephemeral port range
> ------------------------------------------------
>
> Key: MAPREDUCE-6338
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6338
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 2.6.0
> Reporter: Frank Nguyen
> Assignee: Frank Nguyen
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-6338.002.patch
>
>
> The MR AppMaster should only use port ranges defined in the
> yarn.app.mapreduce.am.job.client.port-range property. On initial startup of
> the MRAppMaster, it does use the port range defined in the property.
> However, it also opens up a listener on a random ephemeral port. This is not
> the Jetty listener. It is another listener opened by the MRAppMaster via
> another thread and is recognized by the RM. Other nodes will try to
> communicate to it via that random port. With firewall settings on, the MR
> job will fail because the random port is not opened. This problem has caused
> others to have all OS ephemeral ports opened to have MR jobs run.
> This is related to MAPREDUCE-4079
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
