[
https://issues.apache.org/jira/browse/MAPREDUCE-6338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107128#comment-15107128
]
Alexandr commented on MAPREDUCE-6338:
-------------------------------------
Thank you, [~varun_saxena], unfortunately these JIRAs' don't solve my problem.
Let me explain my issue.
We can see a few strings like this in AM logs when application completes
successfully :
[ContainerLauncher #0]
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy:
Opening proxy : host:45353
[ContainerLauncher #1]
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy:
Opening proxy : host:45358
These ports are random and, of course, job will hang with firewall enabled :
ContainerLauncher #4 org.apache.hadoop.ipc.Client: Retrying connect to server :
host/192.168.121.94:45353. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
Could you please explain why ContainerManagementProtocolProxy opens proxy and
what it actually does ? Is this proxy configurable ? Can I disable proxy ?
Thanks.
> MR AppMaster does not honor ephemeral port range
> ------------------------------------------------
>
> Key: MAPREDUCE-6338
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6338
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 2.6.0
> Reporter: Frank Nguyen
> Assignee: Frank Nguyen
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-6338.002.patch, MAPREDUCE-6338.003.patch
>
>
> The MR AppMaster should only use port ranges defined in the
> yarn.app.mapreduce.am.job.client.port-range property. On initial startup of
> the MRAppMaster, it does use the port range defined in the property.
> However, it also opens up a listener on a random ephemeral port. This is not
> the Jetty listener. It is another listener opened by the MRAppMaster via
> another thread and is recognized by the RM. Other nodes will try to
> communicate to it via that random port. With firewall settings on, the MR
> job will fail because the random port is not opened. This problem has caused
> others to have all OS ephemeral ports opened to have MR jobs run.
> This is related to MAPREDUCE-4079
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
