[
https://issues.apache.org/jira/browse/MAPREDUCE-4168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14261232#comment-14261232
]
Allen Wittenauer commented on MAPREDUCE-4168:
---------------------------------------------
bq. I only need hadoop to talk on my secondary interface and not on all
interfaces.
If one only needs to listen on one interface, then only that interface should
be configured in the *-site.xml files.
bq. I noticed that a major part of the traffic goes over the main interface and
not over the secondary interface that I defined in the slaves file.
The slaves files has absolutely zero to do with what the Java interfaces
configure. The etc/hadoop/slaves file is only for using the shell code and
even then only for the ssh bits. (FWIW, I'm working on fixing the
documentation via HADOOP-10908 to help clarify this, since this is a big point
of confusion.) Almost all TCP/IP stacks by default will send traffic outbound
on any network interface where the other side of the pipe is reachable. This
is *usually* a good thing, but there are times where this is problematic. :)
The biggest problem that we have in Hadoop is that we use the server-side
network configuration as the same information we pass to or expect clients to
use for connection information. In multi-home situations, these may not be
(and usually aren't) the same. So if a service is configured to use 0.0.0.0
(to bind to all interfaces), the client can't use that to connect to it since
that doesn't resolve to anything usable.
Karthik's suggestion of using a custom set of *-site.xml files installed on
client nodes to specify that the RM is actually at 10.1.2.3 (or whatever) is a
viable workaround. But it effectively tosses the problem over the wall to "let
ops deal with it." It's friction in the system we really need to fix rather
than passing the buck to the poor schmuck down the line.
There's also the problem of being able to pick which interfaces: if I only want
2 out of 3, there is no way to do it since the network configuration only
supports one entry. So it's all or nothing. The parent of this JIRA is geared
towards fixing these issues. We should probably take a fresh look at it and fix
it in 3.x, since it will likely be an incompatible change to do this correctly.
> Support multiple network interfaces
> -----------------------------------
>
> Key: MAPREDUCE-4168
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4168
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Tom White
>
> Umbrella jira to track the MapReduce side of HADOOP-8198.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
