[
https://issues.apache.org/jira/browse/MAPREDUCE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated MAPREDUCE-3251:
-----------------------------------------------
Summary: Network ACLs can prevent some clients to talk to MR
ApplicationMaster (was: JobClient should have an option to only to talk to
RM+HistoryServer to get job status)
It was a mistake to describe the solution in the title instead of the problem
itself. Editing title for the same reason.
I thought more about this and I am getting increasingly concerned about option
(1) for two reasons: Special clients like oozie, and clients outside the grid
(which are the ones who will see this issue in the presence of network ACLs)
will be seriously impaired with respect to getting updated information while
the job is running for e.g, they won't see progress information. Also all of
them will need to be very clearly aware of this new API which is a regression
of sorts.
I am leaning towards option (2). Though I agree that the OSes don't have an api
to bind to a particular range of ports, it can still be done by randomly
generating ports and keep trying till we can successfully bind to one of them.
I know it isn't aesthetic, I myself don't like it much, but I do know from my
experience with HOD that it will work. At any rate, it is better than the
broken experience caused by adding new API.
> Network ACLs can prevent some clients to talk to MR ApplicationMaster
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3251
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3251
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Critical
> Fix For: 0.23.1
>
>
> In 0.20.xxx, the JobClient while polling goes to JT to get the job status.
> With YARN, AM can be launched on any port and the client will have to have
> ACL open to that port to talk to AM and get the job status. When the client
> is within the same grid network access to AM is not a problem. But some
> applications may have one installation per set of clusters and may launch
> jobs even across such sets (on job trackers in another set of clusters). For
> that to work only the JT port needs to be open currently. In case of YARN,
> all ports will have to be opened up for things to work. That would be a
> security no-no.
> There are two possible solutions:
> 1) Make the job client only talk to RM (as an option) to get the job
> status.
> 2) Limit the range of ports AM can listen on.
> Option 2) may not be favorable as there is no direct OS API to find a free
> port.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
