[
https://issues.apache.org/jira/browse/MAPREDUCE-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080438#comment-13080438
]
Mahadev konar commented on MAPREDUCE-2489:
------------------------------------------
+1 for 0.20S patch. I have committed that to the branch. Thanks Jeffrey. I'll
wait for the trunk common and mapred patches before closing this jira.
> Jobsplits with random hostnames can make the queue unusable
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2489
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2489
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.205.0, 0.23.0
> Reporter: Jeffrey Naisbitt
> Assignee: Jeffrey Naisbitt
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: MAPREDUCE-2489-0.20s-v2.patch,
> MAPREDUCE-2489-0.20s-v3.patch, MAPREDUCE-2489-0.20s-v4.patch,
> MAPREDUCE-2489-0.20s-v5.patch, MAPREDUCE-2489-0.20s.patch,
> MAPREDUCE-2489-mapred-v2.patch, MAPREDUCE-2489-mapred-v3.patch,
> MAPREDUCE-2489-mapred-v4.patch, MAPREDUCE-2489-mapred-v5.patch,
> MAPREDUCE-2489-mapred.patch
>
>
> We saw an issue where a custom InputSplit was returning invalid hostnames for
> the splits that were then causing the JobTracker to attempt to excessively
> resolve host names. This caused a major slowdown for the JobTracker. We
> should prevent invalid InputSplit hostnames from affecting everyone else.
> I propose we implement some verification for the hostnames to try to ensure
> that we only do DNS lookups on valid hostnames (and fail otherwise). We
> could also fail the job after a certain number of failures in the resolve.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
