[
https://issues.apache.org/jira/browse/MAPREDUCE-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158848#comment-13158848
]
Robert Joseph Evans commented on MAPREDUCE-3460:
------------------------------------------------
+1 to Hitesh's patch at least as a quick fix. I can try and reproduce the
issue here and verify that the patch does indeed fix the issue. I can also add
in a few unit tests for it and turn it into a real patch if you like.
I would also like some feedback on a potential (long term) refactor of the code
which would be done on a separate JIRA after 0.23 stabilizes. It seems to me
that the root cause of this issue is because a special condition for a
FAST_FAIL_MAP was missed. The code right now is written with lots of if else
statements separating out map tasks from reduce tasks and also from failed map
tasks, etc. I think it would be cleaner to replace the if statements with
classes that use polymorphism to change the methods called. This would allow
the different handling of a failed map from a normal map or from a reduce to be
more evident. It would also force the internal data structures that keep track
of the different types of tasks to be combined together. This is just
something that popped into my head while trying to evaluate Hitesh's fix. I
have not really evaluated what it would take to make it work or anything, I
would just like some feedback about the idea before filing a JIRA a
bout it.
> MR AM can hang if containers are allocated on a node blacklisted by the AM
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-3460
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3460
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Priority: Blocker
>
> When an AM is assigned a FAILED_MAP (priority = 5) container on a nodemanager
> which it has blacklisted - it tries to
> find a corresponding container request.
> This uses the hostname to find the matching container request - and can end
> up returning any of the ContainerRequests which may have requested a
> container on this node. This container request is cleaned to remove the bad
> node - and then added back to the RM 'ask' list.
> The AM cleans the 'ask' list after each heartbeat - The RM Allocator is still
> aware of the priority=5 container (in 'remoteRequestsTable') - but this never
> gets added back to the 'ask' set - which is what is sent to the RM.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
