[
https://issues.apache.org/jira/browse/MAPREDUCE-1247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783569#action_12783569
]
Todd Lipcon commented on MAPREDUCE-1247:
----------------------------------------
My worry about a heartbeat thread that's entirely disconnected from the
operation of the TT is that there are certain cases where the TT is "as good as
dead" but not actually dead. For example, if the TT's in a deadlocked state,
your "true heartbeat" would continue to function, whereas the TT is not healthy
and should be considered dead.
I agree that the optimal system would separate these things, and provide some
kind of health check interface to ensure that the service is actually getting
work done. For a more achievable short term goal, I think deferring these slow
operations to other threads is the safer route. Admittedly I don't work on the
guts of this part of the system much, so will defer now to those that do.
> Send out-of-band heartbeat to avoid fake lost tasktracker
> ---------------------------------------------------------
>
> Key: MAPREDUCE-1247
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1247
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: ZhuGuanyin
>
> Currently the TaskTracker report task status to jobtracker through heartbeat,
> sometimes if the tasktracker lock the tasktracker to do some cleanup job,
> like remove task temp data on disk, the heartbeat thread would hang for a
> long time while waiting for the lock, so the jobtracker just thought it had
> lost and would reschedule all its finished maps or un finished reduce on
> other tasktrackers, we call it "fake lost tasktracker", some times it doesn't
> acceptable especially when we run some large jobs. So We introduce a
> out-of-band heartbeat mechanism to send an out-of-band heartbeat in that case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
