[
https://issues.apache.org/jira/browse/MAPREDUCE-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787419#action_12787419
]
Arun C Murthy commented on MAPREDUCE-1266:
------------------------------------------
bq. "We have 1 second of unavoidable slowness, so 3 more per task is OK?"
My response was:
bq. I'm failing to see how *sub-second* heartbeat intervals will help.
----
bq. Are there any substantive reasons against this? I didn't anticipate much
discussion over a change in a single constant that only affects tiny clusters
Adding more config knobs doesn't help... we will need to maintain them even if
they are undocumented. I'm vary of adding more knobs without more thought.
The blog post is quite vague.
For e.g. would setting it to 2s help as much as setting it to .5s? Or 1.5s? If
so should we just set the minimum lower to 1.5s or 2s and get it over with?
> Allow heartbeat interval smaller than 3 seconds for tiny clusters
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1266
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1266
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker, task, tasktracker
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Priority: Minor
>
> For small clusters, the heartbeat interval has a large effect on job latency.
> This is especially true on pseudo-distributed or other "tiny" (<5 nodes)
> clusters. It's not a big deal for production, but new users would have a
> happier first experience if Hadoop seemed snappier.
> I'd like to change the minimum heartbeat interval from 3.0 seconds to perhaps
> 0.5 seconds (but have it governed by an undocumented config parameter in case
> people don't like this change). The cluster size-based ramp up of interval
> will maintain the current scalable behavior for large clusters with no
> negative effect.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
