Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/1056#issuecomment-50853874
Thanks @pwendell and @andrewor14 for your continued reviews.
10 seconds sounds fine to me. Not that it's a shining beacon of
performance, but MapReduce actually uses task->application master heartbeats in
exactly the same way. I.e. it doesn't rely on them for them for starting or
stopping tasks. MR AMs will actually receive heartbeats more frequently than
Spark drivers, as there's one per task instead of one per executor. I just
checked and the interval there is 3 seconds.
It might be best to base the interval on the number of executors, but
that's probably work for a separate patch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
