Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/2746#issuecomment-58736603
@sryza Thanks for the comments. Unfortunately I have made significant
changes recently and much of the code is now outdated. In my original design I
went with a callback-based approach rather than a polling approach because I
wanted the semantics of the former. In particular, I wanted to add/remove
executors only if the respective condition has been satisfied without
interruption for a certain duration, and this is difficult to guarantee
precisely with polling.
HOWEVER, the significant advantage in polling is that we only need one
extra thread rather than one for each timer. I am convinced that the latter
approach is probably both simpler and more scalable, and I'll likely make the
changes shortly.
Please hold off reviewing this PR for now until I make the relevant changes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
