Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/8668#issuecomment-142106259
@KaiXinXiaoLei I think it understood what the change you propose in your
last comment would do. I think it would work, except for a very minor issue
where it might cause unnecessary messages to be sent to allocator when the
number of executors hasn't changed.
e.g.
* at t=1, there are `x` executors running
* at t=2, `addExecutors` is called and `x + 1` executors are requested
* at t=3, no executor has been added yet; `addExecutors` is called with `x
+ 1` again, and instead of not sending the message to the allocator, a message
is sent.
That is fine since these messages are not sent that often (at worst, once a
second), and the allocator should handle this situation fine.
It would be nice to confirm that theory with tests, though; are the current
unit tests enough to make sure that change would do the right thing?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
