Github user zjffdu commented on the issue:
https://github.com/apache/spark/pull/13599
@holdenk @HyukjinKwon @ueshin I have updated the PR, and now it also works
when executor is restarted and even dynamic allocation is enabled. The only
overhead is on the driver side when executor ask for configuration (changes in
https://github.com/apache/spark/pull/13599/files#diff-7d99a7c7a051e5e851aaaefb275a44a1),
Now I will constructing new properties when executor ask for properties
instead of like before that creating properties before hand and never change
even when SparkConf is changed. But I believe the overhead is trivial, so the
improvement could be done in a separated PR.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
