Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/18052
I subscribed this PR as actually I am the user who implemented the similar
one in user side weirdly. But one thing I can tell is, it was buggy and hard to
debugg / test in particular when the workload is intensive. Some jobs were
aborted and I ended up with writing up weird codes to prevent this (it was
maybe my fault but I just wanted to tell my anecdote).
If this needs a different Python specific implementation alone, I would not
rather support this functionality for maintanace issue for now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
