Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/18004
> Honestly, your last comment is so unfair.
Wait, which comment? I pointed out places where you current design has
limitations. I'm asking for a way to make it easy to create asynchronous
listeners, because that makes it easy to solve important problems (like
separating user listeners from others, or singling out very sensitive listeners
like the dynamic allocation one that would benefit from being on its own
separate thread).
I mentioned the UI listeners because you mentioned that "the existing
listeners behave well" when that's not true. Yeah I'm changing them in some
separate stuff I'm working on, but that doesn't mean the code I'm adding
magically solves all issues. I'd still like to have certain listeners not be
affected by some other listener suddenly running into a slow path.
And the very last comment I made is still true. I've actually looked at
your code and it does have issue (I noticed at least one race after a very
quick look). But I'm trying to focus on the architecture first before doing a
thorough code review.
> For extracting a basic block... I do not see how to do that in a easy and
useful way
Define the features you want and create an abstract class that implements
that. So far I've seen you mention (i) process events in a separate thread and
(ii) be able to filter which events get queued in that separate thread. Start
from there.
Or, basically, instead of extending the event logging listener, create a
base class for the new functionality you want, and make the existing listener
extend that class. a.k.a. the opposite of what you're doing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
