Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19981
So the problem is, it takes me a while to understand the new code, because
of the weird design for `AppStatusPlugin`. At least we should have a separate
plugin interface for live UI. Code readability is important to Spark, we should
not sacrifice it without real benefits.
On the other hand, I think it's better to not register a listener if
unnecessary. `SQLListener` is fine because it's a no-op for non-sql events, but
can we guarantee it in all other places like streaming? (need confirmation from
@zsxwing @marmbrus ).
And this is also an internal behavior change(the timing of registering SQL
listeners), which may stop us from catching Spark events in SQL listener in the
future. Do we have a good reason to change it? Having a central place for live
UI setup seems not a strong reason to me.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
