Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
I'm going to commit the fixes recommended by @vanzin soon. Then, I think
the biggest remaining concern is the threads. I'm okay with adding a `start`
and `stop`, but I'm curious as to whether those should potentially be in a
separate Jira. This could potentially go into 2.4, and more methods could be
added as we learn the use cases for plugins. My concern with adding them is
that if it's decided they're obsolete or repetitive (in the case of `init` vs
`start`), but people have already started using some plugins, it'd be annoying
to take away the methods they expect to work. My best argument for adding the
methods is we could try to ensure people conform to a similar structure if we
have, say, the `init` function run in the same thread as the executor, and then
once the executors have initialized we could create a thread that `start`s the
plugins so really plugin writers wouldn't have to even worry about making their
plugins async. Does anyone have strong opinions?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
