Github user NiharS commented on the issue: https://github.com/apache/spark/pull/22192 I tested @vanzin's suggestion of storing the classloader, changing it to replClassLoader, and changing back after init (and then changing again for shutdown). It's working well (and this time I made sure it doesn't just shut down immediately; I ran on a cluster and confirmed it runs the init code and creates a thread, the thread stays alive for the duration of the executor, and the shutdown code runs fully before exiting). While it looks a little weird, the code is synchronous and shouldn't run into an issue with the classloader being set to something it shouldn't be set to while executing the existing Spark code, and ensures the plugins initialize fully during executor initialization.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org