Github user NiharS commented on the issue: https://github.com/apache/spark/pull/22192 @mridulm I'm inclined to agree. I was doing some extra testing on separate thread and it seems to be performing well. I'll push this shortly. I'm going to also test setting the class loader and then changing back, since I'll always be partial towards synchronous operations, but separate thread is certainly seeming like a good policy given the pyspark failures.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org