Github user squito commented on the issue:
https://github.com/apache/spark/pull/21185
@cloud-fan
> I think SparkSession is driver only, how do we access it in executor?
that's the whole problem. Its only meant to be available on the driver,
but it ends up getting set on the executor when running in local mode. Because
it uses an *inheritable* thread local, when the executor thread pool creates a
new thread, the executor thread ends up inheriting the active session of the
driver.
In cluster mode, when the executor is a totally separate JVM, there is no
problem.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
