Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20404
I think the same issue also existed in Scala `SparkSession` code, because
`setDefaultSession` doesn't hold a lock which holds by `getOrCreate`
(SparkSession).
For example:
```scala
val globalSession = SparkSession.build.getOrCreate()
val session1 = globalSession.newSession()
globalSession.sparkContext.stop()
// Thread1
val localSession = SparkSession.build.getOrCreate()
// Thread2 set default session during Thread1's getOrCreate
SparkSession.setDefaultSession(session1)
// After Thread1's getOrCreate is finished, it will still overwrite the
default session to "localSession", not the one (session1) user explicitly set
```
I think this is the similar scenario as mentioned for PySpark.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
