Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20404#discussion_r164372440
--- Diff: python/pyspark/sql/session.py ---
@@ -225,6 +230,9 @@ def __init__(self, sparkContext, jsparkSession=None):
if SparkSession._instantiatedSession is None \
or SparkSession._instantiatedSession._sc._jsc is None:
SparkSession._instantiatedSession = self
+ if self._jvm.SparkSession.getDefaultSession().isEmpty() \
+ or not
jsparkSession.equals(self._jvm.SparkSession.getDefaultSession().get()):
+
self._jvm.SparkSession.setDefaultSession(self._jsparkSession)
--- End diff --
I might miss something, but I guess @felixcheung's concern was fixed by
checking if the default session is defined and not stopped so we can put the
valid session or the same session from JVM without checking anymore.
But I'm okay to leave it as it is as well.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
