Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22295#discussion_r221429353
--- Diff: python/pyspark/sql/session.py ---
@@ -252,6 +253,22 @@ def newSession(self):
"""
return self.__class__(self._sc, self._jsparkSession.newSession())
+ @since(2.5)
+ def getActiveSession(self):
--- End diff --
I think the class method should initialize JVM if non existent (see
functions.py). Probably Spark context too. If exists, it should use the
existing one.
Also, let's define this as a property since that's closer to Scala's usage.
I know it's difficult to define a static property. You can refer
https://github.com/graphframes/graphframes/pull/169/files#diff-e81e6b169c0aa35012a3263b2f31b330R381
or we should consider adding this as a function
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
