HyukjinKwon commented on code in PR #42367:
URL: https://github.com/apache/spark/pull/42367#discussion_r1285319137
##########
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/SparkSession.scala:
##########
@@ -811,30 +826,84 @@ object SparkSession extends Logging {
* If a session exist with the same configuration that is returned instead
of creating a new
* session.
*
+ * This method will update the default and/or active session if they are
not set.
+ *
* @since 3.5.0
*/
def getOrCreate(): SparkSession = {
-
tryCreateSessionFromClient().getOrElse(sessions.get(builder.configuration))
+ val session = tryCreateSessionFromClient()
+ .getOrElse(sessions.get(builder.configuration))
+ // To be compatible with the SQL API we update the default
+ // and active session here if they are not set.
+ defaultSession.compareAndSet(null, session)
+ if (getActiveSession.isEmpty) {
+ setActiveSession(session)
Review Comment:
We should probably either do this same in the `create`, or revert that
behaviour in PySpark (see also https://github.com/apache/spark/pull/41987), cc
@cdkrot @grundprinzip
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
