LantaoJin commented on a change in pull request #26543: [SPARK-29911][SQL]
Uncache cached tables when session closed
URL: https://github.com/apache/spark/pull/26543#discussion_r347788400
##########
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLSessionManager.scala
##########
@@ -75,6 +75,9 @@ private[hive] class SparkSQLSessionManager(hiveServer:
HiveServer2, sqlContext:
override def closeSession(sessionHandle: SessionHandle): Unit = {
HiveThriftServer2.listener.onSessionClosed(sessionHandle.getSessionId.toString)
+ val ctx =
sparkSqlOperationManager.sessionToContexts.getOrDefault(sessionHandle,
sqlContext)
+
ctx.sparkSession.sessionState.catalog.getTempViewNames().foreach(ctx.uncacheTable)
Review comment:
I can work that in another ticket. In current `SparkSQLSessionManager`
implementation, each query will create a new SparkSession and store in the map
`SparkSQLOperationManager.sessionToContexts`. When a session close, this
session will be removed from this map and GC later. `SparkSesson.stop` won't be
invoked at all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
