venkata91 commented on code in PR #59277:
URL: https://github.com/apache/spark/pull/59277#discussion_r4254179459
##########
sql/connect/server/src/main/scala/org/apache/spark/sql/connect/config/Connect.scala:
##########
@@ -150,6 +150,29 @@ object Connect {
.intConf
.createWithDefaultString("1000")
+ val CONNECT_SESSION_ALLOW_RECONNECT_AFTER_CLOSE =
+ buildConf("spark.connect.session.allowReconnectAfterClose")
+ .doc(
+ "When true, the session id can be reused after this session is closed,
for example " +
+ "on inactivity expiry. The id stays reserved, and requests fail with
" +
+ "INVALID_HANDLE.SESSION_CLOSED, until the close finishes. After
that, a client that " +
+ "used the closed session gets INVALID_HANDLE.SESSION_CHANGED if it
sends its " +
+ "observed server-side session id, as the built-in Python and Scala
clients do. " +
+ "Such a client must create a new SparkSession with the same session
id. A client " +
+ "that does not send it silently gets a new session. The new session
is empty: " +
+ "server-side state " +
+ "such as registered UDFs, temporary views, temporary functions,
cached data and the " +
Review Comment:
Good catch, done in COMMIT. `SessionHolder.close()` doesn't touch the shared
`CacheManager`, so I dropped cached data from the lost-state list in both the
conf doc and the `docs/configuration.md` row. I also applied your two PR
description suggestions.
##########
sql/connect/server/src/main/scala/org/apache/spark/sql/connect/config/Connect.scala:
##########
@@ -150,6 +150,29 @@ object Connect {
.intConf
.createWithDefaultString("1000")
+ val CONNECT_SESSION_ALLOW_RECONNECT_AFTER_CLOSE =
+ buildConf("spark.connect.session.allowReconnectAfterClose")
+ .doc(
+ "When true, the session id can be reused after this session is closed,
for example " +
+ "on inactivity expiry. The id stays reserved, and requests fail with
" +
+ "INVALID_HANDLE.SESSION_CLOSED, until the close finishes. After
that, a client that " +
+ "used the closed session gets INVALID_HANDLE.SESSION_CHANGED if it
sends its " +
+ "observed server-side session id, as the built-in Python and Scala
clients do. " +
+ "Such a client must create a new SparkSession with the same session
id. A client " +
+ "that does not send it silently gets a new session. The new session
is empty: " +
+ "server-side state " +
+ "such as registered UDFs, temporary views, temporary functions,
cached data and the " +
Review Comment:
Good catch, done in e74d061846. `SessionHolder.close()` doesn't touch the
shared `CacheManager`, so I dropped cached data from the lost-state list in
both the conf doc and the `docs/configuration.md` row. I also applied your two
PR description suggestions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
