venkata91 commented on PR #59277: URL: https://github.com/apache/spark/pull/59277#issuecomment-6111249464
> Design question (review): a stale client's request after an opted-in expiry can create a non-opted-in session that later re-tombstones the id. Thanks, good catch. Yes, our gateway and notebook clients can keep a client object across an expiry and send requests from it, so a stale request could leave a non-opted-in session behind and get the id tombstoned again. I changed the server so this can't happen: `getOrCreateIsolatedSession` now fails with `INVALID_HANDLE.SESSION_CHANGED` before creating anything when the request carries an observed server-side session id and no session exists for the key. The check runs inside the `computeIfAbsent` creation function, after the format and `SESSION_CLOSED` checks, so the error order is unchanged and nothing is inserted. An existing session with a mismatched id still gets `SESSION_CHANGED` from the usual validation. New tests cover an absent key, a stale client after an opted-in expiry, and a closed key with an observed id. The conf doc and the `docs/configuration.md` row now also say that the old client gets `SESSION_CHANGED`, that the user must create a new `SparkSession` with the same session id whose server-side state (UDFs, temporary views and functions, cached data, current catalog) must be rebuilt, and that the new session must set the conf again to keep the opt-in. Fixed in b4f98b9d37. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
