This is an automated email from the ASF dual-hosted git repository.

sunchao pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new 3bf4fc3bc3c0 [SPARK-58097][CONNECT] Preserve composite (userId, 
sessionId) session identity in Connect UI/status store
3bf4fc3bc3c0 is described below

commit 3bf4fc3bc3c0462743b9fed7896c799fffdc0197
Author: Venkata krishnan Sowrirajan <[email protected]>
AuthorDate: Mon Jul 27 17:28:13 2026 -0700

    [SPARK-58097][CONNECT] Preserve composite (userId, sessionId) session 
identity in Connect UI/status store
    
    ### What changes were proposed in this pull request?
    
    Spark Connect identifies a session by `(userId, sessionId)`, and two users 
may share the same session UUID. The Connect UI listener/store keyed sessions 
by `sessionId` alone, so such sessions collapsed into one record (merged 
execution counts; one user's close finished/removed the other's row). History 
Server replay reproduced the same bug.
    
    This keys `SessionInfo`'s KVStore natural key and the listener's 
live-session map by a synthesized composite `uniqueId = userId/sessionId`, and 
threads `userId` through the UI session page, its links, and the REST 
`sessions/{sessionId}` endpoint (added in SPARK-57941). Specifically:
    
    - **Composite identity:** `SessionInfo` is keyed on `(userId, sessionId)`; 
the session page filters operations on both fields so one user's operations 
don't leak into another's page.
    - **Opaque user ids through the UI:** `userId` is arbitrary and part of the 
key, but the UI wraps requests in `XssSafeRequest` (strips apostrophes, 
HTML-escapes) and `PagedTable` re-echoes decoded params. So `userId` is carried 
as an unpadded base64url token, whose alphabet (`A-Za-z0-9-_`) survives both 
untouched. The REST endpoint takes the same token; an empty token is a valid 
empty user id (the protobuf default), so only an absent parameter is rejected.
    - **Legacy History Server stores:** `getSession` falls back to a field scan 
on a composite-key miss, so disk stores cached by an older Spark (keyed on 
`sessionId` alone) still resolve after an upgrade with no replay.
    
    ### Why are the changes needed?
    
    Two users sharing a session UUID are distinct sessions in Spark Connect 
(see `SparkConnectServiceE2ESuite`), but the UI/status store merged them, 
showing incorrect counts and lifecycle state. Surfaced during review of 
SPARK-57941.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes, within the unreleased master/branch line (the REST API and this fix 
are unreleased):
    
    - The Connect session detail page and the REST `sessions/{sessionId}` 
endpoint now require a `userId` parameter (a base64url token) alongside the 
session id.
    - **Trade-off on legacy persistent stores:** the natural key changes 
without bumping `AppStatusStore.CURRENT_VERSION`. A version bump is the only 
replay knob and is global, forcing every application's disk store (core, SQL, 
Hive Thrift, Connect) to rebuild, which is disproportionate for a Connect-only 
key change. Instead the `getSession` fallback resolves legacy rows in place. 
Its one limitation: rows where two users genuinely shared a UUID *before* the 
upgrade were already merged und [...]
    
    ### How was this patch tested?
    
    Added to `SparkConnectServerListenerSuite`: same-UUID/different-user 
coverage (live + History Server replay) and a legacy `sessionId`-keyed row that 
the fallback resolves. Added to `SparkConnectServerPageSuite`: per-user 
operation isolation and a base64url round-trip through the real 
`XssSafeRequest.stripXSS` transform for values containing `'`, `+`, `=`. Added 
to `ConnectResourceWithActualDataSuite`: REST lookup by the `userId` token over 
the real HTTP filter, plus an empty-user sess [...]
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code (Claude Opus 4.8)
    
    Closes #57219 from venkata91/SPARK-58097-session-identity.
    
    Authored-by: Venkata krishnan Sowrirajan <[email protected]>
    Signed-off-by: Chao Sun <[email protected]>
    (cherry picked from commit d8d08b6c3170739ed22737af1e81bada846b9085)
    Signed-off-by: Chao Sun <[email protected]>
---
 docs/monitoring.md                                 |  6 +-
 .../ui/SparkConnectServerAppStatusStore.scala      | 21 ++++--
 .../connect/ui/SparkConnectServerListener.scala    | 18 ++---
 .../sql/connect/ui/SparkConnectServerPage.scala    | 22 ++++--
 .../connect/ui/SparkConnectServerSessionPage.scala | 15 ++--
 .../status/api/v1/connect/ConnectResource.scala    | 20 +++++-
 .../ui/SparkConnectServerListenerSuite.scala       | 81 ++++++++++++++++++++--
 .../connect/ui/SparkConnectServerPageSuite.scala   | 56 +++++++++++++++
 .../ConnectResourceWithActualDataSuite.scala       | 33 ++++++++-
 9 files changed, 238 insertions(+), 34 deletions(-)

diff --git a/docs/monitoring.md b/docs/monitoring.md
index 0c178110e8f6..64552ba28512 100644
--- a/docs/monitoring.md
+++ b/docs/monitoring.md
@@ -722,8 +722,10 @@ can be identified by their `[attempt-id]`. In the API 
listed below, when running
     </td>
   </tr>
   <tr>
-    <td><code>/applications/[app-id]/connect/sessions/[session-id]</code></td>
-    <td>Details for the given Spark Connect session.</td>
+    
<td><code>/applications/[app-id]/connect/sessions/[session-id]?userId=[user-id]</code></td>
+    <td>Details for the given Spark Connect session. A session is identified 
by the composite
+    <code>(userId, sessionId)</code>, so <code>userId</code> is required and 
must be passed as an
+    unpadded base64url-encoded string.</td>
   </tr>
   <tr>
     <td><code>/applications/[app-id]/connect/operations</code></td>
diff --git 
a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerAppStatusStore.scala
 
b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerAppStatusStore.scala
index b5211b6c3072..523edb446080 100644
--- 
a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerAppStatusStore.scala
+++ 
b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerAppStatusStore.scala
@@ -47,11 +47,14 @@ class SparkConnectServerAppStatusStore(store: KVStore) {
     KVUtils.count(store.view(classOf[SessionInfo]))(_.finishTimestamp == 0)
   }
 
-  def getSession(sessionId: String): Option[SessionInfo] = {
+  def getSession(userId: String, sessionId: String): Option[SessionInfo] = {
     try {
-      Some(store.read(classOf[SessionInfo], sessionId))
+      Some(store.read(classOf[SessionInfo], SessionInfo.uniqueId(userId, 
sessionId)))
     } catch {
-      case _: NoSuchElementException => None
+      case _: NoSuchElementException =>
+        // Fall back for legacy stores keyed on sessionId alone (e.g. a 
History Server disk store
+        // from an older Spark). Collapsed same-UUID rows cannot be recovered.
+        getSessionList.find(s => s.userId == userId && s.sessionId == 
sessionId)
     }
   }
 
@@ -82,11 +85,16 @@ class SparkConnectServerAppStatusStore(store: KVStore) {
 }
 
 private[spark] class SessionInfo(
-    @KVIndexParam val sessionId: String,
+    val sessionId: String,
     val startTimestamp: Long,
     val userId: String,
     val finishTimestamp: Long,
     val totalExecution: Long) {
+  // Natural key. A session is identified by (userId, sessionId), since two 
users may share the
+  // same session UUID; keying on sessionId alone would merge them into one 
record.
+  @KVIndexParam
+  def uniqueId: String = SessionInfo.uniqueId(userId, sessionId)
+
   @JsonIgnore @KVIndex("finishTime")
   private def finishTimeIndex: Long = if (finishTimestamp > 0L) 
finishTimestamp else -1L
   def totalTime: Long = {
@@ -98,6 +106,11 @@ private[spark] class SessionInfo(
   }
 }
 
+private[connect] object SessionInfo {
+  // sessionId is a UUID, so joining with '/' yields a key that is unique per 
(userId, sessionId).
+  def uniqueId(userId: String, sessionId: String): String = 
s"$userId/$sessionId"
+}
+
 private[spark] class ExecutionInfo(
     @KVIndexParam val jobTag: String,
     val statement: String,
diff --git 
a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerListener.scala
 
b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerListener.scala
index 01a775f89744..dfc36470a7d5 100644
--- 
a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerListener.scala
+++ 
b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerListener.scala
@@ -180,7 +180,7 @@ private[connect] class SparkConnectServerListener(
     updateLiveStore(executionData) { executionData =>
       executionData.state = ExecutionState.STARTED
     }
-    Option(sessionList.get(e.sessionId)) match {
+    Option(sessionList.get(SessionInfo.uniqueId(e.userId, e.sessionId))) match 
{
       case Some(sessionData) =>
         updateLiveStore(sessionData) { sessionData => 
sessionData.totalExecution += 1 }
       case None =>
@@ -282,7 +282,7 @@ private[connect] class SparkConnectServerListener(
 
   private def onSessionClosed(e: SparkListenerConnectSessionClosed) = {
     sessionList.compute(
-      e.sessionId,
+      SessionInfo.uniqueId(e.userId, e.sessionId),
       (_, sessionData) => {
         if (sessionData != null) {
           updateStoreWithTriggerEnabled(sessionData) { sessionData =>
@@ -319,11 +319,11 @@ private[connect] class SparkConnectServerListener(
 
   private def getOrCreateSession(
       sessionId: String,
-      userName: String,
+      userId: String,
       startTime: Long): LiveSessionData = {
     sessionList.computeIfAbsent(
-      sessionId,
-      _ => new LiveSessionData(sessionId, startTime, userName))
+      SessionInfo.uniqueId(userId, sessionId),
+      _ => new LiveSessionData(sessionId, startTime, userId))
   }
 
   private def getOrCreateExecution(
@@ -369,7 +369,9 @@ private[connect] class SparkConnectServerListener(
       j.finishTimestamp != 0L
     }
 
-    toDelete.foreach { j => kvstore.delete(j.getClass, j.sessionId) }
+    toDelete.foreach { j =>
+      kvstore.delete(j.getClass, SessionInfo.uniqueId(j.userId, j.sessionId))
+    }
   }
 
   /**
@@ -431,14 +433,14 @@ private[connect] class LiveExecutionData(
 private[connect] class LiveSessionData(
     val sessionId: String,
     val startTimestamp: Long,
-    val userName: String)
+    val userId: String)
     extends LiveEntity {
 
   var finishTimestamp: Long = 0L
   var totalExecution: Int = 0
 
   override protected def doUpdate(): Any = {
-    new SessionInfo(sessionId, startTimestamp, userName, finishTimestamp, 
totalExecution)
+    new SessionInfo(sessionId, startTimestamp, userId, finishTimestamp, 
totalExecution)
   }
   def totalTime: Long = {
     if (finishTimestamp == 0L) {
diff --git 
a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerPage.scala
 
b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerPage.scala
index b34cf8d5cfa7..ea78b2dc59f6 100644
--- 
a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerPage.scala
+++ 
b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerPage.scala
@@ -19,6 +19,7 @@ package org.apache.spark.sql.connect.ui
 
 import java.net.URLEncoder
 import java.nio.charset.StandardCharsets.UTF_8
+import java.util.Base64
 
 import scala.xml.Node
 
@@ -30,6 +31,17 @@ import org.apache.spark.ui._
 import org.apache.spark.ui.UIUtils._
 import org.apache.spark.util.Utils
 
+// userId is part of a session's natural key but is opaque and arbitrary. It 
is carried through UI
+// links as unpadded base64url, whose alphabet (A-Za-z0-9-_) survives 
XssSafeRequest sanitization
+// and the PagedTable parameter re-echo unchanged.
+private[spark] object ConnectUiUtils {
+  def encodeUserId(userId: String): String =
+    Base64.getUrlEncoder.withoutPadding.encodeToString(userId.getBytes(UTF_8))
+
+  def decodeUserId(token: String): String =
+    new String(Base64.getUrlDecoder.decode(token), UTF_8)
+}
+
 /** Page for Spark UI that shows statistics for a Spark Connect Server. */
 private[ui] class SparkConnectServerPage(parent: SparkConnectServerTab)
     extends WebUIPage("")
@@ -276,10 +288,11 @@ private[ui] class SqlStatsPagedTable(
         <a href={sqlURL(request, sqlExecId)}>[{sqlExecId}]</a>
       }
     }
-    val sessionLink = "%s/%s/session/?id=%s".format(
+    val sessionLink = "%s/%s/session/?id=%s&userId=%s".format(
       UIUtils.prependBaseUri(request, parent.basePath),
       parent.prefix,
-      info.sessionId)
+      URLEncoder.encode(info.sessionId, UTF_8.name()),
+      ConnectUiUtils.encodeUserId(info.userId))
 
     <tr>
       <td>
@@ -406,10 +419,11 @@ private[ui] class SessionStatsPagedTable(
   }
 
   override def row(session: SessionInfo): Seq[Node] = {
-    val sessionLink = "%s/%s/session/?id=%s".format(
+    val sessionLink = "%s/%s/session/?id=%s&userId=%s".format(
       UIUtils.prependBaseUri(request, parent.basePath),
       parent.prefix,
-      session.sessionId)
+      URLEncoder.encode(session.sessionId, UTF_8.name()),
+      ConnectUiUtils.encodeUserId(session.userId))
     <tr>
       <td> {session.userId} </td>
       <td> <a href={sessionLink}> {session.sessionId} </a> </td>
diff --git 
a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerSessionPage.scala
 
b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerSessionPage.scala
index 391fc6c09fd1..292b0946382f 100644
--- 
a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerSessionPage.scala
+++ 
b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerSessionPage.scala
@@ -38,10 +38,14 @@ private[ui] class SparkConnectServerSessionPage(parent: 
SparkConnectServerTab)
   def render(request: HttpServletRequest): Seq[Node] = {
     val sessionId = request.getParameter("id")
     require(sessionId != null && sessionId.nonEmpty, "Missing id parameter")
+    // userId arrives as a base64url token; see ConnectUiUtils.
+    val userIdParam = request.getParameter("userId")
+    require(userIdParam != null, "Missing userId parameter")
+    val userId = ConnectUiUtils.decodeUserId(userIdParam)
 
     val content = store.synchronized { // make sure all parts in this page are 
consistent
       store
-        .getSession(sessionId)
+        .getSession(userId, sessionId)
         .map { sessionStat =>
           generateBasicStats(sessionId) ++
             <br/> ++
@@ -56,7 +60,7 @@ private[ui] class SparkConnectServerSessionPage(parent: 
SparkConnectServerTab)
             {sessionStat.totalExecution}
             Request(s)
           </h4> ++
-            generateSQLStatsTable(request, sessionStat.sessionId)
+            generateSQLStatsTable(request, sessionStat.userId, 
sessionStat.sessionId)
         }
         .getOrElse(<div>No information to display for session 
{sessionId}</div>)
     }
@@ -80,9 +84,12 @@ private[ui] class SparkConnectServerSessionPage(parent: 
SparkConnectServerTab)
   }
 
   /** Generate stats of batch statements of the Spark Connect server */
-  private def generateSQLStatsTable(request: HttpServletRequest, sessionID: 
String): Seq[Node] = {
+  private def generateSQLStatsTable(
+      request: HttpServletRequest,
+      userId: String,
+      sessionId: String): Seq[Node] = {
     val executionList = store.getExecutionList
-      .filter(_.sessionId == sessionID)
+      .filter(exec => exec.userId == userId && exec.sessionId == sessionId)
     val numStatement = executionList.size
     val table = if (numStatement > 0) {
 
diff --git 
a/sql/connect/server/src/main/scala/org/apache/spark/status/api/v1/connect/ConnectResource.scala
 
b/sql/connect/server/src/main/scala/org/apache/spark/status/api/v1/connect/ConnectResource.scala
index 6e85ab5ead2b..8185eac4a074 100644
--- 
a/sql/connect/server/src/main/scala/org/apache/spark/status/api/v1/connect/ConnectResource.scala
+++ 
b/sql/connect/server/src/main/scala/org/apache/spark/status/api/v1/connect/ConnectResource.scala
@@ -20,7 +20,7 @@ package org.apache.spark.status.api.v1.connect
 import jakarta.ws.rs._
 import jakarta.ws.rs.core.MediaType
 
-import org.apache.spark.sql.connect.ui.{ExecutionInfo, SessionInfo, 
SparkConnectServerAppStatusStore}
+import org.apache.spark.sql.connect.ui.{ConnectUiUtils, ExecutionInfo, 
SessionInfo, SparkConnectServerAppStatusStore}
 import org.apache.spark.status.api.v1.{BadParameterException, BaseAppResource, 
NotFoundException}
 
 @Produces(Array(MediaType.APPLICATION_JSON))
@@ -37,12 +37,26 @@ private[v1] class ConnectResource extends BaseAppResource {
     sessions.map(prepareSessionData)
   }
 
+  // Sessions are keyed by (userId, sessionId), so userId is required. It is a 
base64url token (see
+  // ConnectUiUtils); an empty token is a valid empty user id, so only an 
absent param is rejected.
   @GET
   @Path("sessions/{sessionId}")
-  def session(@PathParam("sessionId") sessionId: String): SessionData = withUI 
{ ui =>
+  def session(
+      @PathParam("sessionId") sessionId: String,
+      @QueryParam("userId") userId: String): SessionData = withUI { ui =>
+    if (userId == null) {
+      throw new BadParameterException("userId is required.")
+    }
+    val decodedUserId =
+      try {
+        ConnectUiUtils.decodeUserId(userId)
+      } catch {
+        case _: IllegalArgumentException =>
+          throw new BadParameterException("userId must be a base64url-encoded 
string.")
+      }
     val store = new SparkConnectServerAppStatusStore(ui.store.store)
     store
-      .getSession(sessionId)
+      .getSession(decodedUserId, sessionId)
       .map(prepareSessionData)
       .getOrElse(throw new NotFoundException("unknown session id: " + 
sessionId))
   }
diff --git 
a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/ui/SparkConnectServerListenerSuite.scala
 
b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/ui/SparkConnectServerListenerSuite.scala
index e9cd7c692373..be31462f87a8 100644
--- 
a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/ui/SparkConnectServerListenerSuite.scala
+++ 
b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/ui/SparkConnectServerListenerSuite.scala
@@ -51,7 +51,7 @@ class SparkConnectServerListenerSuite
       val (statusStore: SparkConnectServerAppStatusStore, listener: 
SparkConnectServerListener) =
         createAppStatusStore(live)
       listener.onOtherEvent(
-        SparkListenerConnectSessionStarted("sessionId", "user", 
System.currentTimeMillis()))
+        SparkListenerConnectSessionStarted("sessionId", "userId", 
System.currentTimeMillis()))
       listener.onOtherEvent(
         SparkListenerConnectOperationStarted(
           jobTag,
@@ -113,16 +113,16 @@ class SparkConnectServerListenerSuite
         createAppStatusStore(live)
       var time = 0
       listener.onOtherEvent(
-        SparkListenerConnectSessionStarted("sessionId1", "user", 
System.currentTimeMillis()))
+        SparkListenerConnectSessionStarted("sessionId1", "userId", 
System.currentTimeMillis()))
       time += 1
       listener.onOtherEvent(
-        SparkListenerConnectSessionStarted("sessionId2", "user", 
System.currentTimeMillis()))
+        SparkListenerConnectSessionStarted("sessionId2", "userId", 
System.currentTimeMillis()))
       time += 1
       listener.onOtherEvent(SparkListenerConnectSessionClosed("sessionId1", 
"userId", time))
       time += 1
       listener.onOtherEvent(SparkListenerConnectSessionClosed("sessionId2", 
"userId", time))
       listener.onOtherEvent(
-        SparkListenerConnectSessionStarted("sessionId3", "user", 
System.currentTimeMillis()))
+        SparkListenerConnectSessionStarted("sessionId3", "userId", 
System.currentTimeMillis()))
       time += 1
       listener.onOtherEvent(SparkListenerConnectSessionClosed("sessionId3", 
"userId", time))
 
@@ -131,11 +131,78 @@ class SparkConnectServerListenerSuite
       }
       assert(statusStore.getOnlineSessionNum === 0)
       assert(statusStore.getSessionCount === 1)
-      assert(statusStore.getSession("sessionId1") === None)
+      assert(statusStore.getSession("userId", "sessionId1") === None)
       assert(listener.noLiveData())
     }
   }
 
+  Seq(true, false).foreach { live =>
+    test(s"SPARK-58097: sessions sharing a UUID across users stay distinct 
(live = $live)") {
+      val (statusStore: SparkConnectServerAppStatusStore, listener: 
SparkConnectServerListener) =
+        createAppStatusStore(live, sessionLimit = 10)
+      val sessionId = "shared-session-uuid"
+      // Two users share the same session UUID; Spark Connect must keep them 
distinct.
+      listener.onOtherEvent(
+        SparkListenerConnectSessionStarted(sessionId, "userA", 
System.currentTimeMillis()))
+      listener.onOtherEvent(
+        SparkListenerConnectSessionStarted(sessionId, "userB", 
System.currentTimeMillis()))
+      // One operation against userA's session only.
+      listener.onOtherEvent(
+        SparkListenerConnectOperationStarted(
+          ExecuteJobTag("userA", sessionId, "operationId"),
+          "operationId",
+          System.currentTimeMillis(),
+          sessionId,
+          "userA",
+          "userName",
+          "dummy query",
+          Set()))
+      listener.onOtherEvent(SparkListenerConnectSessionClosed(sessionId, 
"userA", 1000L))
+      if (live) {
+        assert(statusStore.getOnlineSessionNum === 1)
+        assert(statusStore.getSession("userB", 
sessionId).exists(_.finishTimestamp === 0))
+      }
+      // Close userB at a distinct time so the history store also records both.
+      listener.onOtherEvent(SparkListenerConnectSessionClosed(sessionId, 
"userB", 2000L))
+
+      if (!live) {
+        kvstore.close(false)
+      }
+
+      // The sessions stay separate: distinct counts and close times, no 
collapse.
+      assert(statusStore.getSessionCount === 2)
+      val sessionA = statusStore.getSession("userA", sessionId)
+      val sessionB = statusStore.getSession("userB", sessionId)
+      assert(sessionA.isDefined && sessionB.isDefined)
+      assert(sessionA.get.totalExecution === 1)
+      assert(sessionB.get.totalExecution === 0)
+      assert(sessionA.get.finishTimestamp === 1000L)
+      assert(sessionB.get.finishTimestamp === 2000L)
+    }
+  }
+
+  test("SPARK-58097: getSession falls back to a scan for legacy 
sessionId-keyed rows") {
+    // A legacy store keys SessionInfo on sessionId alone, so the 
composite-key read misses while
+    // the row is still enumerable via the view. Force that read to fail to 
reproduce it.
+    val legacyStore = new InMemoryStore {
+      override def read[T](klass: Class[T], naturalKey: Object): T = {
+        if (klass == classOf[SessionInfo]) throw new NoSuchElementException()
+        else super.read(klass, naturalKey)
+      }
+    }
+    val store = new SparkConnectServerAppStatusStore(legacyStore)
+    val sessionId = "legacy-session-uuid"
+    legacyStore.write(new SessionInfo(sessionId, 1000L, "userA", 2000L, 3L))
+
+    val session = store.getSession("userA", sessionId)
+    assert(session.isDefined)
+    assert(session.get.userId === "userA")
+    assert(session.get.sessionId === sessionId)
+    assert(session.get.totalExecution === 3L)
+    // A miss with no matching row still returns None.
+    assert(store.getSession("userB", sessionId).isEmpty)
+  }
+
   test(
     "update execution info when event reordering causes job and sql" +
       " start to come after operation closed") {
@@ -240,12 +307,12 @@ class SparkConnectServerListenerSuite
     properties
   }
 
-  private def createAppStatusStore(live: Boolean) = {
+  private def createAppStatusStore(live: Boolean, sessionLimit: Int = 1) = {
     val sparkConf = new SparkConf()
     sparkConf
       .set(ASYNC_TRACKING_ENABLED, false)
       .set(LIVE_ENTITY_UPDATE_PERIOD, 0L)
-      .set(CONNECT_UI_SESSION_LIMIT, 1)
+      .set(CONNECT_UI_SESSION_LIMIT, sessionLimit)
       .set(CONNECT_UI_STATEMENT_LIMIT, 10)
     kvstore = new ElementTrackingStore(new InMemoryStore, sparkConf)
     if (live) {
diff --git 
a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/ui/SparkConnectServerPageSuite.scala
 
b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/ui/SparkConnectServerPageSuite.scala
index 61d416328d0e..7f6af17bc41b 100644
--- 
a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/ui/SparkConnectServerPageSuite.scala
+++ 
b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/ui/SparkConnectServerPageSuite.scala
@@ -20,6 +20,7 @@ package org.apache.spark.sql.connect.ui
 import java.util.{Calendar, Locale}
 
 import jakarta.servlet.http.HttpServletRequest
+import org.apache.commons.text.StringEscapeUtils
 import org.mockito.Mockito.{mock, when, RETURNS_SMART_NULLS}
 import org.scalatest.BeforeAndAfter
 
@@ -110,6 +111,7 @@ class SparkConnectServerPageSuite
 
     val request = mock(classOf[HttpServletRequest])
     when(request.getParameter("id")).thenReturn("sessionId")
+    
when(request.getParameter("userId")).thenReturn(ConnectUiUtils.encodeUserId("userId"))
     val tab = mock(classOf[SparkConnectServerTab], RETURNS_SMART_NULLS)
     when(tab.startTime).thenReturn(Calendar.getInstance().getTime)
     when(tab.store).thenReturn(store)
@@ -131,4 +133,58 @@ class SparkConnectServerPageSuite
       html.contains("collapse-table\" data-bs-toggle=\"collapse\"" +
         " data-bs-target=\"#aggregated-sqlsessionstat\""))
   }
+
+  test("SPARK-58097: session page only shows the requested user's operations") 
{
+    // Two users share the same session UUID, each running a distinct query.
+    kvstore = new ElementTrackingStore(new InMemoryStore, new SparkConf())
+    val listener = new SparkConnectServerListener(kvstore, new SparkConf)
+    val store = new SparkConnectServerAppStatusStore(kvstore)
+    Seq(("userA", "query from A"), ("userB", "query from B")).foreach { case 
(user, query) =>
+      val jobTag = s"jobTag-$user"
+      listener.onOtherEvent(
+        SparkListenerConnectSessionStarted("sharedSession", user, 
System.currentTimeMillis()))
+      listener.onOtherEvent(
+        SparkListenerConnectOperationStarted(
+          jobTag,
+          "operationId",
+          System.currentTimeMillis(),
+          "sharedSession",
+          user,
+          "userName",
+          query,
+          Set()))
+      listener.onOtherEvent(
+        SparkListenerConnectOperationClosed(jobTag, "operationId", 
System.currentTimeMillis()))
+    }
+
+    val request = mock(classOf[HttpServletRequest])
+    when(request.getParameter("id")).thenReturn("sharedSession")
+    
when(request.getParameter("userId")).thenReturn(ConnectUiUtils.encodeUserId("userA"))
+    val tab = mock(classOf[SparkConnectServerTab], RETURNS_SMART_NULLS)
+    when(tab.startTime).thenReturn(Calendar.getInstance().getTime)
+    when(tab.store).thenReturn(store)
+    when(tab.appName).thenReturn("testing")
+    when(tab.headerTabs).thenReturn(Seq.empty)
+    val page = new SparkConnectServerSessionPage(tab)
+    val html = page.render(request).toString().toLowerCase(Locale.ROOT)
+
+    // Only userA's operation is listed; userB's must not leak into userA's 
session page.
+    assert(html.contains("query from a"))
+    assert(!html.contains("query from b"))
+  }
+
+  test("SPARK-58097: encoded userId survives UI request sanitization") {
+    // Mirror XssSafeRequest.stripXSS: strip newlines/apostrophes, then 
HTML-escape (version 4.0).
+    val newlineAndQuote = raw"(?i)(\r\n|\n|\r|%0D%0A|%0A|%0D|'|%27)".r
+    def stripXSS(s: String): String =
+      StringEscapeUtils.escapeHtml4(newlineAndQuote.replaceAllIn(s, ""))
+
+    // Opaque user ids with characters the sanitizer would otherwise strip or 
escape.
+    Seq("a'b", "alice+tag", "user=name", "plain", "a/b&c").foreach { userId =>
+      val token = ConnectUiUtils.encodeUserId(userId)
+      // The base64url token passes through request sanitization unchanged, so 
it round-trips.
+      assert(stripXSS(token) === token, s"token for '$userId' was altered by 
sanitization")
+      assert(ConnectUiUtils.decodeUserId(stripXSS(token)) === userId)
+    }
+  }
 }
diff --git 
a/sql/connect/server/src/test/scala/org/apache/spark/status/api/v1/connect/ConnectResourceWithActualDataSuite.scala
 
b/sql/connect/server/src/test/scala/org/apache/spark/status/api/v1/connect/ConnectResourceWithActualDataSuite.scala
index 2b3c9344b1de..c350d768acf1 100644
--- 
a/sql/connect/server/src/test/scala/org/apache/spark/status/api/v1/connect/ConnectResourceWithActualDataSuite.scala
+++ 
b/sql/connect/server/src/test/scala/org/apache/spark/status/api/v1/connect/ConnectResourceWithActualDataSuite.scala
@@ -25,6 +25,7 @@ import org.json4s.jackson.JsonMethods
 
 import org.apache.spark.SparkConf
 import org.apache.spark.sql.connect.SparkConnectServerTest
+import org.apache.spark.sql.connect.service.SparkListenerConnectSessionStarted
 import org.apache.spark.sql.connect.ui.ExecutionState
 import org.apache.spark.util.Utils
 
@@ -73,6 +74,9 @@ class ConnectResourceWithActualDataSuite extends 
SparkConnectServerTest {
   private def enc(s: String): String =
     java.net.URLEncoder.encode(s, java.nio.charset.StandardCharsets.UTF_8)
 
+  private def encodeUserId(s: String): String =
+    org.apache.spark.sql.connect.ui.ConnectUiUtils.encodeUserId(s)
+
   test("Connect REST API exposes sessions and operations") {
     // Run a real query through a Connect session; this posts the 
session/operation listener
     // events that the SparkConnectServerListener writes into the KVStore.
@@ -95,8 +99,11 @@ class ConnectResourceWithActualDataSuite extends 
SparkConnectServerTest {
       assert(op.get.userId === defaultUserId)
       assert(ExecutionState.values.map(_.toString).contains(op.get.state))
 
-      val one = 
JsonMethods.parse(getOk(s"/sessions/$defaultSessionId")).extract[TestSessionData]
+      val one = JsonMethods
+        
.parse(getOk(s"/sessions/$defaultSessionId?userId=${encodeUserId(defaultUserId)}"))
+        .extract[TestSessionData]
       assert(one.sessionId === defaultSessionId)
+      assert(one.userId === defaultUserId)
 
       val oneOp = JsonMethods
         .parse(getOk(s"/operations/detail?jobTag=${enc(op.get.jobTag)}"))
@@ -106,6 +113,25 @@ class ConnectResourceWithActualDataSuite extends 
SparkConnectServerTest {
     }
   }
 
+  test("SPARK-58097: session with an empty user id is addressable via an empty 
userId token") {
+    // An empty user id is valid (the protobuf default), and its base64url 
token is the empty
+    // string. The JVM client rejects it, so post the event directly to 
populate the store.
+    val sessionId = java.util.UUID.randomUUID.toString
+    spark.sparkContext.listenerBus.post(
+      SparkListenerConnectSessionStarted(sessionId, "", 
System.currentTimeMillis()))
+    spark.sparkContext.listenerBus.waitUntilEmpty(eventuallyTimeout.toMillis)
+
+    eventuallyWithTimeout {
+      val one = JsonMethods
+        .parse(getOk(s"/sessions/$sessionId?userId="))
+        .extract[TestSessionData]
+      assert(one.sessionId === sessionId)
+      assert(one.userId === "")
+    }
+    // An absent userId parameter is still rejected.
+    assert(get(new URI(s"$baseUrl/sessions/$sessionId").toURL)._1 === 400)
+  }
+
   test("SPARK-57941: operation with a user id containing '/' is fetchable via 
the jobTag query") {
     // The job tag embeds the raw user id, so a '/' in it would break a path 
segment. The detail
     // endpoint takes the job tag as a query parameter, which handles such 
user ids.
@@ -130,7 +156,10 @@ class ConnectResourceWithActualDataSuite extends 
SparkConnectServerTest {
   }
 
   test("Connect REST API returns 404 for unknown ids") {
-    assert(get(new URI(s"$baseUrl/sessions/does-not-exist").toURL)._1 === 404)
+    assert(
+      get(
+        new URI(
+          
s"$baseUrl/sessions/does-not-exist?userId=${encodeUserId("nobody")}").toURL)._1 
=== 404)
     assert(get(new 
URI(s"$baseUrl/operations/detail?jobTag=does-not-exist").toURL)._1 === 404)
   }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to