This is an automated email from the ASF dual-hosted git repository.

dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-connect-swift.git


The following commit(s) were added to refs/heads/main by this push:
     new 571058f  [SPARK-60477] Fix `DataFrameStatFunctions.approxQuantile` to 
throw instead of crashing on empty results
571058f is described below

commit 571058fa0c9dffdcc67023aaadce6b9376ee6a8a
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Oct 22 10:03:58 2026 -0700

    [SPARK-60477] Fix `DataFrameStatFunctions.approxQuantile` to throw instead 
of crashing on empty results
    
    ### What changes were proposed in this pull request?
    
    This PR aims to fix the single-column 
`DataFrameStatFunctions.approxQuantile(_:_:_:)` to throw 
`SparkConnectError.invalidState` instead of crashing when the server returns an 
empty list of quantiles. It uses the existing `Array.firstOrThrow()` helper 
instead of `[0]`.
    
    ### Why are the changes needed?
    
    The single-column version returns the first element of the multi-column 
result with `[0]`, which has no bounds check. So, an empty result crashes the 
client process.
    
    ```
    Swift/ContiguousArrayBuffer.swift:695: Fatal error: Index out of range
    ```
    
    A Spark Connect server returns one list of quantiles per requested column 
([SparkConnectPlanner.transformStatApproxQuantile](https://github.com/apache/spark/blob/v4.2.0/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala#L576-L587)
 in [v4.2.0](https://github.com/apache/spark/releases/tag/v4.2.0) 
(2026-07-11)), so this happens only with a misbehaving or incompatible server. 
As described in SPARK-59193, such a server should cause a catchable  [...]
    
    ### Does this PR introduce _any_ user-facing change?
    
    No for normal usage. When the server returns an empty list of quantiles, 
the single-column `approxQuantile` now throws `SparkConnectError.invalidState` 
instead of crashing.
    
    ```
    The server returned an empty response for approxQuantile(_:_:_:).
    ```
    
    ### How was this patch tested?
    
    Pass the CIs. No test case is added because a Spark Connect server returns 
an empty list of quantiles only for an empty list of columns, so the changed 
code path cannot be tested without a mock server. `firstOrThrow()` on an empty 
array is covered by `ExtensionTests.firstOrThrowOnEmptyArray`, and the existing 
`DataFrameStatFunctionsTests.approxQuantile` covers the normal and empty 
`DataFrame` cases.
    
    In addition, this was verified manually against an `apache/spark:4.2.0` 
Spark Connect server with a temporary client interceptor which removes the 
column from the `StatApproxQuantile` request. The single-column 
`approxQuantile` crashed with `Fatal error: Index out of range` before this 
fix, and throws `SparkConnectError.invalidState` after this fix. 
`DataFrameStatFunctionsTests` and `ExtensionTests` also passed against 
`apache/spark:4.2.0` and `apache/spark:4.0.4-java21-scala` Spark C [...]
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Opus 5.5
    
    Closes #620 from dongjoon-hyun/SPARK-60477.
    
    Authored-by: Dongjoon Hyun <[email protected]>
    Signed-off-by: Dongjoon Hyun <[email protected]>
---
 Sources/SparkConnect/DataFrameStatFunctions.swift | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Sources/SparkConnect/DataFrameStatFunctions.swift 
b/Sources/SparkConnect/DataFrameStatFunctions.swift
index f80002e..98934ae 100644
--- a/Sources/SparkConnect/DataFrameStatFunctions.swift
+++ b/Sources/SparkConnect/DataFrameStatFunctions.swift
@@ -77,7 +77,7 @@ public actor DataFrameStatFunctions: Sendable {
   public func approxQuantile(
     _ col: String, _ probabilities: [Double], _ relativeError: Double
   ) async throws -> [Double] {
-    return try await approxQuantile([col], probabilities, relativeError)[0]
+    return try await approxQuantile([col], probabilities, 
relativeError).firstOrThrow()
   }
 
   /// Calculates the approximate quantiles of numerical columns of a 
``DataFrame``.


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to