dongjoon-hyun opened a new pull request, #475:
URL: https://github.com/apache/spark-connect-swift/pull/475
### What changes were proposed in this pull request?
This PR supports automatic retries with exponential backoff for RPC calls in
`SparkConnectClient`.
1. A new `RetryPolicy` implements the same default policy as the Scala side
`org.apache.spark.sql.connect.client.RetryPolicy` and the Python side
`pyspark.sql.connect.client.retries.DefaultPolicy`: `maxRetries=15`,
`initialBackoff=50ms`,
`maxBackoff=60s`, `backoffMultiplier=4.0`, `jitter=500ms`,
`minJitterThreshold=2s`, and
`maxServerRetryDelay=10min`, guaranteeing a maximum tolerated wait of at
least 10 minutes.
2. Retriable errors are `UNAVAILABLE`, `INTERNAL` caused by
`INVALID_CURSOR.DISCONNECTED`, and
any error containing `google.rpc.RetryInfo` in its gRPC status details. A
server-provided
`RetryInfo.retry_delay` overrides the client's backoff, limited by
`maxServerRetryDelay`.
3. A `withRetry` helper wraps both `withGPRC` implementations, so every
non-streaming RPC
(`AnalyzePlan`, `Config`, `Interrupt`, `GetStatus`, artifact RPCs, etc.)
is retried with a
fresh channel per attempt. `Task` cancellation stops the retry loop
immediately, and the last
error is rethrown when the retries are exhausted.
4. Since `ExecutePlan` can have side effects on the server, it is retried
only until the first
response arrives (the operation is not started before that). A complete
solution is
`ReattachExecute`, which is a follow-up.
### Why are the changes needed?
Every RPC call failed immediately on transient network errors, server
restarts, or `UNAVAILABLE`
responses. The PySpark and Scala Spark Connect clients retry these errors by
default, and this PR
brings the Swift client to parity with their `DefaultPolicy`.
### Does this PR introduce _any_ user-facing change?
Yes, transient RPC failures are now retried automatically for up to 15
attempts instead of
failing immediately. Non-retriable errors are still thrown as before.
### How was this patch tested?
Pass the CIs with the newly added `RetryPolicyTests` test suite covering the
retriable-error
classification, the exponential backoff sequence and its cap, the
server-provided retry delay and
its limit, the retry loop attempt counts, and the cancellation propagation.
All existing test
suites pass with a live Spark Connect server.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Fable 5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
