Hyukjin Kwon created SPARK-61390:
------------------------------------
Summary: Use the default retry policy of PySpark in Spark Connect
Rust
Key: SPARK-61390
URL: https://issues.apache.org/jira/browse/SPARK-61390
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
Found in a post-merge review of the connection-layer PRs (SPARK-61319,
SPARK-61355). RetryPolicy::default() backs off from 1 s by a factor of 2 up to
64 s, without jitter and without the server's retry delay. PySpark 4.3.0's
DefaultPolicy (and the Scala client's) backs off from 50 ms by a factor of 4 up
to 60 s, adds up to 500 ms of jitter above 2 s, and honors a server-provided
RetryInfo delay up to 10 minutes, with 15 retries.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]