Hyukjin Kwon created SPARK-61390:
------------------------------------

             Summary: Use the default retry policy of PySpark in Spark Connect 
Rust
                 Key: SPARK-61390
                 URL: https://issues.apache.org/jira/browse/SPARK-61390
             Project: Spark
          Issue Type: Bug
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


Found in a post-merge review of the connection-layer PRs (SPARK-61319, 
SPARK-61355). RetryPolicy::default() backs off from 1 s by a factor of 2 up to 
64 s, without jitter and without the server's retry delay. PySpark 4.3.0's 
DefaultPolicy (and the Scala client's) backs off from 50 ms by a factor of 4 up 
to 60 s, adds up to 500 ms of jitter above 2 s, and honors a server-provided 
RetryInfo delay up to 10 minutes, with 15 retries.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to