L. C. Hsieh created SPARK-61440:
-----------------------------------

             Summary: Make the retries of the calls of the Spark Connect Rust 
client in one place, and do not panic on a stream deadline too far to be a time
                 Key: SPARK-61440
                 URL: https://issues.apache.org/jira/browse/SPARK-61440
             Project: Spark
          Issue Type: Improvement
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: L. C. Hsieh


The Rust client (crates/spark-connect-core/src/client.rs) retries its calls in 
four hand-written loops: the unary and stream-opening calls (with_retry), the 
artifact upload, the background ReleaseExecute, and the reattach of a result 
stream. Each has its own copy of the decision "can the policy retry this 
failure, is a retry left, wait its backoff", so a change to the retry policy 
has to be made four times, and the loops can drift apart.

This makes that decision in one place in retries.rs, as PySpark's Retrying: a 
retry of the policy with its backoff, and the retry made at once that PySpark 
raises as a RetryException, bounded by its elapsed time. The differences 
between the loops that are on purpose stay, as parameters: the calls of a 
client check that the client is still open before a retry and ReleaseExecute 
does not, an attempt of an upload can fail with an error that is not retried, 
and the reader of a reattachable stream shares the attempts of the policy among 
the failures of a response (SPARK-61415).

Two related fixes in the same code:

* The deadline of a stream is computed as Instant::now() + deadline at three 
places, which panics for a configured deadline too large to add to the time. A 
deadline that cannot be a time is never reached, so it is none.
* INVALID_CURSOR.DISCONNECTED and INVALID_HANDLE.OPERATION_NOT_FOUND / 
SESSION_NOT_FOUND are recognized by their name in the message of the error, as 
PySpark does. The match stays, in one named function with tests.

No other change of behavior.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to