[ 
https://issues.apache.org/jira/browse/SPARK-61915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hyukjin Kwon updated SPARK-61915:
---------------------------------
    Fix Version/s: connect-rust-4.4.0

> Send the user agent in the client type only, give a connection one 20 s 
> deadline, and keep the backoff of no maximum as grpcio and PySpark do
> ---------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-61915
>                 URL: https://issues.apache.org/jira/browse/SPARK-61915
>             Project: Spark
>          Issue Type: Bug
>          Components: Connect
>    Affects Versions: connect-rust-4.4.0
>            Reporter: Hyukjin Kwon
>            Priority: Major
>             Fix For: connect-rust-4.4.0
>
>
> Transport gaps against PySpark 4.3.0 (grpcio) from a round-2 review, 
> following SPARK-61847:
> - The Spark user agent is sent as the HTTP user-agent header, so a user_agent 
> with a control character fails to connect with ValueError, where PySpark 
> sends grpcio's own header and carries the user agent only in client_type; a 
> too-long user agent raises at getOrCreate, where PySpark raises at each 
> request built with it.
> - A direct TCP connection is abandoned after 10 s, where grpcio gives the TCP 
> connection, the proxy tunnel and the TLS handshake one 20 s deadline.
> - The core's retry policy grows the backoff by its multiplier when 
> max_backoff is None, where PySpark keeps it at the initial backoff.
> - A raw tab, CR or LF in the connection string fails where Python's urlsplit 
> removes them, and a SPARK_CONNECT_USER_AGENT that UTF-8 cannot encode is 
> silently replaced.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to