Hyukjin Kwon created SPARK-61915:
------------------------------------
Summary: Send the user agent in the client type only, give a
connection one 20 s deadline, and keep the backoff of no maximum as grpcio and
PySpark do
Key: SPARK-61915
URL: https://issues.apache.org/jira/browse/SPARK-61915
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.4.0
Reporter: Hyukjin Kwon
Transport gaps against PySpark 4.3.0 (grpcio) from a round-2 review, following
SPARK-61847:
- The Spark user agent is sent as the HTTP user-agent header, so a user_agent
with a control character fails to connect with ValueError, where PySpark sends
grpcio's own header and carries the user agent only in client_type; a too-long
user agent raises at getOrCreate, where PySpark raises at each request built
with it.
- A direct TCP connection is abandoned after 10 s, where grpcio gives the TCP
connection, the proxy tunnel and the TLS handshake one 20 s deadline.
- The core's retry policy grows the backoff by its multiplier when max_backoff
is None, where PySpark keeps it at the initial backoff.
- A raw tab, CR or LF in the connection string fails where Python's urlsplit
removes them, and a SPARK_CONNECT_USER_AGENT that UTF-8 cannot encode is
silently replaced.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]