Hyukjin Kwon created SPARK-61975:
------------------------------------

             Summary: Emit PySpark's RETRIES_EXCEEDED warning, log stop's 
release failure through the connect logger, render gRPC failures as grpcio 
does, and send PySpark's requests for the first command and the registrations
                 Key: SPARK-61975
                 URL: https://issues.apache.org/jira/browse/SPARK-61975
             Project: Spark
          Issue Type: Bug
          Components: Connect
    Affects Versions: connect-rust-4.4.0
            Reporter: Hyukjin Kwon


A follow-up of SPARK-61915, from a round-3 review of the transport against 
pyspark-client 4.3.0 (grpcio 1.85.1):

1. PySpark's [RETRIES_EXCEEDED] UserWarning (connect/client/retries.py:280-287, 
309-312) was never emitted, as the core's retry loop did not tell a failure 
that is not retryable from retries that ran out. With the server down and 
DefaultPolicy(max_retries=1), PySpark warns on count, conf.get, schema, 
interruptAll, sql, version and stop, and simplefilter("error") raises the 
warning instead of the failure. The drop-in warned on none.

2. stop() logged a release failure to logging.getLogger("pyspark"), which 
reaches stderr through logging's lastResort. PySpark logs it through the 
connect logger (connect/session.py:949-953), which is silent unless 
SPARK_CONNECT_LOG_LEVEL is set (connect/logging.py:25-50).

3. The text of a gRPC failure without status details was tonic's ("[The service 
is currently unavailable] tcp connect error"). PySpark's str(e) is grpcio's 
"<_InactiveRpcError of RPC that terminated with: status = StatusCode.X ...>" 
(core.py:2408-2418, 2456-2459).

4. The first command ExecutePlan of a session carried 
client_observed_server_side_session_id, as the drop-in looked the plan 
compression settings up before it built the request. PySpark builds the request 
first (core.py:1416, 1518-1527, 2714-2736). The udf, udtf, data-source and 
Java-function registrations went through that compression too, with its Config 
lookup, where PySpark sends them as they are (core.py:1111-1185). So an 11 MB 
UDF registration was compressed, and the first registration of a session sent a 
Config request that PySpark does not.

5. The stack-trace conf read while raising a server error was tried once, where 
PySpark's config() retries it (core.py:2071-2080).

6. The docs said that the SPARK_CONNECT_USER_AGENT environment variable is read 
"once", where newSession and cloneSession keep the parent's user agent. They 
did not say that copyFromLocalToFs and cloneSession convert errors that PySpark 
leaves as grpc.RpcError.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to