Hyukjin Kwon created SPARK-62132:
------------------------------------

             Summary: Decode the -bin values of the metadata of a response as 
gRPC core does, and fail a call of undecodable status details as grpcio does, 
in the Python client
                 Key: SPARK-62132
                 URL: https://issues.apache.org/jira/browse/SPARK-62132
             Project: Spark
          Issue Type: Bug
          Components: Connect
    Affects Versions: 5.0.0
            Reporter: Hyukjin Kwon


A follow-up of SPARK-62128. The Rust transport of the Python client 
(pyspark-client-rust) decodes the -bin values of a response's metadata with a 
strict base64 decoder, where gRPC core, of grpcio and pyspark-client 4.3.0, 
drops every trailing '=', decodes whole groups of four and takes a last group 
of two or three characters only if its unused bits are zero: so a value such as 
'YWJj=' decodes to 'abc' in grpcio and fails here. And of a 
grpc-status-details-bin that does not decode, gRPC core fails the call INTERNAL 
"Stream removed (Error parsing 'grpc-status-details-bin' metadata: illegal 
base64 encoding)" and passes the details on, empty, so PySpark's 
rpc_status.from_call raises "Code in Status proto (StatusCode.OK) doesn't match 
status code (StatusCode.INTERNAL)" whatever the server's code, where the 
transport kept the server's code and message; and empty details ('', '=') are 
read as absent where grpcio passes them on.

Expected: as grpcio, of gRPC core's decoding. Only a broken server sends such 
values.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to