Hyukjin Kwon created SPARK-62132:
------------------------------------
Summary: Decode the -bin values of the metadata of a response as
gRPC core does, and fail a call of undecodable status details as grpcio does,
in the Python client
Key: SPARK-62132
URL: https://issues.apache.org/jira/browse/SPARK-62132
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 5.0.0
Reporter: Hyukjin Kwon
A follow-up of SPARK-62128. The Rust transport of the Python client
(pyspark-client-rust) decodes the -bin values of a response's metadata with a
strict base64 decoder, where gRPC core, of grpcio and pyspark-client 4.3.0,
drops every trailing '=', decodes whole groups of four and takes a last group
of two or three characters only if its unused bits are zero: so a value such as
'YWJj=' decodes to 'abc' in grpcio and fails here. And of a
grpc-status-details-bin that does not decode, gRPC core fails the call INTERNAL
"Stream removed (Error parsing 'grpc-status-details-bin' metadata: illegal
base64 encoding)" and passes the details on, empty, so PySpark's
rpc_status.from_call raises "Code in Status proto (StatusCode.OK) doesn't match
status code (StatusCode.INTERNAL)" whatever the server's code, where the
transport kept the server's code and message; and empty details ('', '=') are
read as absent where grpcio passes them on.
Expected: as grpcio, of gRPC core's decoding. Only a broken server sends such
values.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]