[ 
https://issues.apache.org/jira/browse/SPARK-62132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hyukjin Kwon reassigned SPARK-62132:
------------------------------------

    Assignee: Hyukjin Kwon

> Decode the -bin values of the metadata of a response as gRPC core does, and 
> fail a call of undecodable status details as grpcio does, in the Python client
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-62132
>                 URL: https://issues.apache.org/jira/browse/SPARK-62132
>             Project: Spark
>          Issue Type: Bug
>          Components: Connect
>    Affects Versions: 5.0.0
>            Reporter: Hyukjin Kwon
>            Assignee: Hyukjin Kwon
>            Priority: Major
>              Labels: pull-request-available
>
> A follow-up of SPARK-62128. The Rust transport of the Python client 
> (pyspark-client-rust) decodes the -bin values of a response's metadata with a 
> strict base64 decoder, where gRPC core, of grpcio and pyspark-client 4.3.0, 
> drops every trailing '=', decodes whole groups of four and takes a last group 
> of two or three characters only if its unused bits are zero: so a value such 
> as 'YWJj=' decodes to 'abc' in grpcio and fails here. And of a 
> grpc-status-details-bin that does not decode, gRPC core fails the call 
> INTERNAL "Stream removed (Error parsing 'grpc-status-details-bin' metadata: 
> illegal base64 encoding)" and passes the details on, empty, so PySpark's 
> rpc_status.from_call raises "Code in Status proto (StatusCode.OK) doesn't 
> match status code (StatusCode.INTERNAL)" whatever the server's code, where 
> the transport kept the server's code and message; and empty details ('', '=') 
> are read as absent where grpcio passes them on.
> Expected: as grpcio, of gRPC core's decoding. Only a broken server sends such 
> values.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to