Kaicheng Zhou created SPARK-60354:
-------------------------------------
Summary: FieldDescriptor.label is being deprecated
Key: SPARK-60354
URL: https://issues.apache.org/jira/browse/SPARK-60354
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 5.0.0, 4.4.0
Reporter: Kaicheng Zhou
protobuf 7.34 removed {{FieldDescriptor.label}} (deprecated since 6.32 in favor
of {{{}is_repeated{}}}). The latest {{grpcio-status}} and
{{googleapis-common-protos}} allow {{{}protobuf<8{}}}, so a fresh PySpark
Connect client install now gets protobuf 7.
{{SparkConnectClient._truncate}} in
{{python/pyspark/sql/connect/client/core.py}} checks {{{}descriptor.label ==
descriptor.LABEL_REPEATED{}}}. On protobuf 7 it raises {{{}AttributeError{}}},
which {{_proto_to_string}} catches, so every "Executing plan" / "Schema for
plan" / "Explain" debug log shows {{<Truncated message due to truncation
error>}} instead of the plan. On protobuf 6.32–6.33 the same code emits a
{{{}DeprecationWarning{}}}.
Fix: use {{{}descriptor.is_repeated{}}}, available since protobuf 6.31.0.
PySpark already requires {{{}grpcio-status>=1.76.0{}}}, which needs
{{{}protobuf>=6.31.1{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
