Hyukjin Kwon created SPARK-61080:
------------------------------------
Summary: Expose the attributes of the data types, and typeName as
a classmethod, in the Python client as PySpark does
Key: SPARK-61080
URL: https://issues.apache.org/jira/browse/SPARK-61080
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
In PySpark Connect, the data types have the attributes of their parameters, and
{{typeName}} is a classmethod:
* {{StringType("UTF8_LCASE").collation}}, {{CharType(3).length}},
{{VarcharType(4).length}}, {{TimeType(3).precision}}, the {{startField}} and
{{endField}} of {{DayTimeIntervalType}} and {{YearMonthIntervalType}}, and
{{DecimalType(10, 2).hasPrecisionInfo}}
* {{IntegerType.typeName()}} is "integer", of the name of the class
({{cls.__name__[:-4].lower()}}), also for subclasses, and "void",
"timestamp_ntz" and "interval" for {{NullType}}, {{TimestampNTZType}} and
{{CalendarIntervalType}}
* {{StructField.typeName()}} raises the {{PySparkTypeError}}
{{INVALID_TYPENAME_CALL}}
The Rust-backed Python client lacks these attributes ({{AttributeError}}),
{{typeName}} is an instance method, so {{IntegerType.typeName()}} raises a
{{TypeError}}, and {{StructField.typeName()}} raises a plain {{TypeError}}. Add
them as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
