[
https://issues.apache.org/jira/browse/SPARK-61080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-61080:
---------------------------------
Fix Version/s: connect-rust-4.3.0
> Expose the attributes of the data types, and typeName as a classmethod, in
> the Python client as PySpark does
> ------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61080
> URL: https://issues.apache.org/jira/browse/SPARK-61080
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Fix For: connect-rust-4.3.0
>
>
> In PySpark Connect, the data types have the attributes of their parameters,
> and {{typeName}} is a classmethod:
> * {{StringType("UTF8_LCASE").collation}}, {{CharType(3).length}},
> {{VarcharType(4).length}}, {{TimeType(3).precision}}, the {{startField}} and
> {{endField}} of {{DayTimeIntervalType}} and {{YearMonthIntervalType}}, and
> {{DecimalType(10, 2).hasPrecisionInfo}}
> * {{IntegerType.typeName()}} is "integer", of the name of the class
> ({{cls.__name__[:-4].lower()}}), also for subclasses, and "void",
> "timestamp_ntz" and "interval" for {{NullType}}, {{TimestampNTZType}} and
> {{CalendarIntervalType}}
> * {{StructField.typeName()}} raises the {{PySparkTypeError}}
> {{INVALID_TYPENAME_CALL}}
> The Rust-backed Python client lacks these attributes ({{AttributeError}}),
> {{typeName}} is an instance method, so {{IntegerType.typeName()}} raises a
> {{TypeError}}, and {{StructField.typeName()}} raises a plain {{TypeError}}.
> Add them as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
