[
https://issues.apache.org/jira/browse/SPARK-61080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-61080:
-----------------------------------
Labels: pull-request-available (was: )
> Expose the attributes of the data types, typeName as a classmethod, and the
> singleton types in the Python client as PySpark does
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61080
> URL: https://issues.apache.org/jira/browse/SPARK-61080
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> In PySpark Connect, the data types have the attributes of their parameters,
> {{typeName}} is a classmethod, and the types without parameters are
> singletons:
> * {{StringType("UTF8_LCASE").collation}}, {{CharType(3).length}},
> {{VarcharType(4).length}}, {{TimeType(3).precision}}, the {{startField}} and
> {{endField}} of {{DayTimeIntervalType}} and {{YearMonthIntervalType}}, and
> {{DecimalType(10, 2).hasPrecisionInfo}}
> * {{IntegerType.typeName()}} is "integer", of the name of the class
> ({{cls.__name__[:-4].lower()}}), also for subclasses of {{DataType}}, and
> "void", "timestamp_ntz" and "interval" for {{NullType}}, {{TimestampNTZType}}
> and {{CalendarIntervalType}}
> * {{StructField.typeName()}} raises the {{PySparkTypeError}}
> {{INVALID_TYPENAME_CALL}}
> * {{IntegerType() is IntegerType()}}, as for the other types of
> {{DataTypeSingleton}}: {{NullType}}, the integral types, {{FloatType}},
> {{DoubleType}}, {{BinaryType}}, {{BooleanType}}, {{DateType}},
> {{TimestampType}}, {{TimestampNTZType}} and {{CalendarIntervalType}}
> The Rust-backed Python client lacks these attributes ({{AttributeError}}).
> Its {{typeName}} is an instance method, so {{IntegerType.typeName()}} raises
> a {{TypeError}}, and {{CalendarIntervalType().typeName()}} is
> "calendarinterval". {{StructField.typeName()}} raises a plain {{TypeError}},
> and every call of a type class returns a new object. Add them as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
