[
https://issues.apache.org/jira/browse/SPARK-61134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-61134.
----------------------------------
Resolution: Fixed
Issue resolved by pull request 329
[https://github.com/apache/spark-connect-rust/pull/329]
> Make DayTimeIntervalType and YearMonthIntervalType check their fields, and
> the types without parameters their arguments, in the Python client as PySpark
> does
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61134
> URL: https://issues.apache.org/jira/browse/SPARK-61134
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Assignee: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> PySpark's DayTimeIntervalType(startField, endField) and
> YearMonthIntervalType(startField, endField) raise PySparkRuntimeError
> INVALID_INTERVAL_CASTING, with start_field and end_field, for a field that is
> not one of theirs. For example,
> DayTimeIntervalType(endField=DayTimeIntervalType.SECOND) raises with
> start_field "None", and DayTimeIntervalType(123) with "123". The client
> accepts them: DayTimeIntervalType(endField=3) is DayTimeIntervalType(0, 3).
> PySpark's types without parameters, NullType through CalendarIntervalType,
> are the singletons of the DataTypeSingleton metaclass. Its __call__ raises
> TypeError "DataTypeSingleton.__call__() takes 1 positional argument but 2
> were given" for an argument, and "got an unexpected keyword argument" for a
> keyword. The client's raise "CalendarIntervalType.__new__() takes 0
> positional arguments but 1 was given".
> Apache Spark's test_daytime_interval_type_constructor,
> test_yearmonth_interval_type_constructor and
> test_calendar_interval_type_constructor fail on this client. The client
> should check the fields and arguments of these types as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
