[
https://issues.apache.org/jira/browse/SPARK-51413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17947464#comment-17947464
]
Vinod KC edited comment on SPARK-51413 at 4/26/25 3:08 AM:
-----------------------------------------------------------
[~maxgekk] , I have a question regarding the conversion from Spark Time to
Arrow Time and then from Arrow Time back to Spark Time.
Supported time units for Arrow Time Type: SECOND, MILLISECOND, MICROSECOND,
NANOSECOND.
Supported time precisions for Spark Time Type: 0 -> SECOND, 1 -> ?, 2 -> ?, 3:
MILLISECOND, 4 -> ?, 5 -> ?, 6: MICROSECOND.
!image-2025-04-25-19-53-08-993.png!
Do you have any suggestions for managing the precision impact on this
conversion?
was (Author: vinodkc):
[~maxgekk] , I have a question regarding the conversion from Spark Time to
Arrow Time and then from Arrow Time back to Spark Time.
Supported time units for Arrow Time Type: SECOND, MILLISECOND, MICROSECOND,
NANOSECOND.
Supported time precisions for Spark Time Type: 0 -> SECOND, 1 -> None, 2 ->
None, 3: MILLISECOND, 4 -> None, 5 -> None, 6: MICROSECOND.
!image-2025-04-25-19-53-08-993.png!
Do you have any suggestions for managing the precision impact on this
conversion?
> Support TIME in Arrow
> ---------------------
>
> Key: SPARK-51413
> URL: https://issues.apache.org/jira/browse/SPARK-51413
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: Max Gekk
> Priority: Major
> Attachments: image-2025-04-25-19-53-08-993.png
>
>
> Add the support of TimeType in Arrow APIs. See Arrow spec:
> https://github.com/apache/arrow/blob/main/format/Schema.fbs#L256-L275
> and as an example similar changes for TimestampNTZ:
> https://github.com/apache/spark/pull/33875
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
