[
https://issues.apache.org/jira/browse/HBASE-30511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil resolved HBASE-30511.
------------------------------------------
Resolution: Fixed
Merged to master branch. Thanks for reviewing it, [~meszibalu] !
> Fix date and timestamp conversion in Spark 4 writes
> ---------------------------------------------------
>
> Key: HBASE-30511
> URL: https://issues.apache.org/jira/browse/HBASE-30511
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-connectors
> Reporter: mazhengxuan
> Assignee: Wellington Chevreuil
> Priority: Major
> Labels: pull-request-available
>
> The Spark 4 writer converts DateType values by multiplying epoch days by
> milliseconds per day, which can shift the date when it is read back in some
> time zones. Timestamp conversion also truncates negative microsecond values
> toward zero, differing from the Spark 3 conversion for values before the
> epoch.
> We should align these conversions with Spark's date and timestamp semantics
> while keeping the existing HBase storage format, with round-trip tests
> covering multiple time zones, negative timestamps and Spark 3
> interoperability.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
