mazhengxuan created HBASE-30511:
-----------------------------------
Summary: Fix date and timestamp conversion in Spark 4 writes
Key: HBASE-30511
URL: https://issues.apache.org/jira/browse/HBASE-30511
Project: HBase
Issue Type: Sub-task
Components: hbase-connectors
Reporter: mazhengxuan
The Spark 4 writer converts DateType values by multiplying epoch days by
milliseconds per day, which can shift the date when it is read back in some
time zones. Timestamp conversion also truncates negative microsecond values
toward zero, differing from the Spark 3 conversion for values before the epoch.
We should align these conversions with Spark's date and timestamp semantics
while keeping the existing HBase storage format, with round-trip tests covering
multiple time zones, negative timestamps and Spark 3 interoperability.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
