[
https://issues.apache.org/jira/browse/SPARK-43040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianhan Hu updated SPARK-43040:
-------------------------------
Description:
[https://github.com/apache/spark/pull/36726] supports TimestampNTZ type in JDBC
data source and [https://github.com/apache/spark/pull/37013] applies a fix to
pass more test cases with H2.
The problem is that Java Timestamp is a poorly defined class and different JDBC
drivers implement "getTimestamp" and "setTimestamp" with different expected
behaviors in mind. The general conversion implementation would work with some
JDBC dialects and their drivers but not others. This issue is discovered when
testing with PostgreSQL database.
We will need to have dialect specific conversions between JDBC timestamps and
TimestampNTZ.
was:
[https://github.com/apache/spark/pull/36726] supports TimestampNTZ type in JDBC
data source and [https://github.com/apache/spark/pull/37013] applies a fix to
pass more test cases with H2.
The problem is that Java Timestamp is a poorly defined class and different JDBC
drivers implement "getTimestamp" and "setTimestamp" with different expected
behaviors in mind. The general conversion implementation would work with some
JDBC dialects and their drivers but not others.
This issue is discovered when testing with PostgreSQL database.
> Improve TimestampNTZ support in JDBC data source
> ------------------------------------------------
>
> Key: SPARK-43040
> URL: https://issues.apache.org/jira/browse/SPARK-43040
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.3, 3.4.0
> Reporter: Tianhan Hu
> Priority: Major
>
> [https://github.com/apache/spark/pull/36726] supports TimestampNTZ type in
> JDBC data source and [https://github.com/apache/spark/pull/37013] applies a
> fix to pass more test cases with H2.
> The problem is that Java Timestamp is a poorly defined class and different
> JDBC drivers implement "getTimestamp" and "setTimestamp" with different
> expected behaviors in mind. The general conversion implementation would work
> with some JDBC dialects and their drivers but not others. This issue is
> discovered when testing with PostgreSQL database.
> We will need to have dialect specific conversions between JDBC timestamps and
> TimestampNTZ.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
