Aleksandar Trajković created SPARK-60420:
--------------------------------------------
Summary: Add JDBC options to control Oracle TimestampNTZ mapping
and wall-clock conversion separately
Key: SPARK-60420
URL: https://issues.apache.org/jira/browse/SPARK-60420
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.4.0
Reporter: Aleksandar Trajković
Oracle DATE/TIMESTAMP -> TimestampNTZType mapping (SPARK-58876) and the
wall-clock LocalDateTime read/write conversion (SPARK-59865) can currently only
be switched together, through the internal session conf
{{spark.sql.legacy.oracle.timestampNTZMapping.enabled}}. Neither can be chosen
per read or write, and {{preferTimestampNTZ}} is ignored for those Oracle
columns.
This adds two dialect-agnostic JDBC options, which take precedence over the
conf:
* {{zonelessTimestampAsNTZ}}: infers zoneless timestamp types as
TimestampNTZType. Currently only honored by the Oracle dialect.
* {{timestampNTZAsWallClock}} (previously the internal
{{__timestampNTZAsWallClock}}): reads and writes TimestampNTZType values as
{{java.time.LocalDateTime}}.
Both default to the dialect's {{JdbcDialect.timestampNTZAsWallClock}}, which
for Oracle follows the legacy conf and is false for all other dialects, so
default behavior is unchanged.
To let the mapping option reach the dialect, {{JdbcDialect}} gains a
{{getCatalystType}} overload that receives the {{JDBCOptions}}. V1 reads honor
the options per read. For V2, {{JDBCTableCatalog}} declares
{{zonelessTimestampAsNTZ}} as a table-state option so a per-read value reaches
{{loadTable}}, and only pins option values the user did not set.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
