[
https://issues.apache.org/jira/browse/SPARK-57835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-57835:
-----------------------------
Shepherd: Max Gekk
> Define read behavior for persisted nanosecond-typed tables when the preview
> flag is off
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-57835
> URL: https://issues.apache.org/jira/browse/SPARK-57835
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Priority: Major
>
> This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
> precision).
> h2. Problem
> {{DataType.fromJson}} for nanosecond precision calls
> {{checkTimestampNanosTypesEnabled()}} (DataType.scala ~L255-276), so a table
> created with {{spark.sql.timestampNanosTypes.enabled=true}} cannot be loaded
> with the flag off (DataTypeSuite ~L1596-1610). This cross-flag /
> cross-version policy is undefined for real catalog tables/views.
> h2. Goal
> Define and implement a clear policy for reading a persisted nanosecond-typed
> table/view when the preview flag is off (e.g. a specific actionable error),
> covered by metastore / DESCRIBE / SHOW CREATE / view round-trip tests.
> h2. Scope
> Decide the policy; make the catalog restore path raise a clear, consistent
> error (or permit read); add e2e tests including {{DESCRIBE}}, {{SHOW CREATE
> TABLE}}, and view persistence with nanosecond columns.
> h2. Acceptance criteria
> * Loading a nanosecond table with the flag off yields the defined behavior
> with a clear message; DDL command outputs render nanosecond columns correctly.
> h2. Testing
> {{HiveExternalCatalogSuite}}, command/DDL suites.
> h2. Dependencies
> Relates to the isHiveCompatibleDataType sub-task (both touch catalog
> restore); can land independently.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
