[
https://issues.apache.org/jira/browse/SPARK-57463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-57463:
-----------------------------------
Labels: pull-request-available (was: )
> Render nanosecond-precision timestamp types in the Thrift server via the
> Types Framework
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-57463
> URL: https://issues.apache.org/jira/browse/SPARK-57463
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Priority: Major
> Labels: pull-request-available
>
> Umbrella: SPARK-56822 (Timestamps with nanosecond precision).
> Implement the Types Framework thriftTypeName hook for the nanosecond-capable
> timestamp types TIMESTAMP_NTZ(p) and TIMESTAMP_LTZ(p) (p in 7-9) so they are
> usable over the Spark Thrift / JDBC server, reaching parity with the
> microsecond TimestampType / TimestampNTZType.
> Today TimestampNanosTypeApiOps does not override TypeApiOps.thriftTypeName
> (it defaults to None). SparkExecuteStatementOperation resolves a column's
> Thrift TTypeId via TypeApiOps(typ).flatMap(_.thriftTypeName), so nanosecond
> columns have no Thrift type mapping. The reference TimeType ops returns
> Some("STRING_TYPE") (TimeTypeApiOps.thriftTypeName).
> Scope:
> - Override thriftTypeName in TimestampNanosTypeApiOps to return
> Some("STRING_TYPE") (rendered as a string at the column precision, consistent
> with cast-to-string), or the most appropriate TTypeId.
> - Verify SparkExecuteStatementOperation result-set rendering uses the
> framework format/formatUTF8 path for the value.
> - Tests under sql/hive-thriftserver covering NTZ and LTZ at precisions 7-9.
> Note: related Hive CLI rendering through the framework is tracked by
> SPARK-57386.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
