cloud-fan commented on a change in pull request #34647:
URL: https://github.com/apache/spark/pull/34647#discussion_r754252588
##########
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##########
@@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase extends QueryTest with
SQLTestUtils with TestHi
}
}
}
+
+ test("SPARK-36180: Support TimestampNTZ type in Hive") {
+ withTable("tb") {
+ val dt = "2018-11-17 13:33:33.0"
+ val ddl =
+ s"CREATE TABLE tb as SELECT TIMESTAMP_LTZ'$dt' as c0, TIMESTAMP_NTZ
'$dt' as c1"
Review comment:
Hive serde table is very special, as it includes both data source and
metastore. For metastore, the current decision is to store the schema as it is
in a hive-incompatible way. For the data source part, this PR proposes to store
NTZ as Hive's timestamp.
I think there are 2 options
1. do not support NTZ at all for hive serde tables
2. revisit the decision for metastore. We can still store table with ntz in
a hive-compatible way, by storing both ntz and ltz as hive timestamp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
