Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19769#discussion_r151547738
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -284,6 +284,13 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val PARQUET_INT96_TIMESTAMP_CONVERSION =
buildConf("spark.sql.parquet.int96TimestampConversion")
+ .doc("This controls whether timestamp adjustments should be applied to
INT96 data when " +
+ "converting to timestamps, for data written by Impala. This is
necessary because Impala" +
+ "stores INT96 data with a different offset than Hive & Spark.")
--- End diff --
`offset` -> `timezone offset`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
