Github user squito commented on the issue: https://github.com/apache/spark/pull/19702 ok sorry for taking a bit of time to look at this. tl;dr I think what you are proposing here is fine. So to be honest, I'm a bit confused about the parquet versioning -- the types I was mentioned were added here: https://github.com/apache/parquet-format/commit/863875e0be3237c6aa4ed71733d54c91a51deabe#diff-0f9d1b5347959e15259da7ba8f4b6252 to parquet-format v2.4.0 I can't really tell what version of parquet-mr that corresponds to. I can at least see that parquet-mr v1.8.2 doesn't have the new types. I don't even think its master. But IIUC, the latest version of parquet-format introduces new types, independent of the old TIMESTAMP_MILLIS and TIMESTAMP_MICROS. So there won't be any compatibility issues when `isAdjustedToUTC` is added, as that will be in an entirely new type.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org