kosiew commented on code in PR #26420:
URL: https://github.com/apache/datafusion/pull/26420#discussion_r4310387656
##########
datafusion/sqllogictest/test_files/parquet.slt:
##########
@@ -784,6 +784,120 @@ ERROR: Cast error: Failed to convert -9357363680509551 to
datetime for Timestamp
statement ok
drop table int96_from_spark;
+# Without coerce_int96 the timezone has no effect: same type and values as the
+# default read above.
+statement ok
+reset datafusion.execution.parquet.coerce_int96;
+
+statement ok
+CREATE EXTERNAL TABLE int96_from_spark
+STORED AS PARQUET
+LOCATION '../../parquet-testing/data/int96_from_spark.parquet';
+
+query TTT
+describe int96_from_spark;
+----
+a Timestamp(ns) YES
+
+query P
+select * from int96_from_spark
+----
+2024-01-01T20:34:56.123456
+2024-01-01T01:00:00
+1816-03-29T08:56:08.066277376
+2024-12-30T23:00:00
+NULL
+1815-11-08T16:01:01.191053312
+
+statement ok
+drop table int96_from_spark;
+
+# An explicit nanosecond unit applies the timezone.
Review Comment:
Could we add a small INT96 read test using `+08:00` to check the Arrow
timezone and an in-range timestamp rendered at that offset? The current read
tests only use UTC, so this would help catch accidental reinterpretation of the
stored instant. Please reset the setting afterward. This is optional and not a
blocker.
##########
docs/source/library-user-guide/upgrading/56.0.0.md:
##########
@@ -403,6 +403,30 @@ options.execution.time_zone =
Some("America/Denver".parse()?);
let tz: Option<&str> = options.execution.time_zone.as_ref().map(|tz|
tz.as_str());
```
+### `ParquetOptions::coerce_int96_tz` is now a validated `ConfigTimeZone`
+
+`ParquetOptions::coerce_int96_tz` changed from `Option<String>` to
+`Option<ConfigTimeZone>`, the same type as `ExecutionOptions::time_zone`. An
+invalid time zone is now rejected by `SET` and by table options instead of on
Review Comment:
Could we also mention that protobuf decoding now rejects invalid timezones,
even when `coerce_int96` is unset? This means malformed legacy options fail
during restoration rather than at read time, while valid values and the wire
format remain compatible. A short sentence would be enough, and this is not a
blocker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
