jaideeppyne commented on code in PR #3237:
URL: https://github.com/apache/iceberg-rust/pull/3237#discussion_r4475315621
##########
crates/iceberg/src/transform/temporal.rs:
##########
@@ -67,6 +90,7 @@ impl Year {
impl TransformFunction for Year {
fn transform(&self, input: ArrayRef) -> Result<ArrayRef> {
+ let input = timestamp_array_in_utc(input);
Review Comment:
Renamed the retagged binding to `utc_input`, preserving the distinction from
the function input. Published in 3998cc7.
##########
crates/iceberg/src/transform/temporal.rs:
##########
@@ -155,6 +179,7 @@ impl Month {
impl TransformFunction for Month {
fn transform(&self, input: ArrayRef) -> Result<ArrayRef> {
+ let input = timestamp_array_in_utc(input);
Review Comment:
The month transform also uses `utc_input` for both `date_part` calls.
Published in 3998cc7.
##########
crates/iceberg/src/transform/temporal.rs:
##########
@@ -41,6 +41,29 @@ const MICROS_PER_SECOND: i64 = 1_000_000;
/// One second in nanos.
const NANOS_PER_SECOND: i64 = 1_000_000_000;
+/// Re-tag timestamp arrays as UTC so calendar fields match Iceberg's
+/// epoch-based computation. `date_part` converts into the array's timezone
+/// tag before reading year/month; Iceberg always uses UTC.
+fn timestamp_array_in_utc(input: ArrayRef) -> ArrayRef {
+ match input.data_type() {
+ DataType::Timestamp(TimeUnit::Microsecond, Some(_)) => {
+ let arr = input
+ .as_any()
+ .downcast_ref::<TimestampMicrosecondArray>()
+ .unwrap();
+ Arc::new(arr.clone().with_timezone("+00:00"))
+ }
+ DataType::Timestamp(TimeUnit::Nanosecond, Some(_)) => {
+ let arr = input
+ .as_any()
+ .downcast_ref::<TimestampNanosecondArray>()
+ .unwrap();
+ Arc::new(arr.clone().with_timezone("+00:00"))
+ }
+ _ => input,
+ }
+}
Review Comment:
The helper now matches every `Timestamp(unit, Some(_))` and uses the
existing Arrow cast with `UTC_TIME_ZONE`. The regression exercises seconds,
milliseconds, microseconds, and nanoseconds, so public callers receive the same
UTC behavior. Published in 3998cc7; [39 temporal tests, 2,121 library tests and
Clippy
passed](https://github.com/jaideeppyne/iceberg-rust/actions/runs/41944243552).
##########
crates/iceberg/src/transform/temporal.rs:
##########
@@ -2374,6 +2399,21 @@ mod test {
assert_eq!(res.value(4), -1);
}
+ #[test]
+ fn test_year_and_month_ignore_non_utc_timezone_tag() {
+ // -1 micros is 1969-12-31T23:59:59.999999Z. A non-UTC tag must not
+ // shift that into 1970 when computing Iceberg year/month values.
+ let tagged: ArrayRef =
+
Arc::new(TimestampMicrosecondArray::from(vec![-1i64]).with_timezone("Asia/Kathmandu"));
+ let years = super::Year.transform(tagged.clone()).unwrap();
+ let years = years.as_any().downcast_ref::<Int32Array>().unwrap();
+ assert_eq!(years.value(0), -1);
+
+ let months = super::Month.transform(tagged).unwrap();
+ let months = months.as_any().downcast_ref::<Int32Array>().unwrap();
+ assert_eq!(months.value(0), -1);
+ }
Review Comment:
The regression now uses `+05:45`, `-05:00`, and `+00:00`, plus an untagged
control, across all four timestamp units. It checks both sides of the epoch and
preserves nulls, without requiring named-timezone data. Published in 3998cc7;
[hosted tests and Clippy
passed](https://github.com/jaideeppyne/iceberg-rust/actions/runs/41944243552).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
