[
https://issues.apache.org/jira/browse/NIFI-16535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18142116#comment-18142116
]
Sebastian Kannengießer commented on NIFI-16535:
-----------------------------------------------
[https://github.com/apache/nifi/pull/11905]
[~divija21] Sorry, we are in desperate need for this fix so I opened a PR for
this.
> ParquetReader cannot handle timestamp fields with isAdjustedToUTC=false
> -----------------------------------------------------------------------
>
> Key: NIFI-16535
> URL: https://issues.apache.org/jira/browse/NIFI-16535
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: nifi-api-2.12.0
> Environment: Apache NiFi Docker Image with Hadoop and Parquet
> Nar-Bundles version 2.12.0 added
> Reporter: Sebastian Kannengießer
> Priority: Blocker
> Attachments: image-2026-11-02-14-15-56-766.png,
> nifi_local_timestamp_test.parquet
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During our migration tasks from NiFi 1.28.1 to NiFi 2.12.0 we found the
> following problem/bug with NiFi's approach to handling Parquet file
> timestamps with Type: INT64 physical type and Timestamp logical type with
> them being attibuted not being adjusted to UTC.
> {{isAdjustedToUTC=false}}
> as described here:
> [https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#local-semantics-timestamps-not-normalized-to-utc]
>
> NiFi's ParquetReader reads them as LocalDateTime types and fails casting them
> to Long:
> !image-2026-11-02-14-15-56-766.png|width=584,height=205!
> A test parquet file is attached to this ticket.
> We consider this a blocker, at least for us having Parquet-specification
> conforming data, with currently no way circumventing this easily within NiFi.
> I found the related issue [https://issues.apache.org/jira/browse/NIFI-15548]
> which was fixed in 2.8.0. That one seems to have been focusing on the
> different units (micros, millis, nanos) rather than the timestamps being
> local tz ones, though.
>
> A fix for this would be very much appreciated.
>
> Update:
> During some debugging sessions I found that the isAdjustedToUTC=false
> timestamps get converted to Avro logicalType either local-timestamp-millis,
> -micros, -nanos which are not handled yet in AvroTypeUtil.java.
> See Avro spec:
> [https://avro.apache.org/docs/1.12.0/specification/#local_timestamp]
> Also the -nanos version is not handled yet for the zoned timestamp
> logicalTypes as well.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
