Sebastian Kannengießer created NIFI-16535:
---------------------------------------------
Summary: ParquetReader cannot handle timestamp fields with
isAdjustedToUTC=false
Key: NIFI-16535
URL: https://issues.apache.org/jira/browse/NIFI-16535
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: nifi-api-2.12.0
Environment: Apache NiFi Docker Image with Hadoop and Parquet
Nar-Bundles version 2.12.0 added
Reporter: Sebastian Kannengießer
Attachments: image-2026-11-02-14-15-56-766.png,
nifi_local_timestamp_test.parquet
During our migration tasks from NiFi 1.28.1 to NiFi 2.12.0 we found the
following problem/bug with NiFi's approach to handling Parquet file timestamps
with Type: INT64 physical type and Timestamp logical type with them being
attibuted not being not adjusted to UTC.
{{isAdjustedToUTC=false}}
as described here:
[https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#local-semantics-timestamps-not-normalized-to-utc]
NiFi's ParquetReader reads them as LocalDateTime types and fails casting them
to Long:
!image-2026-11-02-14-15-56-766.png|width=584,height=205!
A test parquet file is attached to this ticket.
We consider this a blocker, at least for us having Parquet-specification
conforming data, with currently no way circumventing this easily within NiFi.
I found the relating issue https://issues.apache.org/jira/browse/NIFI-15548
which was fixed in 2.8.0. That one seems to have been focusing on the different
units (micros, millis, nanos) rather than the timestamps being a local one,
though.
A fix for this would be very much appreciated.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
