Jeffrey Charles created FLINK-21350:
---------------------------------------
Summary: ParquetInputFormat incorrectly interprets timestamps
encoded in microseconds as timestamps encoded in milliseconds
Key: FLINK-21350
URL: https://issues.apache.org/jira/browse/FLINK-21350
Project: Flink
Issue Type: Bug
Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
Affects Versions: 1.12.1, 1.12.0
Reporter: Jeffrey Charles
Given a parquet file with a schema that has a field with a physical type of
INT64 and a logical type of TIMESTAMP_MICROS, all of the ParquetInputFormat
sub-classes deserialize the timestamp as tens of thousands of years in the
future.
Looking at the code in
[https://github.com/apache/flink/blob/release-1.12.1/flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/utils/RowConverter.java#L326,]
it looks to me like the row converter is interpreting the field value as if it
contained milliseconds and not microseconds. Specifically both millisecond and
microsecond processing share the same code path to instantiate a
java.sql.timestamp which takes a millisecond value in its constructor and the
microsecond case statement is passing it a value in microseconds. I tested a
change locally where I divide the value by 1000 in the microseconds case
statement and that results in a timestamp with the expected value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
