1fanwang opened a new pull request, #29962: URL: https://github.com/apache/flink/pull/29962
## What is the purpose of the change [FLINK-41306](https://issues.apache.org/jira/browse/FLINK-41306) reports pre-1970 timestamps that make Flink reads fail. Floor division selects the correct Julian day, and floor modulo keeps nanos-of-day within one day in both timezone modes. ## Brief change log - Normalize pre-epoch INT96 values. - Add raw encoding, reader, SQL, and INT64 regression coverage. ## Verifying this change ```bash export JAVA_HOME=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home ./mvnw -pl flink-formats/flink-parquet -Dtest=ParquetRowDataWriterTest test ./mvnw -pl flink-formats/flink-parquet -Dtest=ParquetFileSystemITCase#testPreEpochTimestamp -DfailIfNoTests=false test ``` ```text UTC=false: -28798750000000L (expected 0..86399999999999) UTC=true: -43199750000000L (expected 0..86399999999999) Caused by: java.lang.IllegalArgumentException: nanos > 999999999 or < 0 ``` After the fix: ```bash ./mvnw -pl flink-formats/flink-parquet -Dtest=ParquetRowDataWriterTest test ./mvnw -pl flink-formats/flink-parquet -Dtest=ParquetFileSystemITCase#testPreEpochTimestamp -DfailIfNoTests=false test ``` ```text UTC=false: julianDay=2437101 nanosOfDay=57600250000000 UTC=true: julianDay=2437101 nanosOfDay=43200250000000 Rows: [+I[1, 1960-06-15T12:00], +I[2, 1960-06-15T12:00:00.250], +I[3, 1980-06-15T12:00:00.250]] ``` <details> <summary>Reproducer source: ParquetFileSystemITCase.java</summary> ```sql CREATE TABLE t (id INT, ts TIMESTAMP(3)) WITH ('connector'='filesystem', 'path'='<temporary path>', 'format'='parquet'); INSERT INTO t VALUES (1, TIMESTAMP '1960-06-15 12:00:00.000'), (2, TIMESTAMP '1960-06-15 12:00:00.250'), (3, TIMESTAMP '1980-06-15 12:00:00.250'); SELECT * FROM t; ``` </details> ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with @Public(Evolving): no - The serializers: yes - The runtime per-record code paths (performance sensitive): yes - Anything that affects deployment or recovery: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: GitHub Copilot CLI 1.0.110 (Auto) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
