jubins opened a new pull request, #29948: URL: https://github.com/apache/flink/pull/29948
## What is the purpose of the change Fixes [FLINK-41499](https://issues.apache.org/jira/browse/FLINK-41499). `DateTimeUtils#unixTimestampFloor` (used for `TIMESTAMP`) and `DateTimeUtils#timestampFloor` (used for `TIMESTAMP_LTZ`) compute the epoch day with Java integer division (`(int) (millis / MILLIS_PER_DAY)`), which rounds toward zero. A timestamp before 1970 that is not at midnight is therefore mapped to the *following* day before the Julian floor is applied. `FLOOR(... TO WEEK | MONTH | QUARTER | YEAR ...)` then returns a value that can be later than its input, with no error. `FLOOR ... TO DAY` and smaller units use a different path and are not affected. ```sql SELECT FLOOR(TIMESTAMP '1969-12-31 10:00:00' TO MONTH); -- before: 1970-01-01 00:00:00 expected: 1969-12-01 00:00:00 ``` [FLINK-41280](https://issues.apache.org/jira/browse/FLINK-41280) fixed the same round-toward-zero problem for `CAST` of a `TIMESTAMP` before 1970; `FLOOR` was not covered. This PR changes only `FLOOR`. `CEIL` is intentionally left alone: it returns correct results for these inputs today, and its day-1 behavior is tracked by the CALCITE-3199 TODO in `FunctionGenerator`. ## Brief change log - `DateTimeUtils#timestampFloor`: use `Math.floorDiv(utcTs, MILLIS_PER_DAY)` for the day number of the `YEAR`, `QUARTER`, `WEEK` and `MONTH` branches. - `DateTimeUtils#unixTimestampFloor`: use `Math.floorDiv(timestamp, MILLIS_PER_DAY)` for the day number. - `TimeFunctionsITCase`: add a `FLOOR` test spec for timestamps before 1970. ## Verifying this change This change added tests and can be verified as follows: - Added a `FLOOR` / `TIMESTAMP before 1970` spec to `TimeFunctionsITCase#floorTestCases`. It reads the timestamps from table columns (so the expressions run as generated code and are not constant-folded) and checks `FLOOR` to `WEEK`, `MONTH`, `QUARTER` and `YEAR` for `TIMESTAMP(3)`, and the same units for `CAST(... AS TIMESTAMP_LTZ(3))`. The inputs are the last day of a week-, month-, quarter- and year-end in 1966, which rounding toward zero pushes into the next period. - Without the fix the new cases fail, for example `FLOOR(1966-12-31 10:00 TO WEEK)` returns `1967-01-01T00:00` instead of `1966-12-25T00:00`, and `FLOOR(1966-09-30 10:00 TO QUARTER)` returns `1966-10-01T00:00` instead of `1966-07-01T00:00`. With the fix the whole `TimeFunctionsITCase` (538 tests) passes locally on JDK 21 with the JVM time zone set to UTC, America/Los_Angeles and Europe/London. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): yes, but only by replacing an integer division with `Math.floorDiv` in `FLOOR` for the year, quarter, month and week units - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
