tkaymak opened a new pull request, #38272:
URL: https://github.com/apache/beam/pull/38272
## Summary
`runners/flink/flink_runner.gradle` resolves `previous_versions` (the list
of older Flink majors whose source-overrides should be merged into the
current build) via lexicographic string comparison against `flink_major`:
```groovy
def previous_versions = all_versions.findAll { it < flink_major }
```
This breaks for the current `flink_versions=1.17,1.18,1.19,1.20,2.0` because
lexicographic ordering disagrees with semantic ordering whenever a two-digit
minor crosses a single-digit boundary.
## Impact today
| `flink_major` | Computed `previous_versions` | Correct |
|---|---|---|
| `1.17` | `[]` | `[]` ✓ |
| `1.18` | `["1.17", "1.20"]` ❌ | `["1.17"]` |
| `1.19` | `["1.17", "1.18", "1.20"]` ❌ | `["1.17", "1.18"]` |
| `1.20` | `[]` ❌ | `["1.17", "1.18", "1.19"]` |
| `2.0` | `["1.17", "1.18", "1.19", "1.20"]` | same ✓ |
Concretely:
- **1.18 and 1.19 builds erroneously pull in
`runners/flink/1.20/src/main/java/.../DoFnOperator.java`** as a "previous"
override. If that file relies on Flink 1.20+ APIs, those builds would fail to
compile.
- **1.20 builds drop the 1.19 test overrides** (`MemoryStateBackendWrapper`,
`StreamSources`). Both classes also exist in the shared base, so 1.20's tests
use the shared-base versions today; whether that's correct depends on whether
the 1.19 overrides exist precisely because the shared-base versions don't
compile against the 1.19+ test API.
## Fix
Mirror the Spark approach landed in #38233
(`runners/spark/spark_runner.gradle`):
- Compute `flink_major_index = all_versions.indexOf(flink_major)` and derive
`previous_versions` as `all_versions.subList(0, flink_major_index)`.
- Throw a clear `GradleException` if `flink_major` isn't listed in
`flink_versions` (today the buggy path silently produced `[]`).
- Update `use_override` to `(flink_major_index > 0)`, matching the Spark
formulation.
- Trim whitespace from each `flink_versions` entry, defensive against
`flink_versions=1.17, 1.18, ...` style.
## Context
Promised follow-up to @Abacn from PR #38233 review (issue comment
2026-04-17), where the same lex-comparison bug was flagged in
`spark_runner.gradle` and asked to be split into a separate PR for the Flink
side.
## Test plan
- [ ] CI green on all five Flink versions (1.17, 1.18, 1.19, 1.20, 2.0)
- [ ] `./gradlew :runners:flink:1.20:compileJava` (pre-fix would miss
1.17/1.18/1.19 overrides; post-fix should pull them in)
- [ ] `./gradlew :runners:flink:1.18:compileJava` (pre-fix erroneously
pulled 1.20 override; post-fix should not)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
