laserninja opened a new pull request, #18112: URL: https://github.com/apache/iceberg/pull/18112
Fixes #18111. Truncating `Integer.MAX_VALUE - 1` with width `Integer.MAX_VALUE` currently produces a negative partition key. A scan filtered by `id >= 0` then omits the matching file. Use `Math.floorMod` in the int and narrow-integer helpers to avoid overflowing the intermediate remainder-plus-width addition. This preserves public APIs and existing behavior when the final mathematical result is outside the target type's range. Validation on JDK 17: - Full `:iceberg-api:test` suite and targeted core truncation and table scan regressions passed. The new regressions failed before the fix. - `:iceberg-api:spotlessCheck :iceberg-core:spotlessCheck` passed. - `:iceberg-api:revapi :iceberg-core:revapi` passed. Spark SQL regression (Spark 4.1): create an integer table partitioned by `truncate(2147483647, id)`, insert `2147483646`, and query `WHERE id >= 0`. On the original implementation the query returns no rows in Parquet, Avro, and ORC; with the fix it returns the inserted row in all three formats. The new Spark test, formatting, and Spark test checkstyle pass locally on JDK 17. Additional local validation at `28aff7bfb`: the full CI all-module build command (`./gradlew -DallModules build -x test -x javadoc -x integrationTest`) passed on JDK 17, including formatting and style checks across the engine versions. Apache RAT (`dev/check-license`), `./gradlew checkAllRuntimeDeps -q -DallModules=true`, and `./gradlew -Pquick=true javadoc` also passed. This build excludes tests; the Spark regression results above are from a separate test run. Review follow-up: the same Spark SQL regression now covers Spark 3.5 (Scala 2.12), 4.0, and 4.2 (Scala 2.13), in addition to 4.1. All nine added version/format cases passed locally on JDK 17, together with `spotlessCheck` and `checkstyleTest` for the three changed Spark modules. --- **AI Disclosure** - Model: GPT-6 - Platform/Tool: Codex - Human Oversight: reviewed - Prompt Summary: Reproduce and fix integer truncation overflow, add regression tests, and create an issue and PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
