zhengxiang378928908-code opened a new pull request, #18602: URL: https://github.com/apache/iceberg/pull/18602
With `immediateUpdate` disabled, a record that needs a table update gets its writer key from the table spec (or table schema id) but its raw row, which follows the record's own schema. When field IDs or column order differ, the partition source is read from the wrong position and the job fails with a `ClassCastException`. The key now uses the table's layout only when both the table schema and spec are known, converting the row first, and otherwise the record's own schema, spec and row. Unlike #17685 this leaves the main path alone: its row is already in the table layout, so keying it with the record schema breaks reordered columns (`testHashDistributionWithReorderedColumns`). Fixes #17651. Test plan: four new `TestDynamicIcebergSink` cases. `testHashDistributionWithAlternatingSchemaVariants` and `testHashDistributionToNewBranchWithDataConversion` fail on `main` with the reported `ClassCastException` and pass with this change; the other two pass on both and guard the main path. All 322 tests in `org.apache.iceberg.flink.sink.dynamic` pass on `flink/v2.3`; `spotlessCheck`, `checkstyleMain` and `checkstyleTest` pass. Backports to v1.20/v2.2 can follow once this is agreed. --- **AI Disclosure** - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: partially reviewed. The author directed the investigation and approved opening this PR, but has not yet reviewed the diff line by line. - Generated parts: the reproduction tests, the fix in `DynamicRecordProcessor` and this description were drafted with AI and run locally. Uncertainty: the stack trace in #17651 (1.10.1) points at `emit()`, not the updater path fixed here, so the reporter's exact setup (in particular `immediateUpdate`) is not confirmed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
