developer-rpai opened a new pull request, #18357: URL: https://github.com/apache/iceberg/pull/18357
Fixes #17650. **Root cause.** `CompareSchemasVisitor.struct()` computes the merged field result as: ```java fields.stream().reduce(Result::merge).orElse(Result.SCHEMA_UPDATE_NEEDED) ``` For a zero-field struct the reduce is empty, so the fallback fires: any schema containing an empty struct (for example an Avro zero-field marker record) can never compare `SAME` against any table schema, not even an identical one. In the Dynamic Sink that turns into a commit/re-compare loop and ends in `TableMetadataCache` caching `NOT_FOUND`, whose null schema then NPEs in `DynamicSinkUtil.getEqualityFieldIds`. **Fix.** `SAME` is the identity element of `Result.merge` (`merge` takes the max severity), so an empty struct contributes `SAME` and the existing table-side checks (field counts, name order, required-field coverage) still catch real differences. One-line change in each of the three Flink versions (1.20, 2.2, 2.3), whose copies of this class are identical. **Tests.** Added three regression tests to `TestCompareSchemasVisitor` in each version: - `testEmptyStructSame`: identical schemas with an empty struct field compare `SAME` - `testNestedEmptyStructSame`: same for an empty struct nested in another struct - `testEmptyStructAgainstNonEmptyTableStruct`: empty input struct vs a table struct with an extra optional field still yields `DATA_CONVERSION_NEEDED` All 26 tests in `TestCompareSchemasVisitor` pass (23 existing plus the 3 new). Disclosure: prepared with AI assistance; the change was compiled and unit-tested before opening. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
