yihua opened a new pull request, #19871: URL: https://github.com/apache/hudi/pull/19871
### Describe the issue this Pull Request addresses closes #19870 `AvroSchemaEvolutionUtils.reconcileSchema` dereferences the incoming schema before the Avro NULL check its own comment promises, and `reconcileSchemaRequirements` hands a null source schema back untouched whenever the table schema has no fields. A HoodieStreamer row source whose schema provider reports no target schema, writing to a table whose latest commit carries a zero-column record schema, therefore fails every empty batch with a `NullPointerException` and never advances its checkpoint. ### Summary and Changelog An absent incoming schema now falls back to the table schema everywhere the writer schema is deduced, so an empty batch lands an empty commit and the checkpoint keeps moving. - `AvroSchemaEvolutionUtils.reconcileSchema`: treat a Java `null` like Avro NULL. - `AvroSchemaEvolutionUtils.reconcileSchemaRequirements`: check the source schema before the empty-target early return, so a null source is never returned. - `HoodieSchemaUtils.deduceWriterSchemaWithReconcile`: tolerate an absent incoming schema on the schema-on-read branch as well. - Tests: `TestAvroSchemaEvolutionUtils` (both utilities), `TestHoodieSchemaUtils` (full deduction, reconcile on and off), and `TestHoodieIncrSourceE2E` (a HoodieStreamer run over the S3 incremental source with a schema provider returning null and a zero-column table schema: the empty batch commits with the advanced checkpoint, and an unchanged checkpoint commits only with `allowCommitOnNoCheckpointChange`). Each test fails with the reported NPE without the fix. ### Impact None to public APIs. An empty batch against a table with no usable schema resolves to the table schema instead of throwing. ### Risk Level low ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
