yihua opened a new pull request, #20250: URL: https://github.com/apache/hudi/pull/20250
### Describe the issue this Pull Request addresses closes #20249 Without a transformer, the Hudi Streamer passes every source record through `LazyCastingIterator`, which runs a deep Avro `Schema.equals` per record because the target schema is never the same instance as the record schema. ### Summary and Changelog `LazyCastingIterator` now compares each distinct source schema instance with the target once and returns records of an equal schema as is, which is what `rewriteRecordDeep` already returns for equal schemas. Records of any other schema still go through `rewriteRecordDeep`. New tests in `TestLazyCastingIterator` cover equal-but-distinct schemas, a schema change mid-iterator, and a guard that the deep comparison runs once for 100 records sharing a schema (100 times on master). ### Impact Less CPU and allocation per record on Streamer ingestion without a transformer, no behavior change. Through the real class on a 30-column schema: 804 ns and 4.9 KB per record before, 34 ns after. ### Risk Level low ### Documentation Update none ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
