voonhous opened a new pull request, #19707: URL: https://github.com/apache/hudi/pull/19707
### Describe the issue this Pull Request addresses `hudi-utilities` compile-depends on `com.amazonaws:amazon-kinesis-deaggregator`, whose Amazon Software License is ASF Category X: it may not be a required dependency of an Apache project. Introduced by #18224, in no release yet, but it blocks the next release cut from master. Its classes are also absent from every bundle, so the default read path (`enable.deaggregation=true`) throws `NoClassDefFoundError` in bundle-only deployments, even for non-aggregated records. ### Summary and Changelog - Rewrite `KinesisDeaggregator` to decode the documented KPL format (magic prefix + protobuf payload + trailing MD5) directly with `protobuf-java`, already a direct dependency; remove the ASL dependency. - Remove the now-dead AWS v1 SDK BOM pin (root pom) and the dead KCL/deaggregator includes (`hudi-aws-bundle`); nothing else uses SDK v1. - Differential-tested against the real KCL deaggregator on 17 fixture classes: byte-identical output, with one deliberate deviation -- on a corrupt aggregate (valid digest, out-of-range key index) KCL keeps the sub-records before the bad one and silently drops the rest; the raw record now passes through whole and fails loudly downstream instead. ### Impact None for valid input. Two incidental fixes: non-aggregated records keep `encryptionType` (the old SDK v1 round-trip dropped it), and deaggregation no longer needs `javax.xml.bind`, which KCL 1.8.8 requires but JDK 11+ removed. ### Risk Level Low. Differential-tested against KCL; 12 new unit tests plus an iterator-level test with deaggregation enabled. ### Documentation Update None. No config, API, or packaging contract change. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
