suryaprasanna opened a new pull request, #19227: URL: https://github.com/apache/hudi/pull/19227
### Describe the issue this Pull Request addresses Hudi has no built-in tool to create a "shadow" copy of a production Copy-on-Write table by copying just the base files and skipping the record level iterations and keep it incrementally in sync with the source. Such a shadow is useful for safely exercising table-service, index, or upgrade changes against production-like data without touching the source table. This PR adds `HoodieShadowPipeline` to `hudi-utilities` to address that. ### Summary and Changelog Adds `HoodieShadowPipeline`, a Spark utility that bootstraps a destination COW table by checksum-copying the source base files and building a matching commit, then keeps it in sync via `HoodieDeltaStreamer`/`HoodieIncrSource` under a Zookeeper-based lock. - New package `org.apache.hudi.utilities.pipeline`: `HoodieShadowPipeline`, `HoodieShadowPipelineConfig`, and JCommander converter helpers (`StringToListParameterConverter`, `StringToSetConverter`). - Functional tests: `TestHoodieShadowPipeline` with a `HoodieToolsFunctionalTest` base harness. ### Impact Additive only — a new opt-in CLI tool in `hudi-utilities`. No changes to existing public APIs, storage format, or behavior. ### Risk Level low — self-contained new tool with no changes to existing code paths; covered by functional tests. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
