nsivabalan opened a new pull request, #17724: URL: https://github.com/apache/hudi/pull/17724
### Describe the issue this Pull Request addresses The metadata table previously hard-coded its cleaner, rollback, and finalize-write parallelism to a constant (`MDT_DEFAULT_PARALLELISM = 512`) inside `HoodieMetadataWriteUtils`. At TB scale where the MDT has many file groups, operators may want to tune these independently (e.g., to bound driver/executor footprint). With the constant, this required source changes. ### Summary and Changelog Expose three new metadata-table-scoped configs in `HoodieMetadataConfig`, each defaulting to `512` to preserve existing behavior: - `hoodie.metadata.cleaner.parallelism` - `hoodie.metadata.rollback.parallelism` - `hoodie.metadata.finalize.writes.parallelism` Wire them through `HoodieMetadataWriteUtils.createMetadataWriteConfig()` via the new getters (`getCleanerParallelism`, `getRollbackParallelism`, `getFinalizeWriteParallelism`). Drop the now-redundant `MDT_DEFAULT_PARALLELISM` constant. Added `TestHoodieMetadataConfig#testCleanerRollbackParallelism` covering default, custom, and zero values. Added `TestHoodieMetadataWriteUtils` coverage for the wiring through `createMetadataWriteConfig`. ### Impact No default-behavior change — same `512` values apply unless an operator overrides them. New configs are marked `advanced` for power users tuning MDT operations on very large tables. ### Risk Level low ### Documentation Update Three new config keys are documented inline in `HoodieMetadataConfig` (description + `sinceVersion`). No website update needed since these are advanced operator-facing knobs. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
