nsivabalan opened a new pull request, #18803: URL: https://github.com/apache/hudi/pull/18803
### Change Logs Detailed size metrics in the metadata table cause `HoodieTableFileSystemView` (FSV) to be built for MDT in the driver, which adds memory pressure at scale. This patch introduces a new config `hoodie.metadata.enable.detailed.metrics` (default: `false`) that gates emission of the detailed size metrics. When disabled, `updateSizeMetrics` is skipped. Changes: - New config `HoodieMetadataConfig.ENABLE_DETAILED_METRICS` (default false) - `HoodieMetadataMetrics` carries an `enabledDetailedMetadataMetrics` flag in its constructor - `HoodieBackedTableMetadataWriter#updateSizeMetrics` only runs when detailed metrics are enabled - All `HoodieMetadataMetrics` call sites updated (Spark/Flink/Java writers, `RunIndexActionExecutor`, `BaseTableMetadata`, `SparkHoodieBackedTableMetadataWriterTableVersionSix`) - Tests cover default, metrics-enabled, and detailed-metrics-enabled paths ### Impact Reduced driver memory pressure for large metadata tables. No behavior change unless `hoodie.metadata.enable.detailed.metrics=true` is explicitly set. ### Risk level low ### Documentation Update The new config description is added in `HoodieMetadataConfig`. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Change Logs and Impact were stated clearly - [x] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
