nsivabalan commented on PR #20151: URL: https://github.com/apache/hudi/pull/20151#issuecomment-6206033004
Heads-up @codope — I pushed one more commit after your approval (`1b3064508ba8`), so flagging it rather than letting the approval carry silently. It adds an optional `JavaSparkContext` to `HealthCheckContext` and the driver. Nothing uses it yet; it is plumbing for two things: - the metadata-table sync check on #20154 should delegate its listing comparison to `HoodieMetadataTableValidator` instead of reimplementing a narrower one — your v6 uncommitted-log-files finding is what surfaced that, and the validator already resolves committed-ness from the log block header, which is version-independent - the compaction and cleaner checks walk file slices per file group and file versions per partition on the driver, which will not hold on a table with hundreds of thousands of file groups Both existing constructors are kept, so a caller that supplies no context behaves exactly as before and the timeline-only checks stay runnable without a cluster. A check that needs a context and is not given one will report SKIPPED naming what it needed, matching the contract already used for a missing writer property. Happy to split this into its own PR if you would rather #20151 stay as reviewed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
