prathik-onehouse opened a new pull request, #20573: URL: https://github.com/apache/hudi/pull/20573
### Describe the issue this Pull Request addresses closes #19635 `sync_validate` fails or reports a misleading error in three cases. The catch-up count rebuilt every catch-up instant as a commit from its timestamp, so a MOR target (deltacommits) or a replacecommit (insert overwrite, clustering) in the catch-up range failed with "Cannot find the instant". In mode `complete`, a failed Hive JDBC connection was replaced by a `NullPointerException` from closing the null connection in the `finally` block, hiding the real `SQLException`. A `sourceLatestCommit != null` check could never be false. ### Summary and Changelog The catch-up count now reads each catch-up instant as it is on the timeline, with its own action, through `TimelineUtils.getCommitMetadata`, and only counts completed instants, so a write still in flight is not read as if it had committed. `countRecords` closes the result set and the connection only when they were opened, so the `SQLException` reaches the user. The dead null check is removed. `TestValidateHoodieSyncProcedure` covers a MOR target, a replacecommit in the catch-up range, a pending instant in the catch-up range, and both record-count modes on an unusable HiveServer2 URL. Without the fix those four tests fail and the other four pass. ### Impact `sync_validate` now returns a catch-up count for MOR targets and for catch-up ranges with replacecommits instead of failing, and a Hive connection failure in mode `complete` is reported as the `SQLException`. No config or API change. ### Risk Level low. The change is limited to the `sync_validate` procedure; plain commit catch-up behaves as before. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
