danny0405 opened a new issue, #20752: URL: https://github.com/apache/hudi/issues/20752
### Task Description **What needs to be done:** Add deterministic failure-injection and retry acceptance for Hudi materialized-view refresh through the real Spark pipelines CLI / Spark Connect harness. This carries forward the unfinished failure-testing requirement from #20405. Use the current purge-style full-table truncate contract agreed in #20404 and documented by #20622. A refresh purges/reinitializes the target, reconciles catalog metadata, and writes the recomputed result. The sequence is non-transactional: there is no cross-step rollback or guarantee that the old snapshot survives after purge begins. **Why this task is needed:** Merged #20494 and #20675 establish successful repeated refreshes and exact keyed/keyless results. They do not inject failures around truncate, reconciliation, or result writing. Documenting the empty window does not establish recovery acceptance. ### Acceptance criteria - [ ] Inject a failure before any destructive truncate work and verify the previous target contents remain intact. - [ ] Inject a failure after purge/truncate completes but before result writing; assert the observed empty/reinitialized target and catalog state, without assuming rollback to the previous snapshot. - [ ] Inject failures during catalog reconciliation and result writing. Assert the actual target/catalog state and completed/incomplete write evidence at each boundary; do not assume that a failed client operation implies no data committed. - [ ] Remove each injected fault and rerun the refresh through the CLI. Assert exact recomputed business rows, with no stale or duplicate output; include an empty recomputation. - [ ] Verify failures reach the CLI with useful diagnostics and that cleanup leaves the Connect server usable for the subsequent retry. - [ ] Retain deterministic fault settings, generated definitions, CLI/server logs, and state assertions. Checks must remain effective under Python optimization. - [ ] Update the runnable acceptance documentation with the tested failure boundaries, empty window, retry procedure, and lack of cross-step rollback. ### Scope Test the current purge-style MV refresh. Timeline-preserving truncate, replace-commit/history/savepoint assertions, and streaming checkpoint-generation isolation remain together under #20404 / #20378. Atomic refresh and streaming-table full-refresh acceptance are not established by this task. ### Task Type Test enhancement ### Related Issues Parent feature issue: #20401 Extracted from: #20405 Existing acceptance: #20494, #20675 Harness: #20402 / #20453 Current refresh contract/documentation: #20404, #20622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
