danny0405 opened a new pull request, #20495: URL: https://github.com/apache/hudi/pull/20495
### Describe the issue this Pull Request addresses Part of #20406 and #20401 Phase 2 B2. The sink's query-scoped resources and persisted batch identifier need regression coverage at the boundary where Hudi has committed but Spark has not yet persisted its batch checkpoint. ### Summary and Changelog - Add a test-only V1 source/sink that injects failure after the real Hudi addBatch succeeds and before Spark writes its commit log. - Cover AvailableNow completion, explicit stop and injected failure with a real local streaming query. - Await asynchronous cleanup, check that client/service references are released and the actual timeline-server port no longer accepts connections. - Keep an unrelated query alive and verify it remains usable. - Restart from the same checkpoint and compare exact rows and completed Hudi commit instants. The failure case also verifies that Spark's checkpoint was absent before restart and exists afterward. Validation: Spark 4.1 / Scala 2.13 / JDK 17 test compilation, checkstyle/scalastyle and RAT passed. `TestStreamingSinkRecovery`: 3 tests passed, 0 failures/errors. `git diff --check` and the repository PR-description compliance validator passed. No external code was copied. ### Impact Tests only; no production APIs, storage formats or configuration changes. This covers cleanup and commit/checkpoint recovery, not the separate requirement to reject a multi-flow SDP graph before any target write. Full refresh and checkpoint-generation isolation remain Phase 4 work. ### Risk Level low. Tests use isolated temporary storage and bounded waits. Reflection is confined to test instrumentation for privately owned resources; an actual closed socket and successful recovery establish externally observable behavior. ### Documentation Update none. Test comments explain the injection boundary and existing production behavior is unchanged. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
