ThreeFire1 opened a new pull request, #18722:
URL: https://github.com/apache/dolphinscheduler/pull/18722
## Was this PR generated or assisted by AI?
YES. AI assisted with the investigation, code changes, regression tests, and
this description. I reviewed the proposed approach and the reported test
results.
## Purpose of the pull request
Fixes #18721.
If serial-command cleanup or a pre-commit callback fails after the workflow
update, the transaction rolls back but the shared workflow instance already has
a final state. The Master then skips the retried finish event, and a following
SERIAL_WAIT instance can remain blocked.
This change keeps the shared instance unchanged until the transaction
commits, so a rollback leaves it ready for retry.
## Brief change log
- Update a copy of the workflow instance and delete its serial command in
the same transaction. After commit, update the shared state and end time,
record the completion metric, and publish the finalize event.
- Avoid completing the same workflow twice within an enclosing transaction.
If an earlier `afterCommit` callback throws, use `afterCompletion` to finish
only when the transaction committed and completion has not already been
attempted.
- Keep a completion-metric exception from preventing finalize publication.
- Add rollback, retry, duplicate-event, and transaction-callback tests, plus
Master/H2 tests for automatic recovery and serial queue progress.
## Verify this pull request
Added `WorkflowFinishPersistenceRecoveryTestCase` using the existing Master
integration test framework and H2.
- The rollback matrix covers nine transitions from RUNNING_EXECUTION,
READY_STOP, and READY_PAUSE to SUCCESS, FAILURE, STOP, or PAUSE. It injects
failures in the workflow update, serial-command deletion, and `beforeCommit`,
then checks rollback, retry, and repeated events. These cases use real handlers
and transactions with a mocked execution graph and event bus.
- Separate Master tests verify automatic retry through the real event bus:
SUCCESS with two SERIAL_WAIT instances, and FAILURE, STOP, and PAUSE with
parallel workflows. They check final workflow/task states and resource release
without restarting the Master or manually replaying events.
- Additional tests cover enclosing REQUIRED transactions, duplicate finish
calls, callback failures, metrics, and finalize publication attempts.
The original reproduction fails on unmodified production code: the database
rolls back while the shared instance stays final. It passes with this change.
Focused clean builds passed 35 selected cases in both Surefire executions on
JDK 8 and JDK 11. An earlier JDK 11 build passed 53 cases covering transaction
callback boundaries and related workflow regressions with the same production
change. These selections overlap; they are not a single full-suite run.
Spotless checks and JaCoCo were enabled, and `spotless:apply` was run before
testing.
To run the full workflow-finish and related regression selection:
```sh
./mvnw -B -ntp -pl dolphinscheduler-master -am \
-Dtest=WorkflowFinishPersistenceRecoveryTestCase,WorkflowStartSerialStrategyTestCase,WorkflowInstanceMetricsTestCase,WorkflowBackfillTestCase,WorkflowStartSubWorkflowTestCase
\
-Dsurefire.failIfNoSpecifiedTests=false clean test
```
The tests inject exceptions into real H2 transactions; they do not simulate
a database-server outage. The full project test suite was not run.
## Pull Request Notice
[Pull Request
Notice](https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
