xxubai opened a new pull request, #4304:
URL: https://github.com/apache/amoro/pull/4304
## Why are the changes needed?
When table ownership is transferred between AMS nodes, the new owner
receives a
`tableAdded` event. Previously, `ProcessService` only initialized the
periodic
schedulers for the transferred table and did not recover its persisted active
processes.
As a result, a process left in `SUBMITTED` or `RUNNING` state after the
previous
owner exited could remain stuck indefinitely.
For example, an orphan-clean process with no available external process
identifier would not be resumed by
the new owner, leaving orphan files unprocessed.
The existing recovery path was only invoked during AMS startup. It must also
be
invoked when a table runtime is transferred to the current AMS node.
Close #4300.
## Brief change log
- Query active `SUBMITTED` and `RUNNING` processes by the table IDs owned by
the
current AMS node.
- Recover persisted active processes when handling `tableAdded`, before
starting
the table's periodic schedulers.
## How was this patch tested?
- [x] Add some test cases that check the changes thoroughly including
negative and positive cases if possible
Added coverage for:
- Active process recovery after table ownership transfer.
- Concurrent duplicate `tableAdded` events.
- Recovery of the same persisted process ID with a new external identifier.
- Prevention of duplicate scheduler submissions after recovery.
- Existing startup recovery and recovery-failure isolation.
- [ ] Add screenshots for manual tests if appropriate
Not applicable. This patch does not change the UI.
- [x] Run test locally before making a pull request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
