bobhan1 opened a new pull request, #67867:
URL: https://github.com/apache/doris/pull/67867
### What problem does this PR solve?
Problem Summary:
A Cloud ONCE/PERIODIC warm-up job restored as RUNNING after FE restart or
master failover bypasses the PENDING registration path. Its destination is
therefore absent from the in-memory running-job map, allowing a second job for
that destination to start and encounter a BE job conflict.
Rebuild destination ownership from the final restored job states before the
master becomes ready and starts warm-up scheduling. This covers both image
loading and journal replay, preserves the running job's batch progress, and
avoids retaining ownership for periodic jobs whose latest journal state is
PENDING. If legacy metadata contains multiple RUNNING jobs for one destination,
retain the lowest job ID deterministically and cancel the others through the
existing cleanup/journal path after FE is ready but before scheduling starts
(ONCE becomes CANCELLED; PERIODIC returns to PENDING).
Add FE tests covering image/journal recovery, final periodic states, legacy
ONCE jobs, preserved progress, queued execution with mocked BE RPCs,
deterministic conflict cleanup, and safe owner release.
### Release note
Fix Cloud warm-up destination ownership recovery after FE restart or master
failover so newly submitted jobs wait for a recovered running job on the same
destination.
### Check List (For Author)
- Test:
- [x] Unit Test: 42 tests passed, including 16 recovery cases. A focused
pre-fix test on the master baseline reproduced the missing reservation (1 test,
1 failure).
- Checkstyle: the full FE check found only two pre-existing JUnit 4
import violations in unchanged `cloud/rpc/VersionHelperTest.java`; the separate
check of all three changed files passed.
- Docker regression was not run.
- Behavior changed:
- [x] Yes. Restore destination reservations before scheduling; clean up
conflicting legacy RUNNING jobs deterministically.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
