deardeng opened a new pull request, #66499:
URL: https://github.com/apache/doris/pull/66499
…ance
A backend's load score is
capacityProportion * capCoeff + replicaNumProportion * (1 - capCoeff)
A tablet whose replica data size is zero relocates nothing, so migrating it
leaves the capacity term untouched while still improving the replica count
term. LoadStatisticForTag.isMoreBalanced() therefore accepts the move even
though not a single byte is copied.
The disk usage difference that triggers a balance is measured as
(totalCapacity - availableCapacity), so it also covers non-Doris files and
plain disk size differences, which no tablet migration can ever remove. With
zero sized tablets as the only available lever, the scheduler keeps shifting
replica counts until the composite score converges, and the per partition
replica distribution pays for it: a freshly created table that createTablets()
spread round robin over 16 backends as 16/17 ends up as 12-18, while every
backend still reports a MID load score.
Skip such tablets in BE load balance, both when selecting candidates and
again when the context is finally scheduled, since the two can be far apart in
time and the size is refreshed in between. Zero also means the size has not
been reported yet, because replica data size is not persisted in the FE image
and stays zero after a restart until the next tablet stat update; balancing on
an unknown size is guesswork, so skipping is right there as well.
This is the same rule DiskRebalancer.completeSchedCtx() already applies to
disk balance, and it is unconditional here too, including urgent balance:
moving an empty replica frees no space on a filling disk either.
Add TStat counterBalanceRejectByZeroDataSize for the scheduling time
rejection, and report the number of tablets skipped during selection in the
existing per round log instead of a counter, because after a restart every
tablet of every high load backend hits that path on every round.
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
