Kalvin2077 opened a new pull request, #3781:
URL: https://github.com/apache/celeborn/pull/3781
### What changes were proposed in this pull request?
This PR refactors master-side slot allocation so that strategy-specific
slot-budget computation is
separated from the shared allocation workflow. The existing round-robin and
load-aware algorithms
are moved behind `SlotsAssignStrategy`, while `SlotsAllocator` retains the
common progressive
allocation and best-effort fallback behavior.
The change is intentionally structural. Existing names and allocation
behavior are preserved where
possible so that the old implementation can be compared directly with the
refactored code.
The refactor also exposed several potential correctness issues that are
intentionally left for
separate follow-up changes:
- With replication enabled, remote-only storage and rack awareness disabled,
primary and replica
locations may be assigned to the same worker.
- During budget-free rack-aware fallback, a replica may reuse the primary
worker's `StorageInfo`,
which can produce an incorrect mount-point hint and slot accounting when
workers have different
mount points.
- During budget-free rack-aware fallback with local storage enabled, replica
selection does not
verify that the selected worker has a local disk, so allocation may be
deferred to a worker that
cannot satisfy the requested local placement.
### Why are the changes needed?
`SlotsAllocator` previously mixed strategy-specific budget calculation with
worker selection,
storage selection, rack awareness, interruption awareness, and fallback
handling. Separating the
strategy boundary makes the allocation stages easier to follow and allows
strategies to evolve
without duplicating the common allocation workflow.
### Does this PR resolve a correctness bug?
<!-- Check if yes. The `correctness` label will be added/removed
automatically. -->
- [ ] Yes
### Does this PR introduce _any_ user-facing change?
<!-- Check if yes. -->
- [ ] Yes
### How was this patch tested?
- `build/sbt "celeborn-master/testOnly
org.apache.celeborn.service.deploy.master.slotsalloc.*"`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
