hani-fouladgar commented on PR #11571:
URL: https://github.com/apache/ozone/pull/11571#issuecomment-6346451703
@ashishkumar50
Great point — you've found a real flaw. The current metric uses the minimum
follower index (the single worst follower), but Ratis commits on a majority, so
with leader + one healthy follower the DeletedBlockLog can still commit at full
speed. As written, one persistently-sick follower would throttle deletions
indefinitely even though SCM is fully capable — exactly the unbounded OM-DB
growth you describe.
The fix is to measure lag against the quorum/majority replication point
rather than the worst follower: sort the follower match indices and use the one
that forms a quorum with the leader (for a 3-node group, the best follower).
Then:
- a single lagging follower (leader + another healthy) → quorum lag ≈ 0 → no
throttling, deletion runs at full rate;
- throttling engages only when a majority genuinely can't keep up — a real
"can't commit" state where the backlog is unavoidable regardless.
That also distinguishes the two cases you raised: a single
persistently-lagging follower is left to Ratis InstallSnapshot/bootstrap to
recover, while backpressure only kicks in for actual quorum-level lag. It's a
small change localized to the lag computation — I'll open a follow-up Jira/PR
for it. Thanks for catching this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
