lilei1128 commented on PR #3359: URL: https://github.com/apache/fluss/pull/3359#issuecomment-4554964831
> Hi @lilei1128 , thanks for your contribution to fix this issue. Glad to see you are interested in this task. > > In your PR, it is a good thinking to reuse the existing retry mechanism to solve this problem. However, the current retry loop doesn't truly solve the problem. The retry loop has no delay between attempts, so it degrades to "fail 5 times in milliseconds → pretend success → data residue on disk." > > I'd like to solve this problem by introducing "**per-TS sender**" + "**ReplicaDeletionIneligible state**": > > * Per-TS sender thread gives each TabletServer an independent request queue with backoff-aware retry. A temporarily unreachable TS doesn't burn through retry attempts instantly — the sender sleeps and retries over a meaningful time window. > * ReplicaDeletionIneligible state breaks the current all-or-nothing dilemma. Instead of force-marking unreachable replicas as successfully deleted, they enter Ineligible — which (1) unblocks isEligibleForDeletion (since Ineligible ≠ DeletionStarted, resumeDeletions is no longer stuck), and (2) preserves the truth that deletion hasn't happened yet. When the TS comes back online, replicas transition Ineligible → OfflineReplica → ReplicaDeletionStarted and get genuinely deleted. > > If you are interested, welcome to review this [PR](https://github.com/apache/fluss/pull/3391), leave any comments there, or help to push code to fix based on it. Hi @[reviewer_name], Thank you so much for your detailed and insightful review! I really appreciate you taking the time to explain the flaws in the current retry approach and sharing the better design direction. I fully agree with your points: the existing retry loop without delay can only cause rapid failures and leave data residues on disk, which doesn’t solve the root problem. The solution you proposed — combining **per-TS sender** with backoff-aware retry and **ReplicaDeletionIneligible** state — is elegant and thorough. It not only unblocks the deletion workflow but also keeps the correct state transition logic, which perfectly fixes the all-or-nothing dilemma. I’m very interested in this design. I will take a close look at your PR. Thanks again for your guidance! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
