platinumhamburg opened a new issue, #2146: URL: https://github.com/apache/fluss/issues/2146
### Search before asking - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar. ### Fluss version 0.8.0 (latest release) ### Please describe the bug 🐞 ### Problem During rolling upgrades, a race condition between ISR expansion requests and leader re-election can cause permanent ISR state leakage in TabletServer, preventing all future ISR expansions for affected buckets. ### Reproduction Steps 1. Follower catches up with leader, triggering ISR expansion 2. AdjustIsr request is sent to Coordinator and is in-flight 3. Rolling upgrade triggers leader re-election, calling `makeLeader()` which updates `isrState` to a new `CommittedIsrState` 4. When the in-flight AdjustIsr response returns, it is ignored because `isrState != proposedIsrState` 5. However, the pending request in `AdjustIsrManager.unsentAdjustIsrMap` is NOT cleared 6. All subsequent ISR expansion attempts are rejected with `OPERATION_NOT_ATTEMPTED` because `putIfAbsent()` fails 7. The bucket remains under-replicated permanently until TabletServer restart ### Root Cause Two missing cleanup points: 1. **`updateAssignmentAndIsr()`**: When leader re-election occurs, the method updates `isrState` but doesn't clear pending requests in `AdjustIsrManager` 2. **`submitAdjustIsr()`**: When an AdjustIsr response is ignored due to state mismatch, the method returns early without clearing the pending request from `unsentAdjustIsrMap` ### Impact - Buckets become permanently under-replicated - Typically happens during rolling upgrades when leader re-election occurs ### Solution - Added `clearPendingRequest()` method to `AdjustIsrManager` - Clear pending requests in `updateAssignmentAndIsr()` - Clear pending requests when response is ignored in `submitAdjustIsr()` ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
