ArafatKhan2198 opened a new pull request, #12093: URL: https://github.com/apache/ozone/pull/12093
## What changes were proposed in this pull request? **Sequence** 1. Batch 1 and batch 2 are both waiting in the buffer. 2. A task fails batch 1 on both attempts. Recon sets `tasksFailed`. The rebuild has not started, and the flag lives only in memory. 3. The event thread still takes batch 2. The task succeeds, and its saved position moves to the end of batch 2. 4. Recon restarts before the next sync queues the rebuild. The flag is gone. 5. On startup the saved position matches the OM database position, so nothing is rebuilt. 6. Batch 1 is never applied. For `NSSummaryTask` the namespace summary and du APIs keep reporting too few files and bytes. Later syncs and restarts do not notice. Only a manual rebuild repairs it. New writes before the restart move the OM database position ahead of the task, and startup then does rebuild. The risky window is about one sync interval on a busy cluster, and until the queued rebuild finishes on an idle one. **Change** `processOMUpdateBatch` returns immediately while `tasksFailed` is set. Batches already waiting are not given to any task, so no saved position moves. This is the same rule `consumeOMEvents` already uses for batches that arrive after the failure. Skipping them loses nothing. The next sync still drains the buffer and rebuilds from a checkpoint that contains every skipped batch. After a restart, the task position is behind the OM database position, so the existing startup check rebuilds the task and batch 1 is included. Tasks that did not fail pause only until that rebuild is queued. A task that throws instead of returning a failed result never sets the flag. That case is HDDS-17040 and is not covered here. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-17050 ## How was this patch tested? `testBatchesBehindFailedBatchDoNotAdvanceTaskCursor` in `TestReconTaskControllerImpl`. A task fails a batch ending at sequence 100 on both attempts, which sets `tasksFailed` and leaves its saved position at 0. A later batch ending at 200, which the task would succeed on, is then processed. The position stays at 0 and the task is not called for that batch. Without the change the position becomes 200. ```bash mvn -pl :ozone-recon test -Dtest=TestReconTaskControllerImpl -DskipShade -DskipRecon ``` The tests ran successfully. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
