ai-yang commented on issue #10667: URL: https://github.com/apache/rocketmq/issues/10667#issuecomment-5117653745
Hi, I would like to work on this issue once #10659 reaches a stable merged state. I reproduced the failure path against the current develop branch: an expired checkpoint is staged, an exceptionally completed revive future is discarded, cleanup clears that checkpoint, and the cache can then commit the next offset even though no durable retry record exists. I plan to preserve the flush-before-offset invariant introduced by #9627: 1. Change the cache callback from Consumer<PopConsumerRecord> to Function<PopConsumerRecord, CompletableFuture<Boolean>>. 2. Dispatch all revive futures for a cleanup sweep before waiting, normalize each result independently, and remove a staged checkpoint only after the callback reports that it is safe to do so. A false, null, synchronous throw, or exceptional completion keeps the checkpoint staged and prevents the minimum offset from advancing past it. 3. Bind the cache to a reviveBufferedRecord wrapper. On revive failure, the wrapper will synchronously persist the existing backoff retry record before reporting success to the cache. If that write fails, the original checkpoint remains staged. 4. Reuse the final backoff construction from #10659, preserving attemptTimes, attemptId, and suspend. This intentionally differs from clearing the cache immediately and attaching only a completion callback: that smaller approach still leaves a broker-crash window between the clear/offset advance and durable retry persistence. The regression coverage will include exceptional and false futures, synchronous throws, persistence failure, concurrent dispatch before waiting, offset gating, backoff/max-attempt behavior, suspend preservation, and an enablePopBufferMerge=true cache-to-RocksDB path. I will wait for #10659 to merge, then recheck all open and closed PRs before creating a branch so the work does not overlap. Please let me know if the maintainers prefer a different reliability boundary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
