Sigma-Ma commented on PR #5949: URL: https://github.com/apache/hbase/pull/5949#issuecomment-5043782631
Hi, I would like to take ownership of this issue and continue the fix. I reviewed PR #5949 together with the reproduction reported in HBASE-29786. HBASE-29786 appears to be another occurrence of the same problem on 2.6.2. The current return-to-break change in #5949 is not safe by itself, because clearWALEntryBatch may drain the queue while the reader or shipper is still alive. The reader may enqueue another batch after the drain, and the shipper may already own an in-flight batch which is no longer present in the queue. My proposed fix is to make quota cleanup follow batch ownership: 1. The reader releases its current batch on an unsuccessful enqueue, as it already does, and drains the remaining queue when it exits due to source termination. 2. The shipper releases a polled batch in a finally block if the batch was not successfully shipped. 3. Successfully shipped batches continue to release quota through postShipEdits. This avoids resetting the global totalBufferUsed value and does not depend on increasing the cleanup timeout. I will also add a deterministic latch-based regression test covering both an in-flight batch and a queued batch during peer refresh. Unless there are objections, I will prepare an updated patch against current master and then backport it to branch-2.6. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
