Caideyipi opened a new pull request, #18866: URL: https://github.com/apache/iotdb/pull/18866
## Description Delayed manual ACKs currently keep every delivered consensus tablet batch materialized, so a consumer can exhaust the broker's tablet budget before it finishes a transaction, batch, or checkpoint. This change releases a delivered batch's tablets and response cache after serialization once every original member and WAL seal is readable, while retaining its commit context and a compact delivery record. Releasing the payload does not advance committed progress. ### Delivery and replay - Record exact batch members using delta/zigzag varints, together with the original batch shape, memory charge, and frozen column projection. Redelivery preserves sparse membership and replicated writer records without a local search index. - On NACK, timeout, or disconnect, reserve the original materialization budget and rebuild the batch through an independent WAL iterator. Incomplete WAL keeps the original payload; failed replay keeps the delivery record and returns the reserved memory for a later retry. - Fence serialization, payload replacement, cleanup, and late delivery callbacks with the event monitor and a response identity for each poll attempt. Seek and close clear the delivery indexes. - Protect WAL needed by live delivery records, remove those records after ACK, and refresh the deletion boundary. Restart recovery continues from committed writer positions. ### Resource tradeoffs WAL validation and redelivery add disk reads and materialization work. Delivery metadata remains proportional to the number of outstanding batches; member coordinates are compressed and records retain neither tablets nor serialized payloads. The existing single-entry size limit still applies. ### Validation - English full-reactor main and test-source compilation: `mvn -nsu test-compile -DskipTests`. - Chinese full-reactor main and test-source compilation: `mvn -nsu test-compile -P with-zh-locale -DskipTests`. - Focused subscription/WAL regression: **157 tests, 0 failures, 0 errors, 1 existing skipped test**, covering delivery, conversion, ACK state, initialization, seek, memory admission, WAL backpressure, the receiver, and the WAL iterator. - All nine new delivery tests pass, including consuming ten batches with a two-batch memory budget before committing, out-of-order/late ACKs, redelivery after reclamation, incomplete/unreadable WAL, sparse/replicated members, restart recovery, seek reset, and serialization/cleanup fencing. - Spotless and Checkstyle pass during compilation. English/Chinese message keys and format placeholders match; `git diff --check` passes. <hr> This PR has: - [x] been self-reviewed for concurrent read, concurrent write, and concurrent read/write. - [x] added comments explaining the lifecycle and WAL retention decisions. - [x] added unit tests covering new delivery and replay paths. ##### Key changed classes `ConsensusPrefetchingQueue`, `ConsensusLogToTabletConverter`, `SubscriptionEvent`, `SubscriptionEventSingleResponse`, `CachedSubscriptionPollResponse`, and `SubscriptionReceiverV1`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
