Caideyipi commented on code in PR #18866:
URL: https://github.com/apache/iotdb/pull/18866#discussion_r4269332536
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusPrefetchingQueue.java:
##########
@@ -2696,11 +2761,161 @@ private void retainEventMemory(final SubscriptionEvent
event, final long bytes)
}
}
+ private void onResponseDelivered(
+ final SubscriptionEvent event, final SubscriptionPollResponse response) {
+ acquireReadLock();
+ try {
+ synchronized (event) {
+ final DeliveryBatchRecord record = deliveryBatches.get(event);
+ if (isClosed
+ || closeRequested
+ || pendingSeekRequest != null
+ || Objects.isNull(record)
+ || event.isCommitted()
+ || event.getCurrentResponse() != response
+ || outstandingEventsByCommitContext.get(event.getCommitContext())
!= event) {
+ return;
+ }
+ record.delivered = true;
+ reclaimDeliveredPayload(event, record);
+ }
+ } finally {
+ releaseReadLock();
+ }
+ requestPrefetch();
+ }
+
+ private void reclaimDeliveredPayloads() {
+ // Only materialized events need a check. The number of these is bounded
by the memory budget.
+ for (final SubscriptionEvent event : retainedBytesByEvent.keySet()) {
+ synchronized (event) {
+ final DeliveryBatchRecord record = deliveryBatches.get(event);
+ if (Objects.nonNull(record) && record.delivered &&
!event.isCommitted()) {
+ reclaimDeliveredPayload(event, record);
+ }
+ }
+ }
+ }
+
+ private void reclaimDeliveredPayload(
+ final SubscriptionEvent event, final DeliveryBatchRecord record) {
+ if (event.isPayloadDetached() || !record.delivered) {
+ return;
+ }
+ // Pending entries can precede durable WAL visibility. Verify every member
and fragment before
+ // releasing the only payload copy. This read does not change replay or
committed progress.
+ if (!record.replayable && Objects.isNull(readDeliveryBatch(event, record,
false))) {
+ return;
+ }
Review Comment:
Yes—the previous path repeated a WAL scan/deserialization in the poll
delivery callback. Changed it to reuse the initial complete WAL read (including
all fragments and the seal, with no incomplete scan), propagated across every
batch member; WAL-backed delivery now opens no extra iterator.
Pending/partial batches keep their payload and are validated by the prefetch
worker after delivery. Successful validation remains cached across
redeliveries. This preserves the WAL-readability guard while moving pending
validation off the RPC path.
Added three regressions for reader reuse, deferred validation, and
mixed-batch safety. Focused subscription/WAL validation: **151 tests, 0
failures/errors, 1 existing skip**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]