Caideyipi opened a new pull request, #18710:
URL: https://github.com/apache/iotdb/pull/18710

   ## Description
   
   ### Prevent duplicate auto commits
   
   When pull-consumer auto commit is enabled, polled commit contexts are cached 
for the auto-commit worker. A successful explicit synchronous or asynchronous 
commit previously updated committed progress but left the accepted contexts in 
that cache. The worker then retried those already-acknowledged contexts every 
interval. Because the server had already removed the corresponding in-flight 
events, every retry was rejected and retained for the next interval.
   
   Notify the pull consumer after each successful ACK response and remove only 
the contexts actually accepted by the server. Rejected contexts remain buffered 
so partial commits retain their retry behavior.
   
   ### Avoid stale-commit WARN storms
   
   A missing in-flight context can be caused by a duplicate/stale ACK or NACK 
and is handled as an unaccepted context. Log the per-context detail at DEBUG 
instead of WARN. Also log fully rejected commit batches at DEBUG while 
preserving WARN for genuinely partial acceptance, where some contexts succeeded 
and some failed.
   
   In the reproduced initial-snapshot workload, duplicate commits produced 
about 176,000 per-context WARN records plus 3,000 fully rejected batch WARN 
records in under seven minutes. These changes remove both amplification paths 
while preserving actionable partial-success diagnostics.
   
   ### Tests
   
   - Added lifecycle regression coverage for explicit synchronous commit 
cleanup.
   - Added lifecycle regression coverage for explicit asynchronous commit 
cleanup.
   - Added coverage verifying that partial acceptance removes accepted contexts 
and retains rejected contexts.
   - `mvn -pl iotdb-client/subscription 
-Dtest=SubscriptionConsumerLifecycleTest test`
   - `mvn -pl iotdb-client/subscription,iotdb-core/datanode spotless:apply`
   
   The DataNode module-level compile was also attempted. Its source compilation 
is currently blocked in this checkout by unrelated stale generated/installed 
dependencies (missing planner classes and mismatched Thrift/ANTLR interfaces); 
Checkstyle and Spotless completed successfully for the modified DataNode 
sources.
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added unit tests or modified existing tests to cover new code paths.
   
   <hr>
   
   ##### Key changed/added classes
   
   - `AbstractSubscriptionConsumer`
   - `AbstractSubscriptionPullConsumer`
   - `SubscriptionConsumerLifecycleTest`
   - `SubscriptionPrefetchingQueue`
   - `SubscriptionReceiverV1`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to