Caideyipi opened a new pull request, #18640:
URL: https://github.com/apache/iotdb/pull/18640

   ## Description
   
   ### Restore sink-batch forward progress
   
   #18582 changed the memory reservation owned by `PipeTabletEventBatch` from a 
normal `PipeMemoryBlock` to a `PipeTabletMemoryBlock`. This made sink-batch 
growth subject to the tablet data-structure admission thresholds. Once retained 
tablet memory crossed that sub-quota, a sink batch could no longer grow enough 
to transfer and acknowledge events, so the retained/floating memory could not 
be released either.
   
   This PR restores the batch reservation to a normal pipe memory block. Tablet 
and TsFile allocations remain bounded by their data-structure thresholds, while 
sink batches can use the remaining non-floating memory to make downstream 
progress and release retained events.
   
   ### Back off memory-timeout retries
   
   `PipeAbstractSinkSubtask` previously retried `MEMORY_TIMEOUT` failures 
immediately while applying backoff to the other transient resource failures. 
The immediate retry loop could consume several CPU cores and flood logs while 
memory remained unavailable. Memory timeouts now use the existing non-report 
exception backoff after logging the transient condition.
   
   ### Regression coverage
   
   - Verify that tablet batch payload memory increases total pipe memory 
without increasing tablet-memory accounting.
   - Verify that a sink subtask waits before retrying a memory-timeout failure.
   - `PipeSleepIntervalTest`: 4 tests passed.
   - `PipeMemoryManagerResizeTest` and `PipeTransferBatchReqBuilderTest`: 8 
tests passed after isolated compilation against the module classpath.
   - Spotless, Checkstyle, and `git diff --check` passed for the changed files.
   
   A full DataNode reactor compile was also attempted, but this Windows host 
exhausted native JVM memory while compiling the 2,983 DataNode sources. The 
focused DataNode classes and tests compile and pass independently.
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added comments explaining the why and intent where the behavior is not 
obvious.
   - [x] added unit tests for the new code paths.
   
   <hr>
   
   ##### Key changed/added classes (or packages if there are too many classes) 
in this PR
   
   - `PipeTabletEventBatch`
   - `PipeAbstractSinkSubtask`
   - `PipeTransferBatchReqBuilderTest`
   - `PipeSleepIntervalTest`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to