Caideyipi opened a new pull request, #18649:
URL: https://github.com/apache/iotdb/pull/18649

   ## Description
   
   Backport #18640 to `dev/1.3` by cherry-picking 
e2ac222343b93cd0388cae02e6f637bdd1522ddb.
   
   ### Restore sink-batch forward progress
   
   #18582 changed the memory reservation owned by `PipeTabletEventBatch` from a 
normal `PipeMemoryBlock` to a `PipeTabletMemoryBlock`. This made sink-batch 
growth subject to the tablet data-structure admission thresholds. Once retained 
tablet memory crossed that sub-quota, a sink batch could no longer grow enough 
to transfer and acknowledge events, so the retained/floating memory could not 
be released either.
   
   This change restores the batch reservation to a normal pipe memory block. 
Tablet and TsFile allocations remain bounded by their data-structure 
thresholds, while sink batches can use the remaining non-floating memory to 
make downstream progress and release retained events.
   
   ### Back off memory-timeout retries
   
   `PipeAbstractSinkSubtask` previously retried `MEMORY_TIMEOUT` failures 
immediately while applying backoff to the other transient resource failures. 
Memory timeouts now use the existing non-report exception backoff after logging 
the transient condition.
   
   The cherry-pick had one conflict in `PipeAbstractSinkSubtask`; the 
resolution preserves the `dev/1.3` logging code and applies the upstream 
backoff behavior.
   
   ### Regression coverage
   
   - `mvn spotless:apply -pl iotdb-core/node-commons,iotdb-core/datanode`: 
passed.
   - `mvn test -pl iotdb-core/node-commons -Dtest=PipeSleepIntervalTest`: 
passed (4 tests).
   - `git diff origin/dev/1.3..HEAD --check`: passed.
   - `PipeTransferBatchReqBuilderTest` was attempted, but the standalone 
DataNode build stopped before tests because the local `1.3.7-SNAPSHOT` 
dependency artifacts are older than the current `dev/1.3` sources (missing 
unrelated generated Thrift and cross-module APIs). The changed DataNode source 
itself produced no compiler error; CI will build it with reactor-consistent 
dependencies.
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added unit tests for the changed code paths.
   
   <hr>
   
   ##### Key changed/added classes
   
   - `PipeTabletEventBatch`
   - `PipeAbstractSinkSubtask`
   - `PipeTransferBatchReqBuilderTest`
   - `PipeSleepIntervalTest`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to