Caideyipi commented on PR #18387:
URL: https://github.com/apache/iotdb/pull/18387#issuecomment-6409875846

   Reviewed whether this PR fixes a real problem. **Yes: keep it open.** The 
focused self-review passed; no code changes were needed during this review.
   
   The production path supports the reported failure mechanism:
   
   - `FragmentInstanceManager` uses a shared notification executor with exactly 
four threads.
   - `FragmentInstanceExecution` runs driver cleanup and 
`releaseResourceWhenAllDriversAreClosed()` on that executor; the latter waits 
on `allDriversClosed`.
   - Before this PR, `LocalExecutionPlanner` returns its operator admission 
reservation through the same asynchronous state-listener path. Later fragments 
can already be `FAILED` while their reservation-return callbacks are queued 
behind earlier cleanup. Current `master` 
(`b652dddf41c41a86329845fac4214ec034a87575`) still has this path.
   
   I ran an isolated Java comparison using the actual state-machine and 
memory-block sources from each revision, plus the exact planner memory-release 
registration bodies extracted from those revisions. Four notification workers 
were held in terminal cleanup listeners with a latch, modeling the existing 
driver-close wait. The test admission pool was 16 KiB and each subsequent 
fragment reserved 4 KiB before failing.
   
   | Revision | Consecutive failed fragments completed while cleanup remained 
blocked | Next reservation rejected | Free admission bytes before unblocking |
   | --- | ---: | --- | ---: |
   | PR base `cec4f9b111bccf5302f7ac1e375c7401f425ef8c` | 4 | Yes | 0 |
   | PR head `70a43707ff2d86e917ba732bf0160cf41a03a89b` | 32 | No | 16384 |
   | Current `master` | 4 | Yes | 0 |
   
   After unblocking and draining the executor, all three runs returned to 16384 
free bytes. This demonstrates a real delayed-reservation/admission problem, 
with recovery once notification work drains. The synchronous final-state 
listener fixes this mechanism without moving driver cleanup into the transition 
thread.
   
   Verification boundaries: this was a focused concurrency reproduction 
compiled against locally available supporting dependencies, not a full build or 
an end-to-end SELECT INTO test. The existing Ubuntu DataNode CI log confirms 22 
passing tests across `FragmentInstanceStateMachineTest`, 
`FragmentInstanceContextTest`, and `LocalExecutionPlannerOperatorsMemoryTest`. 
The PR still has one failing Table Cluster IT job: 
`IoTDBTableAggregationNonStream2IT.rateFunctionsNormalTest` reports "Expected a 
PARTIAL aggregation in plan"; its cause was not established in this review.
   
   A useful follow-up would be to add a real four-thread executor/latch 
regression like this alongside the existing no-op-executor test, asserting both 
admission failure before the fix and successful reuse of the reservation after 
the fix.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to