Caideyipi opened a new pull request, #18387:
URL: https://github.com/apache/iotdb/pull/18387
## Description
### Problem
Fragment-instance state listeners run on a shared bounded notification
executor. The operator-memory estimate reserved by `LocalExecutionPlanner` was
returned from one of those asynchronous listeners.
A fragment can report a fast execution failure, such as a SELECT INTO type
mismatch, while notification threads are still closing drivers for earlier
fragments. Consecutive failures can therefore accumulate completed fragments'
admission reservations and eventually fail with a false memory-insufficient
error.
### Design
- Add `FragmentInstanceStateMachine.addFinalStateChangeListener` for short,
non-blocking bookkeeping that must finish in the thread performing the terminal
transition. It invokes each pre-registered listener exactly once and invokes
late registrations immediately.
- Return the operator-memory estimate through this synchronous final-state
listener.
- Move the existing lightweight fragment statistics and
`DataNodeQueryContext` cleanup to the same path.
- Keep driver closing, file-handle release, exchange-pool deregistration,
and other potentially blocking cleanup on the existing asynchronous listener.
This fixes all fast fragment termination paths (finished, failed, canceled,
and aborted), rather than special-casing SELECT INTO or increasing test memory.
### Tests
- `mvn spotless:check -pl iotdb-core/datanode`
- `FragmentInstanceStateMachineTest`
- `FragmentInstanceContextTest`
- `LocalExecutionPlannerOperatorsMemoryTest`
The three unit-test classes pass 20 tests. The new regression test
deliberately uses a notification executor that never runs and verifies that
terminal bookkeeping and operator-memory release still complete synchronously.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] concurrent read and write
- [x] added Javadocs for the new non-trivial method.
- [x] added comments explaining why synchronous bookkeeping is required.
- [x] added unit tests covering the new code paths.
<hr>
##### Key changed/added classes
- `FragmentInstanceStateMachine`
- `FragmentInstanceContext`
- `LocalExecutionPlanner`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
