Caideyipi opened a new pull request, #18272:
URL: https://github.com/apache/iotdb/pull/18272
## Description
A table standalone integration test exposed a race in `MemoryPool`:
```
SharedTsBlockQueue.remove()
-> MemoryPool.free()
-> NullPointerException because the fragment reservation map no longer
exists
```
`deRegisterFragmentInstanceFromQueryMemoryMap` checked whether a fragment's
reservations were zero before taking the map-removal lock. A concurrent
reservation could update the fragment map after that zero snapshot but before
deregistration removed it. The memory block was then charged while its
reservation map was detached, so the downstream consumer failed when freeing
the buffered TsBlock.
This is independent of the Pipe changes in #18266. That PR's head was based
directly on `b8ff9eeaea0d2e5afd79446e8bb4cf654c8c8038` (`origin/master` at the
time), and its diff only touched `PipeMemoryManager` and its test.
## Fix
- Serialize register, reserve, rollback, free, and deregister operations on
the affected fragment reservation map.
- Revalidate the query/fragment map identities after acquiring the fragment
lock so no operation updates a detached map.
- Keep reservations for different fragments concurrent.
## Tests
Added a deterministic concurrency test that pauses `Map.merge` while
deregistration runs. With the `origin/master` implementation, it fails because
the reservation map is removed (`expected 256 but was 0`). With this change:
- `MemoryPoolTest`: 23 tests run, 0 failures
- `mvn spotless:apply -pl iotdb-core/datanode`: success
- Checkstyle reported 0 violations before compilation was blocked by
unrelated local cross-module/generated-source state
The local full reactor could not complete because the Windows Thrift 0.14
generator emitted `javax.annotation.Generated` while this branch uses Jakarta
annotations; the changed production and test sources were therefore compiled
directly with JDK 17 against the Maven-resolved datanode test classpath for the
targeted JUnit run.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added tests that fail without the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
