Caideyipi opened a new pull request, #18298:
URL: https://github.com/apache/iotdb/pull/18298
## Description
### Bound consensus subscription materialization memory
- Add a dedicated subscription share to query memory. It defaults to about
20% when subscription is enabled, is forced to 0 when disabled, and remains
compatible with the legacy eight-part query-memory configuration.
- Enforce one DataNode-wide byte budget across `PreparedEntry`,
`realtimeEntriesByWriter`, `lingerBatch`, `prefetchingQueue`, and
`inFlightEvents`.
- Drain already materialized backlog before converting more WAL entries, and
leave blocked data in WAL when no budget is available.
### Close lifecycle and ownership races
- Release retained bytes on ACK, seek, close, leader deactivation,
fatal-cleanup reconciliation, and recycled-event late ACK paths.
- Prevent a late ACK from an old consumer from stealing an event already
claimed by a new consumer.
- Separate initialization and runtime-activation monitors to avoid
lock-order inversion, and reject outdated commit contexts after seek/reboot
changes.
### Regression coverage
- Add wide-table high-throughput tests with a paused consumer and assert
materialized memory remains bounded.
- Add coverage for ACK release, deactivation cleanup, recycled-event
ownership, oversized entries, and orphaned reservation reconciliation.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] concurrent read
- [x] concurrent write
- [x] concurrent read and write
- [x] added comments explaining the why and intent of non-obvious lifecycle
code.
- [x] added unit tests for new code paths.
Tested with:
```shell
./mvnw -nsu -pl iotdb-core/datanode -am \
-Dtest=DataNodeMemoryConfigTest,SubscriptionMemoryManagerTest,ConsensusPrefetchingQueueTest
\
test -DskipITs -Dsurefire.failIfNoSpecifiedTests=false
```
Result: 28 reactor modules succeeded; 29 tests passed; Checkstyle and
Spotless passed.
<hr>
##### Key changed/added classes
- `DataNodeMemoryConfig`
- `SubscriptionMemoryManager`
- `ConsensusPrefetchingQueue`
- `QueryEngineMemoryMetrics`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
