Caideyipi commented on code in PR #18918:
URL: https://github.com/apache/iotdb/pull/18918#discussion_r4322426772
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/rescon/memory/SystemInfo.java:
##########
@@ -137,7 +137,18 @@ public synchronized boolean reportStorageGroupStatus(
totalStorageGroupMemCost,
REJECT_THRESHOLD);
rejected = true;
- if (chooseMemTablesToMarkFlush(tsFileProcessor)) {
+ boolean currentProcessorSelected =
chooseMemTablesToMarkFlush(tsFileProcessor);
+ if (!currentProcessorSelected
+ && tsFileProcessor != null
+ && tsFileProcessor.getWorkMemTable() != null) {
+ // The writer holds its region's write lock. Waiting for another
processor in that region
+ // to flush would prevent the flush submitter from acquiring the same
lock. Let this writer
+ // finish and flush its own memtable, subject to the hard memory limit
below.
+ tsFileProcessor.setWorkMemTableShouldFlush();
+ flushTaskSubmitThreadPool.submit(tsFileProcessor::submitAFlushTask);
+ currentProcessorSelected = true;
+ }
+ if (currentProcessorSelected) {
Review Comment:
Follow-up: I reproduced a separate RPC regression in fa77c9350c2. The
framed-transport flush reset passes the initial buffer size to
resizeIfNecessary; with this PR's new shrink eligibility for requests at or
below the initial size, repeated large frames shrink a 1,000-byte write buffer
to 550 bytes after the cooldown/count, then require it to grow again. Under a
shared memory cap, that churn can make another live buffer's allocation fail.
An identical four-case harness passes against the pre-PR parent, fails three
cases against fa77c9350c2, and passes all four with a local fix that shrinks
from the completed frame size and only grows during incremental serialization.
The local service-rpc suite passes (51 English tests; 18 Chinese tests), as do
25 focused storage tests.
This is related, but it does not explain all errors in REPORT.md: parsing
the receiver logs shows 6,833 / 4,276 / 11,480 distinct allocation-failure
events, all but 21 on node 46 are read-buffer frame growth failures. Node 46
also has 21 constructor failures for 1 KiB initial buffers, which indicates the
shared 819 MiB RPC budget was exhausted. The PR's unnecessary write-buffer
churn can add pressure, while the existing read failures still need
concurrent-budget investigation. This thread remains open pending that work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]