Caideyipi commented on code in PR #18918:
URL: https://github.com/apache/iotdb/pull/18918#discussion_r4322426772


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/rescon/memory/SystemInfo.java:
##########
@@ -137,7 +137,18 @@ public synchronized boolean reportStorageGroupStatus(
           totalStorageGroupMemCost,
           REJECT_THRESHOLD);
       rejected = true;
-      if (chooseMemTablesToMarkFlush(tsFileProcessor)) {
+      boolean currentProcessorSelected = 
chooseMemTablesToMarkFlush(tsFileProcessor);
+      if (!currentProcessorSelected
+          && tsFileProcessor != null
+          && tsFileProcessor.getWorkMemTable() != null) {
+        // The writer holds its region's write lock. Waiting for another 
processor in that region
+        // to flush would prevent the flush submitter from acquiring the same 
lock. Let this writer
+        // finish and flush its own memtable, subject to the hard memory limit 
below.
+        tsFileProcessor.setWorkMemTableShouldFlush();
+        flushTaskSubmitThreadPool.submit(tsFileProcessor::submitAFlushTask);
+        currentProcessorSelected = true;
+      }
+      if (currentProcessorSelected) {

Review Comment:
   Follow-up: I reproduced a separate RPC regression in fa77c9350c2. The 
framed-transport flush reset passes the initial buffer size to 
resizeIfNecessary; with this PR's new shrink eligibility for requests at or 
below the initial size, repeated large frames shrink a 1,000-byte write buffer 
to 550 bytes after the cooldown/count, then require it to grow again. Under a 
shared memory cap, that churn can make another live buffer's allocation fail. 
An identical four-case harness passes against the pre-PR parent, fails three 
cases against fa77c9350c2, and passes all four with a local fix that shrinks 
from the completed frame size and only grows during incremental serialization. 
The local service-rpc suite passes (51 English tests; 18 Chinese tests), as do 
25 focused storage tests.
   
   This is related, but it does not explain all errors in REPORT.md: parsing 
the receiver logs shows 6,833 / 4,276 / 11,480 distinct allocation-failure 
events, all but 21 on node 46 are read-buffer frame growth failures. Node 46 
also has 21 constructor failures for 1 KiB initial buffers, which indicates the 
shared 819 MiB RPC budget was exhausted. The PR's unnecessary write-buffer 
churn can add pressure, while the existing read failures still need 
concurrent-budget investigation. This thread remains open pending that work.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to