Caideyipi commented on code in PR #18918:
URL: https://github.com/apache/iotdb/pull/18918#discussion_r4322250655
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/rescon/memory/SystemInfo.java:
##########
@@ -137,7 +137,18 @@ public synchronized boolean reportStorageGroupStatus(
totalStorageGroupMemCost,
REJECT_THRESHOLD);
rejected = true;
- if (chooseMemTablesToMarkFlush(tsFileProcessor)) {
+ boolean currentProcessorSelected =
chooseMemTablesToMarkFlush(tsFileProcessor);
+ if (!currentProcessorSelected
+ && tsFileProcessor != null
+ && tsFileProcessor.getWorkMemTable() != null) {
+ // The writer holds its region's write lock. Waiting for another
processor in that region
+ // to flush would prevent the flush submitter from acquiring the same
lock. Let this writer
+ // finish and flush its own memtable, subject to the hard memory limit
below.
+ tsFileProcessor.setWorkMemTableShouldFlush();
+ flushTaskSubmitThreadPool.submit(tsFileProcessor::submitAFlushTask);
+ currentProcessorSelected = true;
+ }
+ if (currentProcessorSelected) {
Review Comment:
Agreed, this is a valid risk and should be addressed before merging. The
fallback can flush the current processor even when its memtable is much smaller
than the processors selected by the normal policy. The workload report shows
172/280/458 reject-status events and zero rejected inserts/606 errors, but it
does not include flush counts or memtable/file-size distribution, so it cannot
rule out small-file amplification. I will treat this as unresolved and collect
those metrics under the same workload before claiming the fallback has no such
cost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]