sollhui opened a new pull request, #66847:
URL: https://github.com/apache/doris/pull/66847

   ### What problem does this PR solve?
   
   Issue Number: None
   
   Related PR: #62007
   
   #### Problem summary
   
   A row-binlog group memtable flush creates two physical flush tasks: one for 
data and one for row binlog. Both tasks increment `flush_running_count`, but 
write backpressure previously compared that physical count with the same 
configured limit used by a normal data-only memtable.
   
   As a result, enabling row binlog effectively halved the number of logical 
memtables allowed to flush concurrently. With a limit of 2, one group memtable 
occupied the whole allowance, so subsequent writes waited for the previous 
group flush and the write/flush pipeline became nearly serial.
   
   #### Changes
   
   - Multiply the effective flush-running-count limit by the number of physical 
flush tasks per memtable group. A row-binlog group uses `configured_limit * 2`; 
a normal request is unchanged.
   - Apply the corrected limit in both storage-compute integrated and separated 
modes.
   - In cloud mode, retain the flush-running-count memory bound and 
additionally pause writes when the S3 upload queue is busy.
   - Initialize the adaptive controller's I/O signal even when adaptive 
flush-thread adjustment is disabled, because cloud write backpressure also 
consumes that signal.
   - Add unit coverage for the group flush-task count and the S3 upload queue 
busy threshold.
   
   #### Impact
   
   Row-binlog imports can keep memtable writing and flushing overlapped instead 
of being prematurely serialized, while cloud mode still bounds memory growth 
when S3 uploads accumulate.
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test: Not run (per request)
   - Behavior changed: Yes. Row-binlog writes use the intended logical flush 
concurrency, and cloud writes honor S3 upload queue pressure.
   - Does this need documentation: No
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to