github-actions[bot] commented on code in PR #66894:
URL: https://github.com/apache/doris/pull/66894#discussion_r3803753238
##########
be/src/load/delta_writer/delta_writer.cpp:
##########
@@ -178,8 +178,10 @@ Status DeltaWriter::write(const Block* block, const
TabletAddRowsPayload& rows,
}
{
SCOPED_TIMER(_wait_flush_limit_timer);
- while (_memtable_writer->flush_running_count() >=
- config::memtable_flush_running_count_limit) {
+ const auto effective_flush_running_count_limit =
+ config::memtable_flush_running_count_limit *
+ (_req.write_req_type == WriteRequestType::GROUP ? 2 : 1);
Review Comment:
[P2] Apply the GROUP-aware limit to the cloud writer too
`CloudTabletsChannel` also creates a `GROUP` request for row-binlog loads,
`CloudGroupRowsetBuilder` installs the same `GroupRowsetWriter`, and
`FlushToken::submit` creates the same DATA + ROW_BINLOG task pair. However,
`CloudDeltaWriter::write` still compares that physical count with the undoubled
config value. With the default limit of 2, one cloud GROUP memtable still
consumes the whole allowance, so cloud imports retain the near-serial behavior
this change fixes locally. Please share this computation with
`CloudDeltaWriter` (or centralize it) and cover both modes.
##########
be/src/load/delta_writer/delta_writer.cpp:
##########
@@ -178,8 +178,10 @@ Status DeltaWriter::write(const Block* block, const
TabletAddRowsPayload& rows,
}
{
SCOPED_TIMER(_wait_flush_limit_timer);
- while (_memtable_writer->flush_running_count() >=
- config::memtable_flush_running_count_limit) {
+ const auto effective_flush_running_count_limit =
Review Comment:
[P2] Keep observing runtime updates while waiting
`memtable_flush_running_count_limit` is registered with `DEFINE_mInt32`, so
operators can update it at runtime. Before this patch the loop reread it every
10 ms; this local `const` snapshots it once, including for DATA writers. If a
writer is waiting at count 2 and the limit is raised to 4, it used to resume on
the next poll but now remains blocked until a task finishes; decreases are
likewise delayed. Please recompute the effective limit from the current config
value in the loop condition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
