This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 536078a1925 [opt](move-memtable) reduce tablet report frequency
(#59973)
536078a1925 is described below
commit 536078a19259839a821372f0b7464d86d8336d1f
Author: hui lai <[email protected]>
AuthorDate: Mon Jan 19 09:41:27 2026 +0800
[opt](move-memtable) reduce tablet report frequency (#59973)
Report tablet load info only on `ADD_SEGMENT` to reduce frequency: `
ADD_SEGMENT` is sent once per segment, while `APPEND_DATA` is sent for
every data batch, which reduces some unnecessary tablet report.
---
be/src/runtime/load_stream.cpp | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/be/src/runtime/load_stream.cpp b/be/src/runtime/load_stream.cpp
index 93df7c025c7..b00681427bb 100644
--- a/be/src/runtime/load_stream.cpp
+++ b/be/src/runtime/load_stream.cpp
@@ -712,15 +712,24 @@ void LoadStream::_dispatch(StreamId id, const
PStreamHeader& hdr, butil::IOBuf*
}
switch (hdr.opcode()) {
- case PStreamHeader::ADD_SEGMENT: // ADD_SEGMENT will be dispatched inside
TabletStream
- case PStreamHeader::APPEND_DATA: {
+ case PStreamHeader::ADD_SEGMENT: {
auto st = _append_data(hdr, data);
if (!st.ok()) {
_report_failure(id, st, hdr);
} else {
+ // Report tablet load info only on ADD_SEGMENT to reduce frequency.
+ // ADD_SEGMENT is sent once per segment, while APPEND_DATA is sent
+ // for every data batch. This reduces unnecessary writes and avoids
+ // potential stream write failures when the sender is closing.
_report_tablet_load_info(id, hdr.index_id());
}
} break;
+ case PStreamHeader::APPEND_DATA: {
+ auto st = _append_data(hdr, data);
+ if (!st.ok()) {
+ _report_failure(id, st, hdr);
+ }
+ } break;
case PStreamHeader::CLOSE_LOAD: {
std::vector<int64_t> success_tablet_ids;
FailedTablets failed_tablets;
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
