This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 1772f78e381 [fix](load) The NodeChannel should be canceled when failed
to add block (#37500)
1772f78e381 is described below
commit 1772f78e3813dd8e1ab308b3cfcd671a621421c9
Author: Xin Liao <[email protected]>
AuthorDate: Tue Jul 9 09:45:51 2024 +0800
[fix](load) The NodeChannel should be canceled when failed to add block
(#37500)
## Proposed changes
F20240704 15:35:33.724236 2556376 vtablet_writer.cpp:614] Check failed:
block.rows() == request->tablet_ids_size() block rows: 12192,
tablet_ids_size: 8128
*** Check failure stack trace: ***
@ 0x5612d9ebf696 google::LogMessage::SendToLog()
@ 0x5612d9ebc0e0 google::LogMessage::Flush()
@ 0x5612d9ebfed9 google::LogMessageFatal::~LogMessageFatal()
@ 0x5612d96a1770
doris::vectorized::VNodeChannel::try_send_pending_block()
@ 0x5612d0541e98 doris::ThreadPool::dispatch_thread()
@ 0x5612d0537251 doris::Thread::supervise_thread()
@ 0x7f02d4061ac3 (unknown)
@ 0x7f02d40f3850 (unknown)
@ (nil) (unknown)
The reason for this issue is due to a failed return from
`append_to_block_by_selector`. The reason for the failure here is that
the memory exceeded the limit. The previous column append was
successful, while the subsequent columns failed to allocate memory. The
failure was directly returned from here, and the subsequent
_cur_add_block_request was not executed.
However, if the NodeChannel is not cancelled, the next add block will
succeed, causing the block's rows to have an additional batch size
(4064) compared to the tablet id's size, ultimately triggering the
failure of the check.
<!--Describe your changes.-->
---
be/src/vec/sink/writer/vtablet_writer.cpp | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/be/src/vec/sink/writer/vtablet_writer.cpp
b/be/src/vec/sink/writer/vtablet_writer.cpp
index 3dece76159c..ba5caead91e 100644
--- a/be/src/vec/sink/writer/vtablet_writer.cpp
+++ b/be/src/vec/sink/writer/vtablet_writer.cpp
@@ -528,8 +528,11 @@ Status VNodeChannel::add_block(vectorized::Block* block,
const Payload* payload)
}
SCOPED_RAW_TIMER(&_stat.append_node_channel_ns);
- RETURN_IF_ERROR(
- block->append_to_block_by_selector(_cur_mutable_block.get(),
*(payload->first)));
+ st = block->append_to_block_by_selector(_cur_mutable_block.get(),
*(payload->first));
+ if (!st.ok()) {
+ _cancel_with_msg(fmt::format("{}, err: {}", channel_info(),
st.to_string()));
+ return st;
+ }
for (auto tablet_id : payload->second) {
_cur_add_block_request->add_tablet_ids(tablet_id);
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
