This is an automated email from the ASF dual-hosted git repository.
sollhui pushed a commit to branch
fix-apache-master-configurable-block-size-lz4-log
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to
refs/heads/fix-apache-master-configurable-block-size-lz4-log by this push:
new b7bdce54b51 [fix](be) Catch block serialize exceptions
b7bdce54b51 is described below
commit b7bdce54b51d7f33c951b17373a57a92cfee09e8
Author: laihui <[email protected]>
AuthorDate: Thu Jun 25 15:18:19 2026 +0800
[fix](be) Catch block serialize exceptions
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Very large string columns can throw from DataTypeString
while Block::serialize estimates or writes serialized bytes. The exception
escaped the Status return path and could terminate the BE thread/process before
VNodeChannel could cancel the load gracefully. Wrap both serialize loops with
RETURN_IF_CATCH_EXCEPTION so the error is returned as Status.
### Release note
Fail oversized block serialization gracefully instead of terminating BE.
### Check List (For Author)
- Test: Manual test
- Ran targeted clang-format helper for be/src/core/block/block.cpp
- Ran git diff --check
- Behavior changed: Yes. Oversized block serialization now returns an error
Status instead of escaping as an uncaught exception.
- Does this need documentation: No
---
be/src/core/block/block.cpp | 26 +++++++++++++++-----------
1 file changed, 15 insertions(+), 11 deletions(-)
diff --git a/be/src/core/block/block.cpp b/be/src/core/block/block.cpp
index e0fca38b2bb..0e0c856d670 100644
--- a/be/src/core/block/block.cpp
+++ b/be/src/core/block/block.cpp
@@ -1014,14 +1014,16 @@ Status Block::serialize(int be_exec_version, PBlock*
pblock,
// calc uncompressed size for allocation
size_t content_uncompressed_size = 0;
- for (const auto& c : *this) {
- PColumnMeta* pcm = pblock->add_column_metas();
- c.to_pb_column_meta(pcm);
- DCHECK(pcm->type() != PGenericType::UNKNOWN) << " forget to set pb
type";
- // get serialized size
- content_uncompressed_size +=
- c.type->get_uncompressed_serialized_bytes(*(c.column),
pblock->be_exec_version());
- }
+ RETURN_IF_CATCH_EXCEPTION({
+ for (const auto& c : *this) {
+ PColumnMeta* pcm = pblock->add_column_metas();
+ c.to_pb_column_meta(pcm);
+ DCHECK(pcm->type() != PGenericType::UNKNOWN) << " forget to set pb
type";
+ // get serialized size
+ content_uncompressed_size +=
c.type->get_uncompressed_serialized_bytes(
+ *(c.column), pblock->be_exec_version());
+ }
+ });
// serialize data values
// when data type is HLL, content_uncompressed_size maybe larger than real
size.
@@ -1037,9 +1039,11 @@ Status Block::serialize(int be_exec_version, PBlock*
pblock,
}
char* buf = column_values.data();
- for (const auto& c : *this) {
- buf = c.type->serialize(*(c.column), buf, pblock->be_exec_version());
- }
+ RETURN_IF_CATCH_EXCEPTION({
+ for (const auto& c : *this) {
+ buf = c.type->serialize(*(c.column), buf,
pblock->be_exec_version());
+ }
+ });
*uncompressed_bytes = content_uncompressed_size;
const size_t serialize_bytes = buf - column_values.data() +
STREAMVBYTE_PADDING;
*compressed_bytes = serialize_bytes;
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
