This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new 2fdc166f994 branch-4.1: [fix](iceberg) Avoid BE abort on partition
predicates (#66835)
2fdc166f994 is described below
commit 2fdc166f99456def3dc45ade7c7bec52e77186e3
Author: Socrates <[email protected]>
AuthorDate: Mon Aug 24 12:27:45 2026 +0800
branch-4.1: [fix](iceberg) Avoid BE abort on partition predicates (#66835)
### What problem does this PR solve?
Issue Number: DORIS-27947
Related PR: #66012, #66481
Problem Summary:
On branch-4.1, Iceberg identity partition columns are classified as
partition keys and are therefore omitted from the Parquet file-column
schema tree. A predicate on such a column still reaches the Parquet
file-level predicate probe. `StructNode::children_column_exists()`
assumed that every probed column had a schema-tree entry, so a partition
predicate could trigger a DCHECK and abort the BE.
This change makes the existence probe return `false` when the schema
tree has no entry for the column. Parquet then skips file-level
predicate pushdown for that partition column while the existing
partition-value fill and upper-level predicate evaluation remain in
effect. The added unit test covers present, known-missing, and absent
schema-tree children.
### Release note
Fix a BE crash when querying Iceberg branch data with an identity
partition predicate.
### Validation
- Formatted the changed files with clang-format 16.0.6.
- `git diff --check` passed.
- BE compilation and unit tests were not run locally because of their
cost.
### Behavior changed
Yes. A column absent from the file schema tree is now treated as
unavailable for file-level predicate pushdown instead of terminating the
BE.
---
be/src/format/table/table_format_reader.h | 4 ++--
be/test/format/table/table_schema_change_helper_test.cpp | 11 +++++++++++
2 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/be/src/format/table/table_format_reader.h
b/be/src/format/table/table_format_reader.h
index 8ed4ad70c11..45c4991962f 100644
--- a/be/src/format/table/table_format_reader.h
+++ b/be/src/format/table/table_format_reader.h
@@ -211,8 +211,8 @@ public:
}
bool children_column_exists(std::string table_column_name) const
override {
- DCHECK(children.contains(table_column_name));
- return children.at(table_column_name).exists;
+ auto child = children.find(table_column_name);
+ return child != children.end() && child->second.exists;
}
void add_not_exist_children(std::string table_column_name) override {
diff --git a/be/test/format/table/table_schema_change_helper_test.cpp
b/be/test/format/table/table_schema_change_helper_test.cpp
index ba1d96e4d6d..9611568d8a3 100644
--- a/be/test/format/table/table_schema_change_helper_test.cpp
+++ b/be/test/format/table/table_schema_change_helper_test.cpp
@@ -30,6 +30,17 @@
namespace doris {
class MockTableSchemaChangeHelper : public TableSchemaChangeHelper {};
+TEST(MockTableSchemaChangeHelper, UnknownStructChildDoesNotExist) {
+ TableSchemaChangeHelper::StructNode root;
+ root.add_children("file_column", "file_column",
+ std::make_shared<TableSchemaChangeHelper::ScalarNode>());
+ root.add_not_exist_children("missing_file_column");
+
+ EXPECT_TRUE(root.children_column_exists("file_column"));
+ EXPECT_FALSE(root.children_column_exists("missing_file_column"));
+ EXPECT_FALSE(root.children_column_exists("partition_column"));
+}
+
TEST(MockTableSchemaChangeHelper, OrcNameNoSchemaChange) {
std::vector<DataTypePtr> data_types;
std::vector<std::string> column_names;
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
