Gabriel39 commented on code in PR #66575:
URL: https://github.com/apache/doris/pull/66575#discussion_r3747150704
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AccessPathPlanCollector.java:
##########
@@ -92,21 +92,20 @@ public Void visitLogicalGenerate(LogicalGenerate<? extends
Plan> generate, State
for (Expression child : function.children()) {
exprCollector.collect(child);
}
+ } else if (function.arity() == 1 &&
function.child(0).getDataType().isVariantType()) {
Review Comment:
Fixed. Multi-argument explode now preserves the full selected Variant
argument because its container shape participates in generator semantics. Added
coverage for explode/explode_outer with whole and nested Variant arguments, and
updated the affected FE expectations.
##########
be/src/format_v2/column_mapper.cpp:
##########
@@ -1819,12 +1819,11 @@ static bool build_variant_leaf_path_projection(const
ColumnMapping& mapping,
std::ranges::all_of(value.substr(digits_begin),
[](unsigned char c) { return
std::isdigit(c); });
};
- if (path.size() != 1 || path[0].empty() || path[0] == "NULL" ||
- path[0].find('.') != std::string::npos || is_numeric_selector(path[0])
||
- !mapping.file_local_id.has_value()) {
- // Thrift currently carries access paths as strings without
segment-kind or escaping
- // metadata. Signed numeric tokens are therefore also ambiguous
between an array selector
- // and an object key, so only a single unambiguous key can be mapped
losslessly to a leaf.
+ if (path.empty() || !mapping.file_local_id.has_value() ||
Review Comment:
Fixed. A projected shredded Variant can now serialize only its retained
descendants when a binary element_at chain evaluates an intermediate object
prefix. Root materialization remains rejected, and a regression test verifies
the deep profile.address chain without exposing projected-out fields.
##########
be/src/format_v2/parquet/parquet_scan.cpp:
##########
@@ -535,14 +711,26 @@ Status plan_parquet_row_groups(const
NativeParquetMetadata& metadata,
std::vector<int> scan_range_selected;
RETURN_IF_ERROR(detail::select_native_row_groups_by_scan_range(
metadata.to_thrift(), scan_range, &row_group_first_rows,
&scan_range_selected));
- std::vector<int> metadata_selected;
- RETURN_IF_ERROR(select_row_groups_by_metadata(
- metadata.to_thrift(), file_schema, request, &scan_range_selected,
&metadata_selected,
- enable_bloom_filter, &plan->pruning_stats, timezone,
runtime_state, file_context,
- column_reader_profile, ParquetMetadataProbeMode::FOOTER_ONLY));
RETURN_IF_ERROR(build_native_row_group_read_plans(metadata, file_schema,
request,
- metadata_selected,
row_group_first_rows, plan,
- timezone, runtime_state,
file_context));
+ scan_range_selected,
row_group_first_rows,
+ plan, timezone,
runtime_state, file_context));
+ std::vector<RowGroupReadPlan> metadata_selected_plans;
+ metadata_selected_plans.reserve(plan->row_groups.size());
+ for (auto& row_group_plan : plan->row_groups) {
Review Comment:
Fixed. The immutable request-level physical leaf set is now built once and
reused across row groups. Only row groups that require a full Variant fallback
copy the baseline and add their fallback leaf delta; runtime-filter request
refreshes rebuild the baseline once. Added a wide 64-column, 3-row-group
regression test that verifies a single leaf-set build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
