github-actions[bot] commented on code in PR #66941:
URL: https://github.com/apache/doris/pull/66941#discussion_r3829744272
##########
be/src/format_v2/parquet/parquet_statistics.cpp:
##########
@@ -1086,10 +1099,10 @@ bool check_shredded_variant_statistics(
const auto shredding = resolve_variant_shredding(file_schema, request,
*predicate);
if (!shredding.has_value() || shredding->typed_value->leaf_column_id <
0 ||
shredding->typed_value->leaf_column_id >=
static_cast<int>(row_group.columns.size()) ||
- // Partially shredded object residuals contain only keys not
present in typed_value,
- // so ancestors cannot shadow this path. The terminal fallback is
the sole guard.
- shredding->fallback_value == nullptr ||
- !fallback_is_all_null(row_group, *shredding->fallback_value) ||
+ // Doris resolves the path from the first ancestor whose typed
value is absent. Typed
+ // leaf statistics are therefore authoritative only when every
possible fallback on
+ // the path is proven NULL for this row group.
+ !all_fallbacks_are_all_null(row_group, shredding->fallback_values)
||
Review Comment:
[P2] Keep pruning for unrelated ancestor residuals
This gate disables shredded-leaf footer and Page Index pruning unless the
root, every intermediate, and terminal `value` column are entirely NULL. For a
conforming partially shredded object, however, ancestor residual keys are
disjoint from the shredded child namespace; if an ancestor object `typed_value`
is absent, its logical value is non-object and cannot continue this object-key
path. Only the terminal wrapper's `value` can supply the compared primitive. A
row group with ordinary unrelated residual keys therefore reads all rows/pages
even when the terminal fallback is all NULL. Please retain the terminal
all-NULL guard without gating on ancestor residuals, and cover a conforming
partially shredded object in both pruning paths. See the [Apache Parquet
VariantShredding object
contract](https://github.com/apache/parquet-format/blob/master/VariantShredding.md#objects).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
