github-actions[bot] commented on code in PR #65253:
URL: https://github.com/apache/doris/pull/65253#discussion_r3584359959
##########
be/src/format/transformer/vparquet_transformer.cpp:
##########
@@ -380,10 +385,21 @@ Status
VParquetTransformer::collect_file_statistics_after_close(TIcebergColumnSt
if (has_any_null_count) {
stats->__set_null_value_counts(null_value_counts);
}
- if (has_any_min_max) {
+ if (!lower_bounds.empty()) {
stats->__set_lower_bounds(lower_bounds);
stats->__set_upper_bounds(upper_bounds);
}
return Status::OK();
}
+
+Status VParquetTransformer::_can_write_iceberg_bounds(int32_t field_id, bool*
can_write) const {
+ for (const auto& field : _iceberg_schema->root_struct().fields()) {
Review Comment:
This root-only check drops valid Parquet bounds for nested primitive fields.
The stats loop above is keyed by each physical Parquet column's
`schema_node()->field_id()`, and `ArrowSchemaUtil::convert_to()` recursively
writes `PARQUET:field_id` for struct children/list elements/map key-values, so
a schema like `struct<c1:int>` reaches this helper with the child field ID
rather than the root struct ID. Because only `root_struct().fields()` are
scanned here, that primitive child ID falls through to `can_write=false`, and
FE commits the file without the lower/upper bounds that Iceberg can use for
nested-field pruning. Please resolve the Iceberg type by field ID recursively,
or otherwise keep primitive nested field IDs while suppressing only
non-primitive container bounds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
