anoopj commented on code in PR #3058:
URL: https://github.com/apache/iceberg-rust/pull/3058#discussion_r3845707843
##########
crates/iceberg/src/arrow/reader/pipeline.rs:
##########
@@ -314,25 +345,29 @@ impl FileScanTaskReader {
// which `get_arrow_projection_mask` maps to "read all columns" (so
`COUNT(*)` still
// gets a row count). Downgrade that to "read no data columns" when a
row-count
// source exists independently of the data columns: the RowNumber
virtual column
- // (installed above under `project_pos`) or a physical metadata leaf
unioned in
- // below. Pure-constant / `COUNT(*)` projections have neither and must
keep reading
- // all columns to preserve the row count. Any future physical metadata
leaf (e.g. a
- // `_row_id` read path) is likewise a row source.
+ // (installed above under `need_row_number`, which covers `_pos` and
`_row_id`
+ // synthesis) or a physical `_last_updated_sequence_number` leaf
unioned in below
+ // (that column does not install RowNumber, so it is a separate
source). Pure-constant
+ // / `COUNT(*)` projections have neither and must keep reading all
columns to preserve
+ // the row count.
//
- // This runs BEFORE the union so the physical leaf is added onto a
`none` base,
- // pruning the read to just that leaf (`union` with an `all` base
stays `all`).
+ // This runs BEFORE the union so the physical leaves are added onto a
`none` base,
+ // pruning the read to just those leaves (`union` with an `all` base
stays `all`).
if project_field_ids_without_metadata.is_empty()
- && (project_pos || coalesce_last_updated_seq_leaf.is_some())
+ && (need_row_number || coalesce_last_updated_seq_leaf.is_some())
Review Comment:
Left it as-is with a comment at the metadata-only downgrade guard marking
it a known follow-up, calling out that it applies to both _row_id and
_last_updated_sequence_number when first_row_id is null. We can do that in a
followup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
