anoopj commented on code in PR #2985:
URL: https://github.com/apache/iceberg-rust/pull/2985#discussion_r3763799539
##########
crates/iceberg/src/arrow/reader/pipeline.rs:
##########
@@ -250,6 +252,42 @@ impl FileScanTaskReader {
let mut record_batch_stream_builder =
ParquetRecordBatchStreamBuilder::new_with_metadata(parquet_file_reader,
arrow_metadata);
+ // Whether the file physically carries the
`_last_updated_sequence_number` column
+ // (some engines, e.g. Iceberg Java on rewrite, write it per-row),
resolved by its
+ // embedded field id against the Parquet schema.
+ let project_last_updated_seq = task
+ .project_field_ids()
+ .contains(&RESERVED_FIELD_ID_LAST_UPDATED_SEQUENCE_NUMBER);
+
+ // Parquet leaf index of the physical column, if present by embedded
field id.
+ // `build_field_id_map` is all-or-nothing (`None` if any column lacks
an id), so a
+ // file mixing id-bearing and id-less columns is rejected below rather
than
+ // coalesced -- safe, and consistent with the rest of the reader.
+ let phys_last_updated_seq_leaf = if project_last_updated_seq {
Review Comment:
Went with accept + simpler comment. Reworded the comment to state the
mixed-id file is rejected as a behavior choice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
