advancedxy commented on code in PR #2695:
URL: https://github.com/apache/iceberg-rust/pull/2695#discussion_r3457295381
##########
crates/iceberg/src/arrow/reader/pipeline.rs:
##########
@@ -248,6 +250,16 @@ impl FileScanTaskReader {
record_batch_transformer_builder.with_constant(RESERVED_FIELD_ID_FILE,
file_datum);
}
+ if task
+ .project_field_ids()
+ .contains(&RESERVED_FIELD_ID_SPEC_ID)
+ && let Some(partition_spec) = &task.partition_spec
Review Comment:
I think we should always add spec_id into the constant map if it's
selected/projected.
The task.partition_spec should be present even for unpartitioned
table(spec_id = 0).
##########
crates/iceberg/src/scan/context.rs:
##########
@@ -96,6 +99,7 @@ impl ManifestFileContext {
delete_file_index: delete_file_index.clone(),
name_mapping: name_mapping.clone(),
case_sensitive,
+ partition_spec: partition_spec.clone(),
Review Comment:
the partition spec is already in mainifest_file, like the partition_spec_id.
maybe we could just use that and avoid embedding partition_spec into
ManifestFileContext
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
