jopdorp commented on code in PR #3253:
URL: https://github.com/apache/iceberg-rust/pull/3253#discussion_r4258446164
##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -85,6 +86,12 @@ pub(crate) trait SnapshotProduceOperation: Send + Sync {
&self,
snapshot_produce: &SnapshotProducer<'_>,
) -> impl Future<Output = Result<Vec<ManifestFile>>> + Send;
+
+ /// Returns whether the snapshot summary resets the table totals. Defaults
to every
+ /// overwrite; an operation that overwrites only some files returns
`false`.
+ fn truncate_full_table(&self) -> bool {
+ self.operation() == Operation::Overwrite
Review Comment:
the note is already on truncate_full_table: partial overwrites return false.
kept the existing default.
##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -396,6 +412,19 @@ impl<'a> SnapshotProducer<'a> {
);
}
+ for data_file in &self.deleted_data_files {
Review Comment:
added the commit check in 683fae4fd and documented that the caller metrics
must match the manifest entries. it counts only data-file deletions written by
this commit.
##########
crates/iceberg/src/transaction/snapshot.rs:
##########
@@ -396,6 +412,19 @@ impl<'a> SnapshotProducer<'a> {
);
}
+ for data_file in &self.deleted_data_files {
+ let partition_spec = table_metadata
+ .partition_spec_by_id(data_file.partition_spec_id)
+ .ok_or_else(|| {
+ invalid_data!("Unknown partition spec id {}",
data_file.partition_spec_id)
+ })?;
+ summary_collector.remove_file(
+ data_file,
+ table_metadata.current_schema().clone(),
Review Comment:
fixed the reported int/long formatting cases in #3432, also included here as
5f3693808. the regression covers identity and truncate partitions after their
source column is dropped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
