JingsongLi commented on code in PR #10575:
URL: https://github.com/apache/paimon/pull/10575#discussion_r4282206536
##########
paimon-core/src/main/java/org/apache/paimon/iceberg/IcebergCommitCallback.java:
##########
@@ -1108,39 +1110,33 @@ private void createMetadataWithBase(
addedFiles.values().stream().mapToLong(p ->
p.getRight().rowCount()).sum();
metrics.addedFilesSize =
addedFiles.values().stream().mapToLong(p ->
p.getRight().fileSize()).sum();
- metrics.deletedDataFiles = removedFiles.size();
+ metrics.deletedDataFiles = deletedFiles.size();
metrics.deletedRecords =
- removedFiles.values().stream().mapToLong(p ->
p.getRight().rowCount()).sum();
+
deletedFiles.stream().mapToLong(IcebergDataFileMeta::recordCount).sum();
metrics.deletedFilesSize =
- removedFiles.values().stream().mapToLong(p ->
p.getRight().fileSize()).sum();
- metrics.changedPartitionCount = modifiedPartitionsSet.size();
-
- IcebergSnapshot baseSnapshot = baseMetadata.currentSnapshot();
+
deletedFiles.stream().mapToLong(IcebergDataFileMeta::fileSizeInBytes).sum();
+ Set<BinaryRow> changedPartitions = new HashSet<>();
Review Comment:
[P2] Include partitions changed only by deletion vectors
This now counts only data-file changes, so an Iceberg v3/DV64 primary-key
table can publish a new delete file for a partition while reporting
`changed-partition-count=0`. I reproduced this with the real Iceberg reader:
write and fully compact 1,000 keys to L2, then delete one existing key and let
lookup/minor compaction process the L0 tombstone. The original data file
remains unchanged, Iceberg reads 999 rows, and the new DELETES manifest has one
ADDED file (`total-position-deletes=1`, `total-delete-files=1`), but this
summary reports zero changed partitions. The prior counter reports one for this
same path. Please include partitions affected by added/removed delete files as
well as data files, and add a v3+DV64 regression case. The current test helper
skips DELETES manifests before collecting changed partitions, so it cannot
detect this regression. [Iceberg's summary
definition](https://iceberg.apache.org/spec/#snapshot-summary) includes
partitions with files added or removed, wi
thout restricting them to data files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
