lilei1128 commented on code in PR #7537:
URL: https://github.com/apache/paimon/pull/7537#discussion_r3386006180
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/source/CompactorSourceBuilder.java:
##########
@@ -141,6 +142,26 @@ public DataStreamSource<RowData> build() {
});
dataStream = new DataStreamSource<>(filterStream);
}
+ CoreOptions coreOptions = table.coreOptions();
Review Comment:
Thanks for the review!
You're right that the compactor sink still runs PartitionExpire during
commit. However, we intentionally kept this behavior rather than disabling
partition expiration in the commit path.
The reason: in the typical deployment pattern where write jobs use
write-only=true, the compaction job's commit is the only path that triggers
PartitionExpire. If we disable it here too, expired partitions would never be
cleaned up automatically — users would have to manually run
ExpirePartitionsAction to delete them. That feels worse than the current side
effect.
So the goal of this option is specifically to **avoid unnecessary compaction
IO** on partitions that are about to be deleted (no file merging, no writing
compacted files that get immediately dropped).
I have updated the documentation (both CoreOptions.java and
dedicated-compaction.mdx) to explicitly note that expired partitions may still
be deleted during the commit phase as a side effect, so users are aware of this
behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
