yathindranath commented on a change in pull request #497: Support retaining
last N snapshots
URL: https://github.com/apache/incubator-iceberg/pull/497#discussion_r333790897
##########
File path: core/src/main/java/org/apache/iceberg/RemoveSnapshots.java
##########
@@ -77,8 +79,34 @@ public ExpireSnapshots expireSnapshotId(long
expireSnapshotId) {
@Override
public ExpireSnapshots expireOlderThan(long timestampMillis) {
- LOG.info("Expiring snapshots older than: {} ({})", new
Date(timestampMillis), timestampMillis);
- this.expireOlderThan = timestampMillis;
+ updateExpireOlderThan(timestampMillis, true);
+ return this;
+ }
+
+ private boolean updateExpireOlderThan(long timestampMillis, boolean
fromExpireOlderThan) {
+ if (expireOlderThan == null ||
+ expireOlderThan > timestampMillis ||
+ this.isExpireOlderThanSet == fromExpireOlderThan) {
+ LOG.info("Expiring snapshots older than: {} ({})", new
Date(timestampMillis), timestampMillis);
+ this.expireOlderThan = timestampMillis;
+ this.isExpireOlderThanSet = fromExpireOlderThan;
+ return true;
+ }
+ return false;
+ }
+
+ @Override
+ public ExpireSnapshots retainLast(int numSnapshots) {
Review comment:
Use case is opposite of `expireSnapshotId`.
If you are experimenting with different optimizations like
`commit.manifest.target-size-bytes` , `RewriteManifests.clusterBy` will end up
with lots of manifest and snapshots. After you get the configurations correct,
you can expire older snapshots.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
