rajarshisarkar commented on a change in pull request #4342:
URL: https://github.com/apache/iceberg/pull/4342#discussion_r837393505
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -155,6 +190,31 @@ public void deleteFiles(Iterable<String> paths) throws
BulkDeletionFailureExcept
}
}
+ private void tagFileToDelete(String path, Set<Tag> deleteTags) throws
S3Exception {
+ S3URI location = new S3URI(path);
Review comment:
Nice catch, missed on added this after merging master.
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -108,6 +118,18 @@ public OutputFile newOutputFile(String path) {
@Override
public void deleteFile(String path) {
+ if (awsProperties.s3DeleteTags() != null &&
!awsProperties.s3DeleteTags().isEmpty()) {
+ try {
+ tagFileToDelete(path, awsProperties.s3DeleteTags());
+ } catch (S3Exception e) {
+ LOG.warn("Failed to add delete tags: {}", path, e);
Review comment:
Made the changes.
##########
File path: aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java
##########
@@ -303,6 +303,32 @@
*/
public static final String S3_WRITE_TAGS_PREFIX = "s3.write.tags.";
+ /**
+ * Used by {@link S3FileIO} to tag objects when deleting. When this config
is set, objects are
+ * tagged with the configured key-value pairs before deletion. This is
considered a soft-delete,
+ * because users are able to configure tag-based object lifecycle policy at
bucket level to
+ * transition objects to different tiers.
+ * <p>
+ * For more details, see
https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html
+ * <p>
+ * Example: s3.delete.tags.my_key=my_val
+ */
+ public static final String S3_DELETE_TAGS_PREFIX = "s3.delete.tags.";
+
+ /**
+ * Number of threads to use for adding delete tags to S3 objects, default to
{@link
+ * Runtime#availableProcessors()}
+ */
+ public static final String S3FILEIO_DELETE_THREADS = "s3.delete.num-threads";
+
+ /**
+ * Determines if S3FileIO delete the object when io.delete() is called,
default to true. Once
Review comment:
I have made the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
