rajarshisarkar opened a new pull request #4342: URL: https://github.com/apache/iceberg/pull/4342
This change adds [S3 Tags](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-tagging.html) to the objects while deleting using `S3FileIO`. Users can pass their custom tags as part of the catalog properties. This would allow the users to [manage their storage lifecycle](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html). Spark SQL launch command: ``` sh spark-sql --conf spark.sql.catalog.my_catalog=org.apache.iceberg.spark.SparkCatalog \ --conf spark.sql.catalog.my_catalog.warehouse=s3://iceberg-warehouse/s3-tagging \ --conf spark.sql.catalog.my_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog \ --conf spark.sql.catalog.my_catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO \ --conf spark.sql.catalog.my_catalog.s3.write.tags.my_key=my_val \ --conf spark.sql.catalog.my_catalog.s3.write.tags.my_key2=my_val2 \ --conf spark.sql.catalog.my_catalog.s3.delete.tags.my_key3=my_val3 ``` Tags in S3 after delete: ``` aws s3api get-object-tagging --bucket iceberg-warehouse --key s3-tagging/metadata/00000-5d37f925-be01-44d0-87fd-15513606ff6b.metadata.json { "TagSet": [ { "Key": "my_key2", "Value": "my_val2" }, { "Key": "my_key3", "Value": "my_val3" }, { "Key": "my_key", "Value": "my_val" } ] } ``` Note: `my_key=my_val` and `my_key2=my_val2` are the tags when the object was written. ---- cc: @jackye1995 @arminnajafi @singhpk234 @amogh-jahagirdar @xiaoxuandev @yyanyy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
