jackye1995 commented on a change in pull request #4259:
URL: https://github.com/apache/iceberg/pull/4259#discussion_r821273557
##########
File path: aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java
##########
@@ -73,11 +74,17 @@
import software.amazon.awssdk.services.glue.model.Table;
import software.amazon.awssdk.services.glue.model.TableInput;
import software.amazon.awssdk.services.glue.model.UpdateDatabaseRequest;
+import software.amazon.awssdk.services.s3.model.Tag;
public class GlueCatalog extends BaseMetastoreCatalog
implements Closeable, SupportsNamespaces, Configurable<Configuration> {
private static final Logger LOG = LoggerFactory.getLogger(GlueCatalog.class);
+ private static final String S3_FILE_IO_IMPL =
"org.apache.iceberg.aws.s3.S3FileIO";
+ private static final String S3_TAGS_FORMAT = "s3.tags.format";
Review comment:
Let's first provide a generic solution, before adding specific tag keys
for users. I think it's better if in `AwsProperties` we introduce the property
prefix `s3.write.tags`, so that users can define any free-form prefixes, for
example:
```
spark-sql --packages $DEPENDENCIES \
--conf
spark.sql.catalog.my_catalog=org.apache.iceberg.spark.SparkCatalog \
--conf
spark.sql.catalog.my_catalog.warehouse=s3://my-bucket/my/key/prefix \
--conf
spark.sql.catalog.my_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog
\
--conf spark.sql.catalog.my_catalog.s3.write.tags.my_key=my_val \
--conf spark.sql.catalog.my_catalog.s3.write.tags.my_key2=my_val2
```
This can be passed to `GlueCatalog.initialize`, and we can extract the tags
`my_key->my_val` and `my_key2->my_val2`, and stored in S3FileIO to be used
during a write.
`write` in `s3.write.tags` is useful in case we would like to introduce
other tagging, such as `s3.delete.tags`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
