jackye1995 commented on a change in pull request #4259:
URL: https://github.com/apache/iceberg/pull/4259#discussion_r823021154
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/BaseS3File.java
##########
@@ -19,20 +19,23 @@
package org.apache.iceberg.aws.s3;
+import java.util.List;
import org.apache.iceberg.aws.AwsProperties;
import org.apache.iceberg.metrics.MetricsContext;
import software.amazon.awssdk.http.HttpStatusCode;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.HeadObjectRequest;
import software.amazon.awssdk.services.s3.model.HeadObjectResponse;
import software.amazon.awssdk.services.s3.model.S3Exception;
+import software.amazon.awssdk.services.s3.model.Tag;
abstract class BaseS3File {
private final S3Client client;
private final S3URI uri;
private final AwsProperties awsProperties;
private HeadObjectResponse metadata;
private final MetricsContext metrics;
+ private List<Tag> tags;
Review comment:
private final, also should be named `writeTags`
##########
File path: aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java
##########
@@ -124,8 +125,8 @@ private LockManager initializeLockManager(Map<String,
String> properties) {
private FileIO initializeFileIO(Map<String, String> properties) {
String fileIOImpl = properties.get(CatalogProperties.FILE_IO_IMPL);
- if (fileIOImpl == null) {
- FileIO io = new S3FileIO();
+ if (fileIOImpl == null || fileIOImpl.equals(S3_FILE_IO_IMPL)) {
+ FileIO io = new S3FileIO(properties);
Review comment:
I don't think you need to do this, properties will be passed in during
`io.initialize(properties);` at next line anyway
##########
File path: aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java
##########
@@ -215,6 +216,11 @@
*/
public static final String CLIENT_ASSUME_ROLE_REGION =
"client.assume-role.region";
+ /**
+ * S3 Tag prefix used by {@link S3FileIO}
Review comment:
can be more descriptive, we need to at least say this is used to tag
objects when writing. Also would be great to provide:
1. an example of how to set write tags using prefix
2. a link to S3 tagging documentation
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -50,6 +54,7 @@
private transient S3Client client;
private MetricsContext metrics = MetricsContext.nullMetrics();
private final AtomicBoolean isResourceClosed = new AtomicBoolean(false);
+ private List<Tag> tags;
Review comment:
should this be a set instead? Could you check if the object hashcode is
implemented?
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -81,14 +86,38 @@ public S3FileIO(SerializableSupplier<S3Client> s3,
AwsProperties awsProperties)
this.awsProperties = awsProperties;
}
+ /**
+ * Constructor with custom s3 supplier and s3 tags
+ * <p>
+ * Calling {@link S3FileIO#initialize(Map)} will overwrite information set
in this constructor.
+ * param s3 s3 supplier
+ * @param properties catalog properties
+ */
+ public S3FileIO(SerializableSupplier<S3Client> s3, Map<String, String>
properties) {
Review comment:
the logic should be moved to `initialize`
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -50,6 +54,7 @@
private transient S3Client client;
private MetricsContext metrics = MetricsContext.nullMetrics();
private final AtomicBoolean isResourceClosed = new AtomicBoolean(false);
+ private List<Tag> tags;
Review comment:
also similar comment, should be named `writeTags`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
