singhpk234 commented on a change in pull request #4342:
URL: https://github.com/apache/iceberg/pull/4342#discussion_r831255929
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -94,6 +107,19 @@ public S3FileIO(SerializableSupplier<S3Client> s3) {
public S3FileIO(SerializableSupplier<S3Client> s3, AwsProperties
awsProperties) {
this.s3 = s3;
this.awsProperties = awsProperties;
+ if (executorService == null) {
+ synchronized (S3OutputStream.class) {
+ if (executorService == null) {
+ executorService = MoreExecutors.getExitingExecutorService(
+ (ThreadPoolExecutor) Executors.newFixedThreadPool(
+ awsProperties.s3FileIoDeleteTagThreads(),
+ new ThreadFactoryBuilder()
+ .setDaemon(true)
+ .setNameFormat("iceberg-s3fileio-deletetags-%d")
+ .build()));
+ }
+ }
+ }
Review comment:
Can we shift this to a private function
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -65,6 +77,7 @@
private transient S3Client client;
private MetricsContext metrics = MetricsContext.nullMetrics();
private final AtomicBoolean isResourceClosed = new AtomicBoolean(false);
+ private static volatile ExecutorService executorService;
Review comment:
[question] Any reason, executor service need's to static ?
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -125,6 +163,20 @@ public void deleteFile(String path) {
*/
@Override
public void deleteFiles(Iterable<String> paths) throws
BulkDeletionFailureException {
+ if (awsProperties.s3DeleteTags() != null &&
!awsProperties.s3DeleteTags().isEmpty()) {
+ Tasks.foreach(paths)
+ .noRetry()
+ .executeWith(executorService)
+ .suppressFailureWhenFinished()
+ .onFailure((path, exc) -> LOG.warn("Failed to add delete tags: {}",
path, exc))
+ .run(this::doSoftDelete);
+ return;
Review comment:
we should not return here as the flow will newer reach below if
deleteTagging is enabled
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
