rdblue commented on a change in pull request #3120:
URL: https://github.com/apache/iceberg/pull/3120#discussion_r742128756
##########
File path: data/src/main/java/org/apache/iceberg/data/DeleteFilter.java
##########
@@ -62,6 +68,13 @@
MetadataColumns.DELETE_FILE_PATH,
MetadataColumns.DELETE_FILE_POS);
+ private static final int READ_DELETES_WORKER_POOL_SIZE_DEFAULT = 0; // read
delete files in serial.
+ private static final int READ_DELETES_WORKER_POOL_SIZE =
SystemProperties.getInteger(
+ SystemProperties.READ_DELETE_FILES_WORKER_POOL_SIZE,
READ_DELETES_WORKER_POOL_SIZE_DEFAULT);
+ private static final ExecutorService READ_DELETES_SERVICE =
READ_DELETES_WORKER_POOL_SIZE <= 1 ? null :
+ MoreExecutors.getExitingExecutorService((ThreadPoolExecutor)
Executors.newFixedThreadPool(
+ READ_DELETES_WORKER_POOL_SIZE, new
ThreadFactoryBuilder().setNameFormat("Read-delete-Service-%d").build()));
Review comment:
It looks like the changes other than the ones in this file are to make
it easier to use `ParallelIterable`. Can we separate those changes from these
for Flink and `DeleteFilter`? A separate PR for the `CloseableIterable` changes
would make it easier to review this when there are more Flink changes to handle
the executor service and pass it into `DeleteFilter`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
