rdblue commented on a change in pull request #3120:
URL: https://github.com/apache/iceberg/pull/3120#discussion_r717125639
##########
File path: core/src/main/java/org/apache/iceberg/TableProperties.java
##########
@@ -123,6 +123,9 @@ private TableProperties() {
public static final String SPLIT_OPEN_FILE_COST =
"read.split.open-file-cost";
public static final long SPLIT_OPEN_FILE_COST_DEFAULT = 4 * 1024 * 1024; //
4MB
+ public static final String READ_DELETE_FILES_WORKER_POOL_SIZE =
"read.deletes.num-threads";
+ public static final int READ_DELETE_FILES_WORKER_POOL_SIZE_DEFAULT = 1; //
read delete files in serial.
Review comment:
I don't think this makes much sense as a table property. Table
properties are for table configuration, but this is an engine concern. Many
engines handle parallelism internally so this wouldn't be appropriate. I think
that Flink should expose a setting and manage the thread pool for using a
delete file reader pool.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
