marton-bod commented on a change in pull request #2228:
URL: https://github.com/apache/iceberg/pull/2228#discussion_r573135683
##########
File path: mr/src/main/java/org/apache/iceberg/mr/InputFormatConfig.java
##########
@@ -58,8 +58,11 @@ private InputFormatConfig() {
public static final String LOCATION_PROVIDER =
"iceberg.mr.location.provider";
public static final String ENCRYPTION_MANAGER =
"iceberg.mr.encription.manager";
- public static final String COMMIT_THREAD_POOL_SIZE =
"iceberg.mr.commit.thread.pool.size";
- public static final int COMMIT_THREAD_POOL_SIZE_DEFAULT = 10;
+ public static final String OUTPUT_TABLES = "iceberg.mr.output.tables";
+ public static final String COMMIT_TABLE_THREAD_POOL_SIZE =
"iceberg.mr.commit.table.thread.pool.size";
+ public static final int COMMIT_TABLE_THREAD_POOL_SIZE_DEFAULT = 10;
Review comment:
What's the average number of tables we'll write to in a typical
insert/multitable-insert? Isn't it typically a lot lower than 10? I'm worried
we'll pay a performance penalty for starting a threadpool with at least 10
threads every time we perform a write, even when we write to a single table
only (which I'd think is the most common scenario).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
