xloya commented on a change in pull request #3977:
URL: https://github.com/apache/iceberg/pull/3977#discussion_r801204803
##########
File path: core/src/main/java/org/apache/iceberg/TableProperties.java
##########
@@ -210,6 +211,20 @@ private TableProperties() {
public static final String SPARK_WRITE_PARTITIONED_FANOUT_ENABLED =
"write.spark.fanout.enabled";
public static final boolean SPARK_WRITE_PARTITIONED_FANOUT_ENABLED_DEFAULT =
false;
+ public static final String PARTITIONED_FANOUT_WRITERS_CACHE_SIZE =
"write.partition.fanout.writers-cache-size";
+ public static final int PARTITIONED_FANOUT_WRITERS_CACHE_SIZE_DEFAULT =
Integer.MAX_VALUE;
+
+ public static final String PARTITIONED_FANOUT_WRITERS_CACHE_EVICT_MS =
+ "write.partition.fanout.writers-cache-eviction-ms";
+ public static final long PARTITIONED_FANOUT_WRITERS_CACHE_EVICT_MS_DEFAULT =
TimeUnit.MINUTES.toMillis(5);
Review comment:
@rdblue The reason for adding the timeout option is that it is often
difficult to determine the memory consumption based on the number of writers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
