lcspinter commented on a change in pull request #2325:
URL: https://github.com/apache/iceberg/pull/2325#discussion_r595335982
##########
File path: hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java
##########
@@ -50,25 +55,37 @@
import org.apache.iceberg.exceptions.NoSuchTableException;
import org.apache.iceberg.hadoop.HadoopFileIO;
import org.apache.iceberg.io.FileIO;
-import org.apache.iceberg.relocated.com.google.common.base.Joiner;
+import
org.apache.iceberg.relocated.com.google.common.annotations.VisibleForTesting;
import org.apache.iceberg.relocated.com.google.common.base.MoreObjects;
import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
import org.apache.iceberg.relocated.com.google.common.collect.ImmutableList;
import org.apache.iceberg.relocated.com.google.common.collect.Maps;
+import
org.apache.iceberg.relocated.com.google.common.util.concurrent.ThreadFactoryBuilder;
+import org.apache.iceberg.util.Pair;
import org.apache.iceberg.util.PropertyUtil;
import org.apache.thrift.TException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-public class HiveCatalog extends BaseMetastoreCatalog implements Closeable,
SupportsNamespaces, Configurable {
+public class HiveCatalog extends BaseMetastoreCatalog implements
SupportsNamespaces, Configurable {
private static final Logger LOG = LoggerFactory.getLogger(HiveCatalog.class);
+ public static final String CACHE_CLEANER_INTERVAL =
"iceberg.hive.client-pool-cache-cleaner-interval";
+ private static final long CACHE_CLEANER_INTERVAL_DEFAULT =
TimeUnit.SECONDS.toMillis(30);
+ public static final String CACHE_EVICTION_INTERVAL =
"iceberg.hive.client-pool-cache-eviction-interval";
+ private static final long CACHE_EVICTION_INTERVAL_DEFAULT =
TimeUnit.MINUTES.toMillis(5);
Review comment:
@marton-bod There is no particular reason for choosing this value.
If we keep the ClientPool and its connections open we might end up with a
memory leak. (With the removal of the finalize() method nobody is closing
them). To tackle this issue, I decided to close every unused ClientPool after a
certain amount of time. This threshold is configurable and should be updated
based on the load/usage of the cluster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
