[
https://issues.apache.org/jira/browse/HIVE-26794?focusedWorklogId=837829&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-837829
]
ASF GitHub Bot logged work on HIVE-26794:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Jan/23 08:54
Start Date: 09/Jan/23 08:54
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on code in PR #3817:
URL: https://github.com/apache/hive/pull/3817#discussion_r1064397982
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java:
##########
@@ -72,6 +72,17 @@ public DataSource create(Configuration hdpConfig, int
maxPoolSize) throws SQLExc
config.setPoolName(poolName);
}
+ // It's kind of a waste to create a fixed size connection pool as same as
the TxnHandler#connPool,
+ // TxnHandler#connPoolMutex is mostly used for MutexAPI that is primarily
designed to
+ // provide coarse-grained mutex support to maintenance tasks running
inside the Metastore,
+ // add minimumIdle=2 and idleTimeout=5min to the pool, so that the
connection pool can retire
+ // the idle connection aggressively, this will make Metastore more
scalable especially if
+ // there is a leader in the warehouse.
+ if ("mutex".equals(poolName)) {
+ config.setMinimumIdle(Math.min(maxPoolSize, 2));
Review Comment:
For hikari, the idle connections would be evicted from the pool only when
minimumIdle is defined to be less than maximumPoolSize, however for DbCP, the
idle object evictor thread runs only when timeBetweenEvictionRunsMillis is
positive:
https://github.com/apache/commons-pool/blob/master/src/main/java/org/apache/commons/pool2/impl/BaseGenericObjectPool.java#L1810-L1830
Fixed Hikari not to override the idle config if the user has already
provided the `hikaricp.minimumIdle`.
Issue Time Tracking
-------------------
Worklog Id: (was: 837829)
Time Spent: 4h 50m (was: 4h 40m)
> Explore retiring TxnHandler#connPoolMutex idle connections
> ----------------------------------------------------------
>
> Key: HIVE-26794
> URL: https://issues.apache.org/jira/browse/HIVE-26794
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
> Reporter: Zhihua Deng
> Assignee: Zhihua Deng
> Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Instead of creating a fixed size connection pool for TxnHandler#MutexAPI, the
> pool can be assigned to a more dynamic size pool due to:
> * TxnHandler#MutexAPI is primarily designed to provide coarse-grained mutex
> support to maintenance tasks running inside the Metastore, these tasks are
> not user faced;
> * A fixed size connection pool as same as the pool used in ObjectStore is a
> waste for other non leaders in the warehouse;
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
