[
https://issues.apache.org/jira/browse/HIVE-22469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969533#comment-16969533
]
David Mollitor edited comment on HIVE-22469 at 11/7/19 8:09 PM:
----------------------------------------------------------------
Managing 10 connections, bunch of extra noise in the logs.
{code:none}
2019-11-06T11:45:30,741 DEBUG [HikariPool-3 housekeeper] pool.HikariPool:
HikariPool-3 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-11-06T11:45:30,741 DEBUG [HikariPool-2 housekeeper] pool.HikariPool:
HikariPool-2 - Pool stats (total=10, active=1, idle=9, waiting=0)
2019-11-06T11:45:40,809 DEBUG [HikariPool-1 connection closer] pool.PoolBase:
HikariPool-1 - Closing connection
org.apache.derby.impl.jdbc.EmbedConnection@442851558 (XID = 184), (SESSIONID =
19), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null) : (connection has passed maxLifetime)
2019-11-06T11:45:40,812 DEBUG [HikariPool-1 connection adder] pool.HikariPool:
HikariPool-1 - Added connection
org.apache.derby.impl.jdbc.EmbedConnection@2095252197 (XID = 17066), (SESSIONID
= 3), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null)
2019-11-06T11:45:49,020 DEBUG [HikariPool-1 connection closer] pool.PoolBase:
HikariPool-1 - Closing connection
org.apache.derby.impl.jdbc.EmbedConnection@416110958 (XID = 174), (SESSIONID =
9), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null) : (connection has passed maxLifetime)
2019-11-06T11:45:49,022 DEBUG [HikariPool-1 connection adder] pool.HikariPool:
HikariPool-1 - Added connection
org.apache.derby.impl.jdbc.EmbedConnection@73417930 (XID = 17068), (SESSIONID =
5), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null)
2019-11-06T11:45:49,363 DEBUG [HikariPool-1 connection closer] pool.PoolBase:
HikariPool-1 - Closing connection
org.apache.derby.impl.jdbc.EmbedConnection@1998766497 (XID = 180), (SESSIONID =
15), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null) : (connection has passed maxLifetime)
2019-11-06T11:45:49,364 DEBUG [HikariPool-1 connection adder] pool.HikariPool:
HikariPool-1 - Added connection
org.apache.derby.impl.jdbc.EmbedConnection@1163742961 (XID = 17070), (SESSIONID
= 7), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null)
2019-11-06T11:45:52,762 DEBUG [HikariPool-1 housekeeper] pool.HikariPool:
HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-11-06T11:45:53,260 DEBUG [HikariPool-1 connection closer] pool.PoolBase:
HikariPool-1 - Closing connection
org.apache.derby.impl.jdbc.EmbedConnection@478356754 (XID = 178), (SESSIONID =
13), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null) : (connection has passed maxLifetime)
2019-11-06T11:45:53,261 DEBUG [HikariPool-1 connection adder] pool.HikariPool:
HikariPool-1 - Added connection
org.apache.derby.impl.jdbc.EmbedConnection@1959963107 (XID = 17072), (SESSIONID
= 9), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null)
2019-11-06T11:46:00,298 DEBUG [HikariPool-1 connection closer] pool.PoolBase:
HikariPool-1 - Closing connection
org.apache.derby.impl.jdbc.EmbedConnection@605147393 (XID = 182), (SESSIONID =
17), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null) : (connection has passed maxLifetime)
2019-11-06T11:46:00,299 DEBUG [HikariPool-1 connection adder] pool.HikariPool:
HikariPool-1 - Added connection
org.apache.derby.impl.jdbc.EmbedConnection@245074122 (XID = 17074), (SESSIONID
= 11), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null)
2019-11-06T11:46:00,741 DEBUG [HikariPool-3 housekeeper] pool.HikariPool:
HikariPool-3 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-11-06T11:46:00,741 DEBUG [HikariPool-2 housekeeper] pool.HikariPool:
HikariPool-2 - Pool stats (total=10, active=1, idle=9, waiting=0)
{code}
was (Author: belugabehr):
Managing 10 connections, bunch of extra noise in the logs.
{code}
2019-11-06T11:45:30,741 DEBUG [HikariPool-3 housekeeper] pool.HikariPool:
HikariPool-3 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-11-06T11:45:30,741 DEBUG [HikariPool-2 housekeeper] pool.HikariPool:
HikariPool-2 - Pool stats (total=10, active=1, idle=9, waiting=0)
2019-11-06T11:45:40,809 DEBUG [HikariPool-1 connection closer] pool.PoolBase:
HikariPool-1 - Closing connection
org.apache.derby.impl.jdbc.EmbedConnection@442851558 (XID = 184), (SESSIONID =
19), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null) : (connection has passed maxLifetime)
2019-11-06T11:45:40,812 DEBUG [HikariPool-1 connection adder] pool.HikariPool:
HikariPool-1 - Added connection
org.apache.derby.impl.jdbc.EmbedConnection@2095252197 (XID = 17066), (SESSIONID
= 3), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null)
2019-11-06T11:45:49,020 DEBUG [HikariPool-1 connection closer] pool.PoolBase:
HikariPool-1 - Closing connection
org.apache.derby.impl.jdbc.EmbedConnection@416110958 (XID = 174), (SESSIONID =
9), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null) : (connection has passed maxLifetime)
2019-11-06T11:45:49,022 DEBUG [HikariPool-1 connection adder] pool.HikariPool:
HikariPool-1 - Added connection
org.apache.derby.impl.jdbc.EmbedConnection@73417930 (XID = 17068), (SESSIONID =
5), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null)
2019-11-06T11:45:49,363 DEBUG [HikariPool-1 connection closer] pool.PoolBase:
HikariPool-1 - Closing connection
org.apache.derby.impl.jdbc.EmbedConnection@1998766497 (XID = 180), (SESSIONID =
15), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null) : (connection has passed maxLifetime)
2019-11-06T11:45:49,364 DEBUG [HikariPool-1 connection adder] pool.HikariPool:
HikariPool-1 - Added connection
org.apache.derby.impl.jdbc.EmbedConnection@1163742961 (XID = 17070), (SESSIONID
= 7), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null)
2019-11-06T11:45:52,762 DEBUG [HikariPool-1 housekeeper] pool.HikariPool:
HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-11-06T11:45:53,260 DEBUG [HikariPool-1 connection closer] pool.PoolBase:
HikariPool-1 - Closing connection
org.apache.derby.impl.jdbc.EmbedConnection@478356754 (XID = 178), (SESSIONID =
13), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null) : (connection has passed maxLifetime)
2019-11-06T11:45:53,261 DEBUG [HikariPool-1 connection adder] pool.HikariPool:
HikariPool-1 - Added connection
org.apache.derby.impl.jdbc.EmbedConnection@1959963107 (XID = 17072), (SESSIONID
= 9), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null)
2019-11-06T11:46:00,298 DEBUG [HikariPool-1 connection closer] pool.PoolBase:
HikariPool-1 - Closing connection
org.apache.derby.impl.jdbc.EmbedConnection@605147393 (XID = 182), (SESSIONID =
17), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null) : (connection has passed maxLifetime)
2019-11-06T11:46:00,299 DEBUG [HikariPool-1 connection adder] pool.HikariPool:
HikariPool-1 - Added connection
org.apache.derby.impl.jdbc.EmbedConnection@245074122 (XID = 17074), (SESSIONID
= 11), (DATABASE =
/home/hiveptest/35.193.82.55-hiveptest-2/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db),
(DRDAID = null)
2019-11-06T11:46:00,741 DEBUG [HikariPool-3 housekeeper] pool.HikariPool:
HikariPool-3 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-11-06T11:46:00,741 DEBUG [HikariPool-2 housekeeper] pool.HikariPool:
HikariPool-2 - Pool stats (total=10, active=1, idle=9, waiting=0)
{code}
> Lower Metastore DB Connection Pool Size in QTests
> -------------------------------------------------
>
> Key: HIVE-22469
> URL: https://issues.apache.org/jira/browse/HIVE-22469
> Project: Hive
> Issue Type: Improvement
> Components: Test, Tests
> Affects Versions: 3.2.0
> Reporter: David Mollitor
> Assignee: David Mollitor
> Priority: Minor
> Attachments: HIVE-22469.1.patch, HIVE-22469.2.patch
>
>
> Hive Metastore uses the 'HikariCP' database connection pool. The default
> number of connections to the database is 10. For the Qtests, connecting to a
> local DerbyDB, there need not be more than 2 connections. Anymore simply adds
> undo overhead and by looking at the QTest logs, I see a bunch of 'connection
> refused' from HikariCP. It may be the case that the standalone DB does not
> support that many concurrent connections anyway.
> [https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-MetaStore]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
