[hive] branch master updated: HIVE-26419: Use a different pool for DataNucleus' secondary connection factory (Yu-Wen Lai, reviewed by Denys Kuzmenko)

[dkuzmenko]

This is an automated email from the ASF dual-hosted git repository.

dkuzmenko pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git


The following commit(s) were added to refs/heads/master by this push:
     new 1c3406ea598 HIVE-26419: Use a different pool for DataNucleus' 
secondary connection factory (Yu-Wen Lai, reviewed by Denys Kuzmenko)
1c3406ea598 is described below

commit 1c3406ea598e0c2d866b20747602c1a01fa5a425
Author: Yu-Wen <hsnuso...@users.noreply.github.com>
AuthorDate: Mon Jul 25 03:16:13 2022 -0700

    HIVE-26419: Use a different pool for DataNucleus' secondary connection 
factory (Yu-Wen Lai, reviewed by Denys Kuzmenko)
    
    Closes #3466
---
 .../apache/hadoop/hive/metastore/PersistenceManagerProvider.java    | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git 
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/PersistenceManagerProvider.java
 
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/PersistenceManagerProvider.java
index 1c2edbbbad8..cccb1f72e0c 100644
--- 
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/PersistenceManagerProvider.java
+++ 
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/PersistenceManagerProvider.java
@@ -253,8 +253,12 @@ public class PersistenceManagerProvider {
     } else {
       try {
         DataSource ds = (maxPoolSize > 0) ? dsp.create(conf, maxPoolSize) : 
dsp.create(conf);
+        // The secondary connection factory is used for schema generation, and 
for value generation operations.
+        // We should use a different pool for the secondary connection factory 
to avoid resource starvation.
+        // Since DataNucleus uses locks for schema generation and value 
generation, 2 connections should be sufficient.
+        DataSource ds2 = forCompactor ? ds : dsp.create(conf, /* maxPoolSize 
*/ 2);
         dsProp.put(PropertyNames.PROPERTY_CONNECTION_FACTORY, ds);
-        dsProp.put(PropertyNames.PROPERTY_CONNECTION_FACTORY2, ds);
+        dsProp.put(PropertyNames.PROPERTY_CONNECTION_FACTORY2, ds2);
         dsProp.put(ConfVars.MANAGER_FACTORY_CLASS.getVarname(),
             "org.datanucleus.api.jdo.JDOPersistenceManagerFactory");
         pmf = JDOHelper.getPersistenceManagerFactory(dsProp);