Thanks Jacopo!

I have spent some time tweaking my local settings to get the load-demo target to execute as quickly as possible. I will share with you what I have learned.

I have two computers I build on: a dual Xeon server (12 CPUs) and an Intel i7 laptop (4 CPUs).

The entity data loader performance depends on the database response time, not on the number of CPUs. Even with an extremely fast RAID array or SSD, anything more than 4 threads isn't needed. This rule is true on both computers.

Where load-demo execution time = n:

1 thread = n
2 threads = n - x
3 threads = n - 2x
4 threads = n - 3x
5 threads = n - 3x
6 threads = n - 2x
7 threads = n - x
8 threads = n
9 threads = n + x
...

With your previous change, 12 CPUs = 12 threads = n * 2. The data load took twice as long as when I was able to configure it for 4 threads.

Adrian Crum
Sandglass Software
www.sandglass-software.com

On 9/25/2014 9:42 AM, Jacopo Cappellato wrote:
Sorry for the delay of my response.
I am going to commit a change this morning and then I will follow up to further 
discuss when I will have a free slot of time.

Jacopo

On Sep 25, 2014, at 9:19 AM, Adrian Crum <[email protected]> 
wrote:

Actually, I would like to restore this feature. We have a 12 CPU server that is 
being bogged down by this change. Always using one thread per CPU is an overly 
simplistic approach and it can make performance suffer.


Adrian Crum
Sandglass Software
www.sandglass-software.com

On 9/24/2014 9:44 AM, Jacopo Cappellato wrote:
Adrian,

thank you for the review! I am going to followup with you an this shortly as 
soon as I'll have a free moment.

Jacopo

On Sep 23, 2014, at 8:05 PM, Adrian Crum <[email protected]> 
wrote:

The configurable thread count (maxWorkerPoolSize) is a useful feature. It is a 
shame to see it removed.

Adrian Crum
Sandglass Software
www.sandglass-software.com

On 9/23/2014 6:04 PM, [email protected] wrote:
Author: jacopoc
Date: Tue Sep 23 17:04:20 2014
New Revision: 1627092

URL: http://svn.apache.org/r1627092
Log:
Moved the thread pool executor object from GenericDAO to the DatabaseUtil class 
where it was actually used: this change simplifies the API and allows to 
shutdown the pool when it is no more needed.
Removed a series of methods that are no more needed after this refactoring or 
were not used.

Modified:
     ofbiz/trunk/framework/base/src/org/ofbiz/base/concurrent/ExecutionPool.java
     
ofbiz/trunk/framework/catalina/src/org/ofbiz/catalina/container/CatalinaContainer.java
     ofbiz/trunk/framework/entity/dtd/entity-config.xsd
     ofbiz/trunk/framework/entity/src/org/ofbiz/entity/DelegatorFactory.java
     
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/config/model/Datasource.java
     
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericDAO.java
     
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericHelper.java
     
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericHelperDAO.java
     ofbiz/trunk/framework/entity/src/org/ofbiz/entity/jdbc/DatabaseUtil.java

Modified: 
ofbiz/trunk/framework/base/src/org/ofbiz/base/concurrent/ExecutionPool.java
URL: 
http://svn.apache.org/viewvc/ofbiz/trunk/framework/base/src/org/ofbiz/base/concurrent/ExecutionPool.java?rev=1627092&r1=1627091&r2=1627092&view=diff
==============================================================================
--- ofbiz/trunk/framework/base/src/org/ofbiz/base/concurrent/ExecutionPool.java 
(original)
+++ ofbiz/trunk/framework/base/src/org/ofbiz/base/concurrent/ExecutionPool.java 
Tue Sep 23 17:04:20 2014
@@ -44,7 +44,7 @@ public final class ExecutionPool {
      public static final String module = ExecutionPool.class.getName();
      public static final ExecutorService GLOBAL_BATCH = new ThreadPoolExecutor(0, 
Integer.MAX_VALUE, 5, TimeUnit.SECONDS, new SynchronousQueue<Runnable>(), new 
ExecutionPoolThreadFactory(null, "OFBiz-batch"));
      public static final ForkJoinPool GLOBAL_FORK_JOIN = new ForkJoinPool();
-    private static final ExecutorService pulseExecutionPool = 
Executors.newFixedThreadPool(autoAdjustThreadCount(-1), new 
ExecutionPoolThreadFactory(null, "OFBiz-ExecutionPoolPulseWorker"));
+    private static final ExecutorService pulseExecutionPool = 
Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors(), new 
ExecutionPoolThreadFactory(null, "OFBiz-ExecutionPoolPulseWorker"));

      protected static class ExecutionPoolThreadFactory implements 
ThreadFactory {
          private final ThreadGroup group;
@@ -65,22 +65,8 @@ public final class ExecutionPool {
          }
      }

-    private static int autoAdjustThreadCount(int threadCount) {
-        if (threadCount == 0) {
-            return 1;
-        } else if (threadCount < 0) {
-            int numCpus = Runtime.getRuntime().availableProcessors();
-            return Math.abs(threadCount) * numCpus;
-        } else {
-            return threadCount;
-        }
-    }
-
-    public static ScheduledExecutorService getScheduledExecutor(ThreadGroup 
group, String namePrefix, int threadCount, boolean preStart) {
-        return getScheduledExecutor(group, namePrefix, threadCount, 0, 
preStart);
-    }
      public static ScheduledExecutorService getScheduledExecutor(ThreadGroup 
group, String namePrefix, int threadCount, long keepAliveSeconds, boolean 
preStart) {
-        ScheduledThreadPoolExecutor executor = new 
ScheduledThreadPoolExecutor(autoAdjustThreadCount(threadCount), new 
ExecutionPoolThreadFactory(group, namePrefix));
+        ScheduledThreadPoolExecutor executor = new 
ScheduledThreadPoolExecutor(threadCount, new ExecutionPoolThreadFactory(group, 
namePrefix));
          if (keepAliveSeconds > 0) {
              executor.setKeepAliveTime(keepAliveSeconds, TimeUnit.SECONDS);
              executor.allowCoreThreadTimeOut(true);
@@ -114,7 +100,7 @@ public final class ExecutionPool {
      }

      static {
-        int numberOfExecutionPoolPulseWorkers = autoAdjustThreadCount(-1);
+        int numberOfExecutionPoolPulseWorkers = 
Runtime.getRuntime().availableProcessors();
          for (int i = 0; i < numberOfExecutionPoolPulseWorkers; i++) {
              pulseExecutionPool.execute(new ExecutionPoolPulseWorker());
          }

Modified: 
ofbiz/trunk/framework/catalina/src/org/ofbiz/catalina/container/CatalinaContainer.java
URL: 
http://svn.apache.org/viewvc/ofbiz/trunk/framework/catalina/src/org/ofbiz/catalina/container/CatalinaContainer.java?rev=1627092&r1=1627091&r2=1627092&view=diff
==============================================================================
--- 
ofbiz/trunk/framework/catalina/src/org/ofbiz/catalina/container/CatalinaContainer.java
 (original)
+++ 
ofbiz/trunk/framework/catalina/src/org/ofbiz/catalina/container/CatalinaContainer.java
 Tue Sep 23 17:04:20 2014
@@ -616,7 +616,7 @@ public class CatalinaContainer implement
              return;
          }

-        ScheduledExecutorService executor = 
ExecutionPool.getScheduledExecutor(CATALINA_THREAD_GROUP, "catalina-startup", 
-1, true);
+        ScheduledExecutorService executor = 
ExecutionPool.getScheduledExecutor(CATALINA_THREAD_GROUP, "catalina-startup", 
Runtime.getRuntime().availableProcessors(), 0, true);
          try {
              List<Future<Context>> futures = new ArrayList<Future<Context>>();


Modified: ofbiz/trunk/framework/entity/dtd/entity-config.xsd
URL: 
http://svn.apache.org/viewvc/ofbiz/trunk/framework/entity/dtd/entity-config.xsd?rev=1627092&r1=1627091&r2=1627092&view=diff
==============================================================================
--- ofbiz/trunk/framework/entity/dtd/entity-config.xsd (original)
+++ ofbiz/trunk/framework/entity/dtd/entity-config.xsd Tue Sep 23 17:04:20 2014
@@ -411,7 +411,6 @@ under the License.
          <xs:attribute type="xs:string" name="table-type"/>
          <xs:attribute type="xs:string" name="character-set"/>
          <xs:attribute type="xs:string" name="collate"/>
-        <xs:attribute type="xs:integer" name="max-worker-pool-size" 
default="0"/>
      </xs:attributeGroup>
      <xs:element name="sql-load-path">
          <xs:complexType>

Modified: 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/DelegatorFactory.java
URL: 
http://svn.apache.org/viewvc/ofbiz/trunk/framework/entity/src/org/ofbiz/entity/DelegatorFactory.java?rev=1627092&r1=1627091&r2=1627092&view=diff
==============================================================================
--- ofbiz/trunk/framework/entity/src/org/ofbiz/entity/DelegatorFactory.java 
(original)
+++ ofbiz/trunk/framework/entity/src/org/ofbiz/entity/DelegatorFactory.java Tue 
Sep 23 17:04:20 2014
@@ -36,7 +36,7 @@ public abstract class DelegatorFactory i
      public static final String module = DelegatorFactoryImpl.class.getName();
      private static final ConcurrentHashMap<String, Future<Delegator>> delegators = new 
ConcurrentHashMap<String, Future<Delegator>>();
      private static final ThreadGroup DELEGATOR_THREAD_GROUP = new 
ThreadGroup("DelegatorFactory");
-    private static final ScheduledExecutorService executor = 
ExecutionPool.getScheduledExecutor(DELEGATOR_THREAD_GROUP, "delegator-startup", 
-1, 10, true);
+    private static final ScheduledExecutorService executor = 
ExecutionPool.getScheduledExecutor(DELEGATOR_THREAD_GROUP, "delegator-startup", 
Runtime.getRuntime().availableProcessors(), 10, true);

      public static Delegator getDelegator(String delegatorName) {
          Future<Delegator> future = getDelegatorFuture(delegatorName);

Modified: 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/config/model/Datasource.java
URL: 
http://svn.apache.org/viewvc/ofbiz/trunk/framework/entity/src/org/ofbiz/entity/config/model/Datasource.java?rev=1627092&r1=1627091&r2=1627092&view=diff
==============================================================================
--- 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/config/model/Datasource.java 
(original)
+++ 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/config/model/Datasource.java 
Tue Sep 23 17:04:20 2014
@@ -74,7 +74,6 @@ public final class Datasource {
      private final String tableType; // type = xs:string
      private final String characterSet; // type = xs:string
      private final String collate; // type = xs:string
-    private final int maxWorkerPoolSize; // type = xs:integer
      private final List<SqlLoadPath> sqlLoadPathList; // <sql-load-path>
      private final List<ReadData> readDataList; // <read-data>
      private final InlineJdbc inlineJdbc; // <inline-jdbc>
@@ -161,22 +160,6 @@ public final class Datasource {
          this.tableType = element.getAttribute("table-type").intern();
          this.characterSet = element.getAttribute("character-set").intern();
          this.collate = element.getAttribute("collate").intern();
-        String maxWorkerPoolSize = 
element.getAttribute("max-worker-pool-size").intern();
-        if (maxWorkerPoolSize.isEmpty()) {
-            this.maxWorkerPoolSize = 1;
-        } else {
-            try {
-                int maxWorkerPoolSizeInt = Integer.parseInt(maxWorkerPoolSize);
-                if (maxWorkerPoolSizeInt == 0) {
-                    maxWorkerPoolSizeInt = 1;
-                } else if (maxWorkerPoolSizeInt < -2) {
-                    maxWorkerPoolSizeInt = -2;
-                }
-                this.maxWorkerPoolSize = maxWorkerPoolSizeInt;
-            } catch (Exception e) {
-                throw new GenericEntityConfException("<datasource> element 
max-worker-pool-size attribute is invalid" + lineNumberText);
-            }
-        }
          List<? extends Element> sqlLoadPathElementList = 
UtilXml.childElementList(element, "sql-load-path");
          if (sqlLoadPathElementList.isEmpty()) {
              this.sqlLoadPathList = Collections.emptyList();
@@ -384,11 +367,6 @@ public final class Datasource {
          return this.collate;
      }

-    /** Returns the value of the <code>max-worker-pool-size</code> attribute. 
*/
-    public int getMaxWorkerPoolSize() {
-        return this.maxWorkerPoolSize;
-    }
-
      /** Returns the <code>&lt;sql-load-path&gt;</code> child elements. */
      public List<SqlLoadPath> getSqlLoadPathList() {
          return this.sqlLoadPathList;

Modified: 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericDAO.java
URL: 
http://svn.apache.org/viewvc/ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericDAO.java?rev=1627092&r1=1627091&r2=1627092&view=diff
==============================================================================
--- 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericDAO.java 
(original)
+++ 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericDAO.java 
Tue Sep 23 17:04:20 2014
@@ -31,12 +31,8 @@ import java.util.List;
  import java.util.Map;
  import java.util.Set;
  import java.util.TreeSet;
-import java.util.concurrent.Callable;
  import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.ExecutorService;
-import java.util.concurrent.Future;

-import org.ofbiz.base.concurrent.ExecutionPool;
  import org.ofbiz.base.util.Debug;
  import org.ofbiz.base.util.UtilValidate;
  import org.ofbiz.entity.Delegator;
@@ -78,7 +74,6 @@ public class GenericDAO {
      private final GenericHelperInfo helperInfo;
      private final ModelFieldTypeReader modelFieldTypeReader;
      private final Datasource datasource;
-    private final ExecutorService executor;

      public static GenericDAO getGenericDAO(GenericHelperInfo helperInfo) {
          String cacheKey = helperInfo.getHelperFullName();
@@ -94,11 +89,6 @@ public class GenericDAO {
          this.helperInfo = helperInfo;
          this.modelFieldTypeReader = 
ModelFieldTypeReader.getModelFieldTypeReader(helperInfo.getHelperBaseName());
          this.datasource = 
EntityConfig.getDatasource(helperInfo.getHelperBaseName());
-        this.executor = ExecutionPool.getScheduledExecutor(GENERIC_DAO_THREAD_GROUP, 
"OFBiz-entity-datasource(" + helperInfo.getHelperFullName() + ")", 
datasource.getMaxWorkerPoolSize(), false);
-    }
-
-    public <T> Future<T> submitWork(Callable<T> callable) throws 
GenericEntityException {
-        return this.executor.submit(callable);
      }

      private void addFieldIfMissing(List<ModelField> fieldsToSave, String 
fieldName, ModelEntity modelEntity) {
@@ -1228,13 +1218,13 @@ public class GenericDAO {
      /* ====================================================================== 
*/

      public void checkDb(Map<String, ModelEntity> modelEntities, List<String> 
messages, boolean addMissing) {
-        DatabaseUtil dbUtil = new DatabaseUtil(this.helperInfo, this.executor);
+        DatabaseUtil dbUtil = new DatabaseUtil(this.helperInfo);
          dbUtil.checkDb(modelEntities, messages, addMissing);
      }

      /** Creates a list of ModelEntity objects based on meta data from the 
database */
      public List<ModelEntity> induceModelFromDb(Collection<String> messages) {
-        DatabaseUtil dbUtil = new DatabaseUtil(this.helperInfo, this.executor);
+        DatabaseUtil dbUtil = new DatabaseUtil(this.helperInfo);
          return dbUtil.induceModelFromDb(messages);
      }
  }

Modified: 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericHelper.java
URL: 
http://svn.apache.org/viewvc/ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericHelper.java?rev=1627092&r1=1627091&r2=1627092&view=diff
==============================================================================
--- 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericHelper.java 
(original)
+++ 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericHelper.java 
Tue Sep 23 17:04:20 2014
@@ -49,8 +49,6 @@ public interface GenericHelper {
       */
      public String getHelperName();

-    public <T> Future<T> submitWork(Callable<T> callable) throws 
GenericEntityException;
-
      /** Creates a Entity in the form of a GenericValue and write it to the 
database
       *@return GenericValue instance containing the new instance
       */

Modified: 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericHelperDAO.java
URL: 
http://svn.apache.org/viewvc/ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericHelperDAO.java?rev=1627092&r1=1627091&r2=1627092&view=diff
==============================================================================
--- 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericHelperDAO.java
 (original)
+++ 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/datasource/GenericHelperDAO.java
 Tue Sep 23 17:04:20 2014
@@ -57,10 +57,6 @@ public class GenericHelperDAO implements
          return this.helperInfo.getHelperFullName();
      }

-    public <T> Future<T> submitWork(Callable<T> callable) throws 
GenericEntityException {
-        return genericDAO.submitWork(callable);
-    }
-
      /** Creates a Entity in the form of a GenericValue and write it to the 
database
       *@return GenericValue instance containing the new instance
       */

Modified: 
ofbiz/trunk/framework/entity/src/org/ofbiz/entity/jdbc/DatabaseUtil.java
URL: 
http://svn.apache.org/viewvc/ofbiz/trunk/framework/entity/src/org/ofbiz/entity/jdbc/DatabaseUtil.java?rev=1627092&r1=1627091&r2=1627092&view=diff
==============================================================================
--- ofbiz/trunk/framework/entity/src/org/ofbiz/entity/jdbc/DatabaseUtil.java 
(original)
+++ ofbiz/trunk/framework/entity/src/org/ofbiz/entity/jdbc/DatabaseUtil.java 
Tue Sep 23 17:04:20 2014
@@ -40,6 +40,7 @@ import java.util.Set;
  import java.util.TreeSet;
  import java.util.concurrent.Callable;
  import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
  import java.util.concurrent.Future;
  import java.util.concurrent.FutureTask;

@@ -47,7 +48,6 @@ import org.ofbiz.base.concurrent.Executi
  import org.ofbiz.base.util.Debug;
  import org.ofbiz.base.util.UtilTimer;
  import org.ofbiz.base.util.UtilValidate;
-import org.ofbiz.base.util.UtilXml;
  import org.ofbiz.entity.GenericEntityException;
  import org.ofbiz.entity.config.model.Datasource;
  import org.ofbiz.entity.config.model.EntityConfig;
@@ -62,8 +62,6 @@ import org.ofbiz.entity.model.ModelRelat
  import org.ofbiz.entity.model.ModelViewEntity;
  import org.ofbiz.entity.transaction.TransactionFactoryLoader;
  import org.ofbiz.entity.transaction.TransactionUtil;
-import org.w3c.dom.Document;
-import org.w3c.dom.Element;

  /**
   * Utilities for Entity Database Maintenance
@@ -85,18 +83,12 @@ public class DatabaseUtil {
      protected String password = null;

      boolean isLegacy = false;
-    protected ExecutorService executor;

      // OFBiz DatabaseUtil
      public DatabaseUtil(GenericHelperInfo helperInfo) {
-        this(helperInfo, null);
-    }
-
-    public DatabaseUtil(GenericHelperInfo helperInfo, ExecutorService 
executor) {
          this.helperInfo = helperInfo;
          this.modelFieldTypeReader = 
ModelFieldTypeReader.getModelFieldTypeReader(helperInfo.getHelperBaseName());
          this.datasourceInfo = 
EntityConfig.getDatasource(helperInfo.getHelperBaseName());
-        this.executor = executor;
      }

      // Legacy DatabaseUtil
@@ -108,31 +100,6 @@ public class DatabaseUtil {
          this.isLegacy = true;
      }

-    protected <T> Future<T> submitWork(Callable<T> callable) {
-        if (this.executor == null) {
-            FutureTask<T> task = new FutureTask<T>(callable);
-            task.run();
-            return task;
-        }
-        return this.executor.submit(callable);
-    }
-
-    protected <T> List<Future<T>> submitAll(Collection<? extends Callable<T>> 
tasks) {
-        List<Future<T>> futures = new ArrayList<Future<T>>(tasks.size());
-        if (this.executor == null) {
-            for (Callable<T> callable: tasks) {
-                FutureTask<T> task = new FutureTask<T>(callable);
-                task.run();
-                futures.add(task);
-            }
-            return futures;
-        }
-        for (Callable<T> callable: tasks) {
-            futures.add(this.executor.submit(callable));
-        }
-        return futures;
-    }
-
      protected Connection getConnection() throws SQLException, 
GenericEntityException {
          Connection connection = null;
          if (!isLegacy) {
@@ -216,6 +183,9 @@ public class DatabaseUtil {
          if (isLegacy) {
              throw new RuntimeException("Cannot run checkDb on a legacy database 
connection; configure a database helper (entityengine.xml)");
          }
+
+        ExecutorService executor = 
Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
+
          UtilTimer timer = new UtilTimer();
          timer.timerString("Start - Before Get Database Meta Data");

@@ -233,7 +203,7 @@ public class DatabaseUtil {
          timer.timerString("After Get All Table Names");

          // get ALL column info, put into hashmap by table name
-        Map<String, Map<String, ColumnCheckInfo>> colInfo = 
this.getColumnInfo(tableNames, checkPks, messages);
+        Map<String, Map<String, ColumnCheckInfo>> colInfo = 
getColumnInfo(tableNames, checkPks, messages, executor);
          if (colInfo == null) {
              String message = "Could not get column information from the database, 
aborting.";
              if (messages != null) messages.add(message);
@@ -455,7 +425,7 @@ public class DatabaseUtil {

                  if (addMissing) {
                      // create the table
-                    tableFutures.add(submitWork(new 
CreateTableCallable(entity, modelEntities, tableName)));
+                    tableFutures.add(executor.submit(new 
CreateTableCallable(entity, modelEntities, tableName)));
                  }
              }
          }
@@ -478,7 +448,7 @@ public class DatabaseUtil {
              List<Future<AbstractCountingCallable>> fkIndicesFutures = new 
LinkedList<Future<AbstractCountingCallable>>();
              for (ModelEntity curEntity: entitiesAdded) {
                  if (curEntity.getRelationsOneSize() > 0) {
-                    fkIndicesFutures.add(submitWork(new 
AbstractCountingCallable(curEntity, modelEntities) {
+                    fkIndicesFutures.add(executor.submit(new 
AbstractCountingCallable(curEntity, modelEntities) {
                          public AbstractCountingCallable call() throws 
Exception {
                              count = createForeignKeyIndices(entity, 
datasourceInfo.getConstraintNameClipLength(), messages);
                              return this;
@@ -507,7 +477,7 @@ public class DatabaseUtil {
              List<Future<AbstractCountingCallable>> disFutures = new 
LinkedList<Future<AbstractCountingCallable>>();
              for (ModelEntity curEntity: entitiesAdded) {
                  if (curEntity.getIndexesSize() > 0) {
-                    disFutures.add(submitWork(new 
AbstractCountingCallable(curEntity,  modelEntities) {
+                    disFutures.add(executor.submit(new 
AbstractCountingCallable(curEntity,  modelEntities) {
                      public AbstractCountingCallable call() throws Exception {
                          count = createDeclaredIndices(entity, messages);
                          return this;
@@ -755,17 +725,19 @@ public class DatabaseUtil {

          }

-
+        executor.shutdown();
          timer.timerString("Finished Checking Entity Database");
      }

      /** Creates a list of ModelEntity objects based on meta data from the 
database */
      public List<ModelEntity> induceModelFromDb(Collection<String> messages) {
+        ExecutorService executor = 
Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
+
          // get ALL tables from this database
          TreeSet<String> tableNames = this.getTableNames(messages);

          // get ALL column info, put into hashmap by table name
-        Map<String, Map<String, ColumnCheckInfo>> colInfo = 
this.getColumnInfo(tableNames, true, messages);
+        Map<String, Map<String, ColumnCheckInfo>> colInfo = 
getColumnInfo(tableNames, true, messages, executor);

          // go through each table and make a ModelEntity object, add to list
          // for each entity make corresponding ModelField objects
@@ -789,66 +761,10 @@ public class DatabaseUtil {
              newEntList.add(newEntity);
          }

+        executor.shutdown();
          return newEntList;
      }

-    public Document induceModelFromDb(String packageName) {
-        Document document = UtilXml.makeEmptyXmlDocument("entitymodel");
-        Element root = document.getDocumentElement();
-        root.appendChild(document.createElement("title"));
-        root.appendChild(document.createElement("description"));
-        root.appendChild(document.createElement("copyright"));
-        root.appendChild(document.createElement("author"));
-        root.appendChild(document.createElement("version"));
-
-        // messages list
-        List<String> messages = new ArrayList<String>();
-
-        // get ALL tables from this database
-        TreeSet<String> tableNames = this.getTableNames(messages);
-
-        // get ALL column info, put into hashmap by table name
-        Map<String, Map<String, ColumnCheckInfo>> colInfo = 
this.getColumnInfo(tableNames, true, messages);
-
-        boolean isCaseSensitive = false;
-        DatabaseMetaData dbData = this.getDatabaseMetaData(null, messages);
-        if (dbData != null) {
-            try {
-                isCaseSensitive = dbData.supportsMixedCaseIdentifiers();
-            } catch (SQLException e) {
-                Debug.logError(e, "Error getting db meta data about case 
sensitive", module);
-            }
-        }
-
-        if (UtilValidate.isNotEmpty(packageName)) {
-            String catalogName = null;
-            try {
-                catalogName = this.getConnection().getCatalog();
-            } catch (Exception e) {
-                // ignore
-            }
-            packageName = "org.ofbiz.ext." + (catalogName != null ? catalogName : 
"unknown");
-        }
-
-
-        // iterate over the table names is alphabetical order
-        for (String tableName: new TreeSet<String>(colInfo.keySet())) {
-            Map<String, ColumnCheckInfo> colMap = colInfo.get(tableName);
-            ModelEntity newEntity = new ModelEntity(tableName, colMap, 
modelFieldTypeReader, isCaseSensitive);
-            root.appendChild(newEntity.toXmlElement(document, "org.ofbiz.ext." 
+ packageName));
-        }
-
-        // print the messages to the console
-        for (String message: messages) {
-            Debug.logInfo(message, module);
-        }
-        return document;
-    }
-
-    public Document induceModelFromDb() {
-        return this.induceModelFromDb("");
-    }
-
      public DatabaseMetaData getDatabaseMetaData(Connection connection, 
Collection<String> messages) {
          if (connection == null) {
              connection = getConnectionLogged(messages);
@@ -1129,7 +1045,7 @@ public class DatabaseUtil {
          };
      }

-    public Map<String, Map<String, ColumnCheckInfo>> getColumnInfo(Set<String> 
tableNames, boolean getPks, Collection<String> messages) {
+    private Map<String, Map<String, ColumnCheckInfo>> getColumnInfo(Set<String> 
tableNames, boolean getPks, Collection<String> messages, ExecutorService executor) {
          // if there are no tableNames, don't even try to get the columns
          if (tableNames.size() == 0) {
              return new HashMap<String, Map<String, ColumnCheckInfo>>();
@@ -1261,7 +1177,7 @@ public class DatabaseUtil {
                          List<Future<AbstractCountingCallable>> pkFetcherFutures = new 
LinkedList<Future<AbstractCountingCallable>>();
                          for (String curTable: tableNames) {
                              curTable = 
curTable.substring(curTable.indexOf('.') + 1); //cut off schema name
-                            
pkFetcherFutures.add(submitWork(createPrimaryKeyFetcher(dbData, 
lookupSchemaName, needsUpperCase, colInfo, messages, curTable)));
+                            
pkFetcherFutures.add(executor.submit(createPrimaryKeyFetcher(dbData, 
lookupSchemaName, needsUpperCase, colInfo, messages, curTable)));
                          }
                          for (AbstractCountingCallable pkFetcherCallable: 
ExecutionPool.getAllFutures(pkFetcherFutures)) {
                              pkCount += pkFetcherCallable.updateData(messages);




Reply via email to