Generic JDBC Connector - Connections not properly closed?

[Claire Fautsch]

Hello,

we are using Sqoop (v 1.99.3) to transfer data from our Vertica DB to HDFS.
To connect to the DB we use a slightly modified version of the Generic JDBC
Connector.
After using it for some time, we realized that we ended up with more and
more connections held by Sqoop in our DB. Somehow they were not properly
closed after the Sqoop jobs finished.

After having a look into the connector, I found two places where it seems
that connections are not properly closed.

I am wondering if this should be filed as bug, or if I am missing
something.

Here below the changes we made:

First, in the class GenericJdbcExportDestroyer, in the method
moveDataToDestination Table. A GenericJdbcExecutor is created, but it is
missing executor.close.

Our modified version now looks like


 private void moveDataToDestinationTable(ConnectionConfiguration
connectorConf,
    boolean success, String stageTableName, String tableName) {
    GenericJdbcExecutor executor =
      new GenericJdbcExecutor(connectorConf.connection.jdbcDriver,
        connectorConf.connection.connectionString,
        connectorConf.connection.username,
        connectorConf.connection.password);

    try {
        if (success) {
            LOG.info("Job completed, transferring data from stage table to
" +
                    "destination table.");
            executor.migrateData(stageTableName, tableName);
        } else {
            LOG.warn("Job failed, clearing stage table.");
            executor.deleteTableData(stageTableName);
        }
    } finally { //added part
        executor.close();
    }


  }

Second, in the class GenericJdbcImportInitializer, in the method getSchema.
By calling the method configureJdbcProperties, an executor is created, but
once again, never closed. We changed this method, to include executor.close
in the finally block

 finally {
        if(rs != null) {
            try {
              rs.close();
            } catch (SQLException e) {
              LOG.info("Ignoring exception while closing ResultSet", e);
            }
        }
        if (executor != null) { //added part
            executor.close();
        }
    }

   Thanks in advance for any feedback!

   Best regards
   Claire Fautsch