Hi Hitesh,

Instead of limiting the row count please use one of the following 2
approaches:

1. ResultSet rsColumns = null;
    DatabaseMetaData meta = conn.getMetaData();
    rsColumns = meta.getColumns(null, null, "tablename", null);
    while (rsColumns.next()) {
      System.out.println(rsColumns.getString("TYPE_NAME"));
      System.out.println(rsColumns.getString("COLUMN_NAME"));
    }

Example given here:
http://www.java2s.com/Code/Java/Database-SQL-JDBC/GetColumnNameAndTypeForATable.htm

2. Execute the select statement putting a always false where clause like
"select * from table where 1 = 2"

I would prefer option 1 over option 2.

-Chinmay.


On Wed, Oct 12, 2016 at 8:38 PM, Hitesh Kapoor <hit...@datatorrent.com>
wrote:

> Hi All,
>
> This issue occurs when we try to insert records in a table which has a lot
> of data.
> The setup method of JdbcPOJOInsertOutputOperator generates the metadata of
> the columns in the table. To do so it fires a query of the form "Select *
> from tablename" and the extracts the required meta data like column
> name,data type and if it allowed to be NULL.
> When the table has a lot of data this "Select" query takes up a lot of time
> (more than 30 sec) and the operator gets killed.
> The fix is straight forward and simple as suggested by Sandeep is to limit
> the maximum rows returned by the select query to 1. I am using the JDBC
> function setMaxRows() to achieve this.
> Will be opening a PR for the same. This fix won't have corresponding unit
> test cases and I will test the changes externally via an app.
>
> Regards,
> Hitesh Kapoor
>

Reply via email to