+1

On Thu, Oct 13, 2016 at 7:48 PM, Hitesh Kapoor <hit...@datatorrent.com>
wrote:

> Hi All,
>
> Thank you for your suggestions.
> Following the approach as suggested by Chinmay in option 1.
> Will make the necessary code changes, test it and create a PR.
>
> --Hitesh
>
> On Thu, Oct 13, 2016 at 11:20 AM, Pradeep A. Dalvi <p...@apache.org>
> wrote:
>
> > +1 for Option 1 to use conn.getMetaData()
> >
> > --prad
> >
> > On Wed, Oct 12, 2016 at 11:39 PM, Chinmay Kolhatkar <chin...@apache.org>
> > wrote:
> >
> > > Hi Hitesh,
> > >
> > > Instead of limiting the row count please use one of the following 2
> > > approaches:
> > >
> > > 1. ResultSet rsColumns = null;
> > >     DatabaseMetaData meta = conn.getMetaData();
> > >     rsColumns = meta.getColumns(null, null, "tablename", null);
> > >     while (rsColumns.next()) {
> > >       System.out.println(rsColumns.getString("TYPE_NAME"));
> > >       System.out.println(rsColumns.getString("COLUMN_NAME"));
> > >     }
> > >
> > > Example given here:
> > > http://www.java2s.com/Code/Java/Database-SQL-JDBC/
> > > GetColumnNameAndTypeForATable.htm
> > >
> > > 2. Execute the select statement putting a always false where clause
> like
> > > "select * from table where 1 = 2"
> > >
> > > I would prefer option 1 over option 2.
> > >
> > > -Chinmay.
> > >
> > >
> > > On Wed, Oct 12, 2016 at 8:38 PM, Hitesh Kapoor <hit...@datatorrent.com
> >
> > > wrote:
> > >
> > > > Hi All,
> > > >
> > > > This issue occurs when we try to insert records in a table which has
> a
> > > lot
> > > > of data.
> > > > The setup method of JdbcPOJOInsertOutputOperator generates the
> metadata
> > > of
> > > > the columns in the table. To do so it fires a query of the form
> > "Select *
> > > > from tablename" and the extracts the required meta data like column
> > > > name,data type and if it allowed to be NULL.
> > > > When the table has a lot of data this "Select" query takes up a lot
> of
> > > time
> > > > (more than 30 sec) and the operator gets killed.
> > > > The fix is straight forward and simple as suggested by Sandeep is to
> > > limit
> > > > the maximum rows returned by the select query to 1. I am using the
> JDBC
> > > > function setMaxRows() to achieve this.
> > > > Will be opening a PR for the same. This fix won't have corresponding
> > unit
> > > > test cases and I will test the changes externally via an app.
> > > >
> > > > Regards,
> > > > Hitesh Kapoor
> > > >
> > >
> >
>

Reply via email to