Matt Burgess reassigned NIFI-5303:

    Assignee: Matt Burgess

> QueryDatabaseTable encounters errors when processing rows in DB2
> ----------------------------------------------------------------
>                 Key: NIFI-5303
>                 URL: https://issues.apache.org/jira/browse/NIFI-5303
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Matt Burgess
>            Assignee: Matt Burgess
>            Priority: Major
> Due to a bug in the DB2 driver, if next() is called twice on a result set 
> when there are no more rows, it will throw an exception to that effect. 
> Supposedly you can set allowNextOnExhaustedResultSet=1 on the JDBC URL or 
> connection properties, but in my testing that has not alleviated the problem.
> ExecuteSQL retrieves the entire result set at once, so the first call to 
> next() when all rows have been fetched correctly returns false, and 
> processing continues. However in QueryDatabaseTable, due to Max Fragments and 
> Max Records Per Flowfile properties, it is possible that there are still rows 
> remaining, but the current code cannot tell the difference and will try again 
> until no rows have been fetched. 
> For example, in a table with 5 rows, with Max Rows Per FlowFile set to 4, the 
> first pass will correctly process 4 rows. The second pass will process 1 and 
> next() will return false. Then another pass is made to see if there are any 
> more results, so the second call to next() will produce the aforementioned 
> error.
> Additional logic should be added to QueryDatabaseTable to determine whether 
> all rows have been fetched. For example, if neither Max Fragments or Max Rows 
> Per Flowfile have been set, then the first pass will retrieve all rows and 
> the code does not have to check again. If Max Rows Per Flowfile has been set 
> and the number of processed rows is less than that, then all rows have been 
> processed and the code does not have to check again. Otherwise if exactly the 
> "right" number of rows have been processed, then the next pass will call 
> next() to return false, and since zero records have been processed on that 
> pass, the code will continue as it originally did.

This message was sent by Atlassian JIRA

Reply via email to