Matt Burgess created NIFI-5303:
----------------------------------
Summary: QueryDatabaseTable encounters errors when processing rows
in DB2
Key: NIFI-5303
URL: https://issues.apache.org/jira/browse/NIFI-5303
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Reporter: Matt Burgess
Due to a bug in the DB2 driver, if next() is called twice on a result set when
there are no more rows, it will throw an exception to that effect. Supposedly
you can set allowNextOnExhaustedResultSet=1 on the JDBC URL or connection
properties, but in my testing that has not alleviated the problem.
ExecuteSQL retrieves the entire result set at once, so the first call to next()
when all rows have been fetched correctly returns false, and processing
continues. However in QueryDatabaseTable, due to Max Fragments and Max Records
Per Flowfile properties, it is possible that there are still rows remaining,
but the current code cannot tell the difference and will try again until no
rows have been fetched.
For example, in a table with 5 rows, with Max Rows Per FlowFile set to 4, the
first pass will correctly process 4 rows. The second pass will process 1 and
next() will return false. Then another pass is made to see if there are any
more results, so the second call to next() will produce the aforementioned
error.
Additional logic should be added to QueryDatabaseTable to determine whether all
rows have been fetched. For example, if neither Max Fragments or Max Rows Per
Flowfile have been set, then the first pass will retrieve all rows and the code
does not have to check again. If Max Rows Per Flowfile has been set and the
number of processed rows is less than that, then all rows have been processed
and the code does not have to check again. Otherwise if exactly the "right"
number of rows have been processed, then the next pass will call next() to
return false, and since zero records have been processed on that pass, the code
will continue as it originally did.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
