Hi Vinay, Just to make sure that I understand everything here. You are suggesting that we should have a property that indicates the maximum number of records to add to a single FlowFile, so that if a query matches 1,000,000 rows, and we configure it with a max of 1,000 rows per FlowFile, for instance, we can push out the first 1,000 rows quickly, then the next 1,000 rows, then the next 1,000 rows.
Is this accurate? Thanks -Mark > On Dec 3, 2015, at 1:27 AM, Vinay <[email protected]> wrote: > > Thanks Aldrin for your comments and aligning your thought for having > "setFetchSize controlled by a processor property". > > > Below are my parameters > > DB Datasource : DB2 ( Universal Type 4 JDBC driver ) > > Sample NIFI Flow > Sending > Generate Flow File -> ExecuteSQL -> Data Port > > > Receiving > RemoteProcessorGroup -> ExecuteStream Processor -> PutHDFS > > Wish we could have multiple Flowfile supported for retrieving large result > set with "setFetchSize" property supported in ExecuteSQL so that above > Sample NIFI Flow is benefitted. > > Thanks! > > > > > > -- > View this message in context: > http://apache-nifi-developer-list.39713.n7.nabble.com/Streaming-large-result-set-ExecuteSQL-tp5573p5577.html > Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
