[ https://issues.apache.org/jira/browse/NIFI-5601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672240#comment-16672240 ]
ASF subversion and git services commented on NIFI-5601: ------------------------------------------------------- Commit d8d220ccb86d1797f56f34649d70a1acff278eb5 in nifi's branch refs/heads/master from [~ca9mbu] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d8d220c ] NIFI-5601: Add fragment.* attributes to GenerateTableFetch Signed-off-by: Peter Wicks <patric...@gmail.com> This closes #3074 > Add fragment.* attributes to GenerateTableFetch > ----------------------------------------------- > > Key: NIFI-5601 > URL: https://issues.apache.org/jira/browse/NIFI-5601 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions > Reporter: Matt Burgess > Assignee: Matt Burgess > Priority: Major > > GenerateTableFetch will generate SQL statements in multiple flow files > corresponding to "pages" of data for a single execution of the processor. > QueryDatabaseTable does something similar, although it also does the fetching > of the rows and adds fragment.* attributes to the outgoing flow files, in > case a merge or other correlation needs to be done later. > GenerateTableFetch (although it doesn't produce source data per se) is > usually used in conjunction with ExecuteSQL to fetch the actual pages of > rows, and sometimes these would need to be merged/correlated. To that end, > this Jira proposes to add the same fragment.* attributes to the outgoing GTF > flow files for use downstream. > Note that if the SQL statements are distributed among a cluster (for parallel > execution by ExecuteSQL), they will not be able to be merged, as different > nodes may get different fragments and often no single node will get all the > fragments. -- This message was sent by Atlassian JIRA (v7.6.3#76005)