[ https://issues.apache.org/jira/browse/NIFI-5601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672241#comment-16672241 ]
ASF GitHub Bot commented on NIFI-5601: -------------------------------------- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3074 > Add fragment.* attributes to GenerateTableFetch > ----------------------------------------------- > > Key: NIFI-5601 > URL: https://issues.apache.org/jira/browse/NIFI-5601 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions > Reporter: Matt Burgess > Assignee: Matt Burgess > Priority: Major > > GenerateTableFetch will generate SQL statements in multiple flow files > corresponding to "pages" of data for a single execution of the processor. > QueryDatabaseTable does something similar, although it also does the fetching > of the rows and adds fragment.* attributes to the outgoing flow files, in > case a merge or other correlation needs to be done later. > GenerateTableFetch (although it doesn't produce source data per se) is > usually used in conjunction with ExecuteSQL to fetch the actual pages of > rows, and sometimes these would need to be merged/correlated. To that end, > this Jira proposes to add the same fragment.* attributes to the outgoing GTF > flow files for use downstream. > Note that if the SQL statements are distributed among a cluster (for parallel > execution by ExecuteSQL), they will not be able to be merged, as different > nodes may get different fragments and often no single node will get all the > fragments. -- This message was sent by Atlassian JIRA (v7.6.3#76005)