[
https://issues.apache.org/jira/browse/NIFI-5601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672240#comment-16672240
]
ASF subversion and git services commented on NIFI-5601:
-------------------------------------------------------
Commit d8d220ccb86d1797f56f34649d70a1acff278eb5 in nifi's branch
refs/heads/master from [~ca9mbu]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d8d220c ]
NIFI-5601: Add fragment.* attributes to GenerateTableFetch
Signed-off-by: Peter Wicks <[email protected]>
This closes #3074
> Add fragment.* attributes to GenerateTableFetch
> -----------------------------------------------
>
> Key: NIFI-5601
> URL: https://issues.apache.org/jira/browse/NIFI-5601
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Priority: Major
>
> GenerateTableFetch will generate SQL statements in multiple flow files
> corresponding to "pages" of data for a single execution of the processor.
> QueryDatabaseTable does something similar, although it also does the fetching
> of the rows and adds fragment.* attributes to the outgoing flow files, in
> case a merge or other correlation needs to be done later.
> GenerateTableFetch (although it doesn't produce source data per se) is
> usually used in conjunction with ExecuteSQL to fetch the actual pages of
> rows, and sometimes these would need to be merged/correlated. To that end,
> this Jira proposes to add the same fragment.* attributes to the outgoing GTF
> flow files for use downstream.
> Note that if the SQL statements are distributed among a cluster (for parallel
> execution by ExecuteSQL), they will not be able to be merged, as different
> nodes may get different fragments and often no single node will get all the
> fragments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
