[
https://issues.apache.org/jira/browse/NIFI-5601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-5601:
-------------------------------
Status: Patch Available (was: In Progress)
> Add fragment.* attributes to GenerateTableFetch
> -----------------------------------------------
>
> Key: NIFI-5601
> URL: https://issues.apache.org/jira/browse/NIFI-5601
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Priority: Major
>
> GenerateTableFetch will generate SQL statements in multiple flow files
> corresponding to "pages" of data for a single execution of the processor.
> QueryDatabaseTable does something similar, although it also does the fetching
> of the rows and adds fragment.* attributes to the outgoing flow files, in
> case a merge or other correlation needs to be done later.
> GenerateTableFetch (although it doesn't produce source data per se) is
> usually used in conjunction with ExecuteSQL to fetch the actual pages of
> rows, and sometimes these would need to be merged/correlated. To that end,
> this Jira proposes to add the same fragment.* attributes to the outgoing GTF
> flow files for use downstream.
> Note that if the SQL statements are distributed among a cluster (for parallel
> execution by ExecuteSQL), they will not be able to be merged, as different
> nodes may get different fragments and often no single node will get all the
> fragments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
