[
https://issues.apache.org/jira/browse/NIFI-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794416#comment-17794416
]
Bryan Bende commented on NIFI-6515:
-----------------------------------
[~larsfrancke] I think that concept could work, assuming there is a straight
forward way to calculate the splits of the parquet file. Then the flow could be
something like:
ListXyz -> GenerateParquetSplits (or something) -> FetchParquet (modified to
optionally read only a split)
This way the Fetch could be even more parallelized and produce smaller flow
files sooner.
> FetchParquet max FlowFile size
> ------------------------------
>
> Key: NIFI-6515
> URL: https://issues.apache.org/jira/browse/NIFI-6515
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Matt Gilman
> Priority: Major
>
> FetchParquet cannot transfer out multiple FlowFiles. We should introduce a
> new property to set the size of the outgoing FlowFile and then transfer as
> many FlowFiles as needed based on the fetched data.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
