[
https://issues.apache.org/jira/browse/NIFI-14864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zenkovac updated NIFI-14864:
----------------------------
Description:
switching from nifi 1.19 to 2.5 and using ConsumeKafka cant get to consume
flowfiles with more than ~500 records per flowfile despite having millions of
messages available in kafka topic each flowfile
This has a penalty performance for me because I consume thousands of flowfiles
vs a few in nifi 1.19 hich means less disc i/o usage.
this is my config:
*Processing Strategy: RECORD*
*Max Uncommitted Time* 10 sec
was:
switching from nifi 1.19 to 2.5 and using ConsumeKafka cant get to consume
flowfiles with more than ~500 records per flowfile despite having millions of
messages available in kafka topic each flowfile
This has a penalty performance for me because I consume thousands of flowfiles
vs a few in nifi 1.19 hich means less disc i/o usage.
this is my config:
*Processing Strategy: RECORD*
*Max Uncommitted Time* 10 sec
despite having millions of messages available in kafka topic each flowfile
> ConsumeKafka performance
> ------------------------
>
> Key: NIFI-14864
> URL: https://issues.apache.org/jira/browse/NIFI-14864
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 2.5.0
> Environment: nifi 2.5, kafka server 2.8
> Reporter: Zenkovac
> Priority: Major
>
> switching from nifi 1.19 to 2.5 and using ConsumeKafka cant get to consume
> flowfiles with more than ~500 records per flowfile despite having millions of
> messages available in kafka topic each flowfile
> This has a penalty performance for me because I consume thousands of
> flowfiles vs a few in nifi 1.19 hich means less disc i/o usage.
> this is my config:
> *Processing Strategy: RECORD*
> *Max Uncommitted Time* 10 sec
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
