Can you share screenshots, logs, and a more detailed description of what you're doing, observing with nifi and the system and what you expect it to be doing.
Thanks On Thu, Apr 4, 2019 at 10:56 AM l vic <lvic4...@gmail.com> wrote: > No, actually what happens is - NiFi stops responding ( if I use it without > rate contol).... > > > On Thu, Apr 4, 2019 at 10:42 AM Joe Witt <joe.w...@gmail.com> wrote: > >> Hello >> >> There isn't really a feedback mechanism based on load on the Kafka >> topic. When you say overrunning the topic do you mean that you don't want >> there to be a large lag between consumers and their current offset and if >> that grows you want NiFi to slow down? I dont believe there is anything >> inherent to the kafka producer protocol that would inform of us this. We >> could periodically poll for this information and optional back-off. >> >> Is this what you have in mind? >> >> Thanks >> >> On Thu, Apr 4, 2019 at 10:34 AM l vic <lvic4...@gmail.com> wrote: >> >>> I have to ingest large (200,000 messages) data set into Kafka topic as >>> quickly as possible without overrunning topic... Right now I just use rate >>> limiter to do it but can be there some better "adaptive" way to do it? >>> Thank you... >>> -V >>> >>
