On 2018/09/25 15:34:03, Mark Payne <[email protected]> wrote:
> Ashwin,
>
> You'll want to Right-Click on the RPG and then choose to configure Remote
> Ports. From there, you can
> configure how many threads should be used to pull data from each port. So
> you've updated the Output Port
> to use up to 3 threads per node to provide the data, but each node is still
> only using 1 thread to pull the data.
> By configuring the Remote Ports, you can configure the RPG to use 3 threads
> to pull the data as well.
>
> Thanks
> -Mark
>
>
> > On Sep 25, 2018, at 11:29 AM, ashwin konale <[email protected]> wrote:
> >
> > Hi I have the following flow which I am trying to increase the throughput.
> > I am runnign 3 node nifi cluster(v1.6)
> >
> > CDC -> processGroupOutputPort -> externalOutputPort
> > RPG -> downstream
> >
> > Flowfiles are always queued up between processGroupOutputPort to
> > externalOutputPort. I dont see any configuration to increase the number of
> > threads for this operation as well. I have set backpressure to be 10,000
> > flowfiles. Since the CDC produces at much faster rate than this, I want to
> > increase the throughput of the flow. Also I see in the UI
> > externalOutputPort has number of threads running as 3 all the time no
> > matter how much I increase number of threads to consume in RPG.
> >
> > Any pointers here will be much helpful.
> >
> > Ashwin
>
> Hello Mark, Thanks a lot for the reply.
I have already configured number of threads to in remote ports in RPG. But no
matter how many threads I set it to, the number of threads running in output
port is showing as 3. And there is no setting to update the number of threads
for the output port. I quickly went through LocalPort.java .
(https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/connectable/LocalPort.java)
Looks like it transfers in the batches of 100. Is there any way to increase
the throughput.?
