Hey Kevin,
The DistributeLoad processor in itself does not distribute data across
to the other nodes your cluster. Can you explain your flow in a little
more detail? I am afraid i an missing something here. If you want to spread
data across your nodes, I encourage you to use a remote process group. This
will auto scale and auto load balance data to all your nodes. The setting
you mentioned is for node communication but not data transmission between
nodes. Increasing this values can help when you have a large number of
nodes, but in your case with only four nodes it should not make much
impact. Again these nodes comms are not for data. What processors are
queues building behind?
Matt
On May 27, 2016 11:41 AM, "Kevin Verhoeven" <[email protected]>
wrote:
> I’m new to NiFi and I have a small problem. I’m running NiFi
> 0.7.0-SNAPSHOT on Windows 2012 R2 VMs. I’ve noticed that the queues on some
> of my Processors are very large, sometimes as high as 10,000 flowfiles. The
> flowfiles are eventually processed but at a slow pace.
>
>
>
> I run a cluster with 4 nodes. The initial Get Processor runs on the
> Primary Node to only request a single file and I then use the
> DistributeLoad Processor with the Next Available setting to spread the load
> across the cluster nodes. However, I see that the queue is highest on the
> Primary Node and the cluster nodes see very little work.
>
>
>
> My question is: will I increase throughput to the cluster nodes if I
> increase the nifi.cluster.node.protocol.threads from 2 to something higher?
> What effect does nifi.cluster.node.protocol.threads have on the nodes?
>
>
>
> Thanks,
>
>
>
> Kevin
>
