Any nifi developers on this list that have any suggestions? On Wed, Aug 8, 2018 at 7:38 AM Joe Gresock <[email protected]> wrote:
> I am running a 7-node NiFi 1.6.0 cluster that performs fairly well when > it's simply processing its own data (putting records in Elasticsearch, > MongoDB, running transforms, etc.). However, when we add receiving > Site-to-Site traffic to the mix, the CPU spikes to the point that the nodes > can't talk to each other, resulting in the inability to view or modify the > flow in the console. > > I have tried some basic things to mitigate this: > - Requested that the sending party use a comma-separated list of all 7 of > our nodes in their Remote Process Group that points to our cluster, in > hopes that that will help balance the requests > - Requested that the sending party use some of the batching settings on > the Remote Port (i.e., Count = 20, Size = 100 MB, Duration = 10 sec) > - Reduced the thread count on our Input Port to 2 > > Are there any known nifi.properties that can be set to help mitigate this > problem? Again, it only seems to be a problem when we are both receiving > site-to-site traffic and doing our normal processing, but taking each of > those activities in isolation seems to be okay. > > Thanks, > Joe > -- I know what it is to be in need, and I know what it is to have plenty. I have learned the secret of being content in any and every situation, whether well fed or hungry, whether living in plenty or in want. I can do all this through him who gives me strength. *-Philippians 4:12-13*
