RE: [EXT] Re: Maximum Memory for NiFi?

2018-10-08 Thread Peter Wicks (pwicks)
@nifi.apache.org Subject: [EXT] Re: Maximum Memory for NiFi? Generally the larger the heap, the more likely to have long GC pauses. I'm surprised that you would need a 70GB heap given NiFi's design where the content of the flow files is generally not held in memory, unless many of the processors

Re: Maximum Memory for NiFi?

2018-10-05 Thread Bryan Bende
Generally the larger the heap, the more likely to have long GC pauses. I'm surprised that you would need a 70GB heap given NiFi's design where the content of the flow files is generally not held in memory, unless many of the processors you are using are not written in an optimal way to process

Maximum Memory for NiFi?

2018-10-04 Thread Peter Wicks (pwicks)
We've had some more clustering issues, and found that some nodes are running out of memory when we have unexpected spikes in data, then we run into a GC stop-the-world event... We lowered our thread count, and that has allowed the cluster to stabilize for the time being. Our hardware is pretty