I switched to Java 8 and I'm good now. Also I did change the log to WARN only and I think that tremendously helped in reducing the java memory usage increase. Now my CPU, I/O and RAM are much, much more stable.
On Fri, Jul 1, 2016 at 10:43 AM Matthew Clarke <[email protected]> wrote: > It is interesting that you saw 20 full garbage collects in 24 hours. You > may want to use jconsoles to view your heap usage for your NiFi process I'd > to see what is using the most heap space. > On Jun 30, 2016 8:29 PM, "Matthew Clarke" <[email protected]> > wrote: > >> Old gen would be considered full garbage collection. You are good. >> On Jun 30, 2016 8:22 PM, "Stéphane Maarek" <[email protected]> >> wrote: >> >>> Got it thanks, >>> So I assume none of this below if full garbage collection? This has >>> happened in less than a day >>> >>> *G1 Young Generation* 23568 times 00:07:49.718 >>> *G1 Old Generation* 20 times 00:00:13.226 >>> >>> On Thu, Jun 30, 2016 at 10:22 PM Mark Payne <[email protected]> >>> wrote: >>> >>>> Stephane, >>>> >>>> This is normal and expected. Even though there is no data flowing >>>> through your system, the framework >>>> does have some maintenance that it has to do, such as pruning old >>>> archived data from the Content Repository >>>> and Provenance Repository, checkpointing the FlowFile Repository, >>>> keeping stats for all of the processors, etc., etc. >>>> >>>> If you were to see "Full Garbage Collections" taking place then there >>>> may be some concern but minor garbage collections >>>> are pretty frequent and shouldn't cause concern. >>>> >>>> Thanks >>>> -Mark >>>> >>>> >>>> On Jun 30, 2016, at 12:37 AM, Stéphane Maarek < >>>> [email protected]> wrote: >>>> >>>> Heap (32.0%) >>>> *Max* 5 GB >>>> *Total* 4 GB >>>> *Used* 1.62 GB >>>> *Free* 2.38 GB >>>> dd >>>> >>>> I have an issue, I followed this link to "optimize" HDF >>>> https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.html >>>> >>>> >>>> Basically, I have two UDP receivers but they don't receive any data >>>> right now. They produce 14,000 tasks each every 5 minutes (which is fine). >>>> The issue is that my Heap increases to the top 4 GB every minute I would >>>> say, triggering a garbage collection event (I use G1). Is that normal? I >>>> would expect the memory to stay constant if no data goes through my system >>>> at all >>>> >>>> Thanks, >>>> Stephane >>>> >>>> >>>>
