Arun V This is a strong indication to review any custom components or processors in the flow which can consume large quantities of memory.
OutOfMemory errors in a general sense indicate either a single large thing caused the heap to be exhausted or you see errors like this about overhead limit exceeded which indicates another type which is there is simply too much retained information in the heap. Look at your code for the CableLabsDataMappingProcessor for instance. Does it have any class member variables that hold stuff like any kind of maps/caches/etc.. If so I'd start there. This sounds, with limited information, like a memory leak. When you increased the size of the memory did the flow stay online longer than before by some time interval? Point is increasing memory buys time but does not solve heap exhaustion in the event of a memory leak. Thanks On Thu, Oct 10, 2024 at 5:51 AM Varadarajan, Arun < arun.varadara...@paramount.com> wrote: > > Hi Team, > > We see our NIFI workflow getting hanged intermittently once / twice per > week. After which , even if we restart the processor [say ConsumeAMQP] , > this is not consuming the messages from the RMQ [though the queue had > messages]. > > After restarting the POD [in K8] , the workflow functions as expected. > Please note , we had already increased the POD memory and the issue is > still occurring. > > Also we are observing the below error for some time , and want to check if > this is the cause and if so , what could be the fix. > > CableLabsDataMappingProcessor[id=8a6f5d24-1fa5-38c6-1748-d1e9fe54ee93] > failed to process session due to java.lang.OutOfMemoryError: GC overhead > limit exceeded; Processor Administratively Yielded for 1 sec: > java.lang.OutOfMemoryError: GC overhead limit exceeded > java.lang.OutOfMemoryError: GC overhead limit exceeded > 2024-10-09 00:51:45,229 WARN [Timer-Driven Process Thread-1] > o.a.n.controller.tasks.ConnectableTask Administratively Yielding > CableLabsDataMappingProcessor[id=8a6f5d24-1fa5-38c6-1748-d1e9fe54ee93] due > to uncaught Exception: java.lang.OutOfMemoryError: GC overhead limit > exceeded > java.lang.OutOfMemoryError: GC overhead limit exceeded > > Any help would be greatly appreciated. > > Regards, > Arun V > >
