Josef Zahner created NIFI-8435:
----------------------------------
Summary: PutKudu 1.13.2 Memory Leak
Key: NIFI-8435
URL: https://issues.apache.org/jira/browse/NIFI-8435
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.13.2
Environment: NiFi 1.13.2, 8-Node Cluster running on CentOS 7, Kudu
1.10.0
Reporter: Josef Zahner
Attachments: grafana_heap_overview.png,
visualvm_bytes_detail_view.png, visualvm_total_bytes_used.png
We just upgraded from NiFi 1.11.4 to 1.13.2 and face a huge issue with PutKudu.
PutKudu on the 1.13.2 eats up all the heap memory and garbage collection can't
anymore free up the memory. We allow Java to use 31GB memory and as you can see
with NiFi 1.11.4 it will be used like it should with GC. However with NiFi
1.13.2 with our actual load it fills up the memory relatively fast. Manual GC
via visualvm tool didn't help at all to free up memory.
!grafana_heap_overview.png!
Visual VM shows the following culprit: !visualvm_total_bytes_used.png!
!visualvm_bytes_detail_view.png!
The bytes array shows millions of char data which isn't cleaned up. In fact
here 14,9GB memory (heapdump has been taken after a while of full load). If we
check the same on NiFi 1.11.4, the bytes array is nearly empty, around a few
hundred MBs.
As you could imagine we can't upload the heap dump as currently we have only
productive data on the system. But don't hesitate to ask questions about the
heapdump if you need more information.
I haven't done any screenshot of the processor config, but I can do that if you
wish (we are back to NiFi 1.11.4 at the moment).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
