[
https://issues.apache.org/jira/browse/NIFI-9767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502703#comment-17502703
]
David Handermann commented on NIFI-9767:
----------------------------------------
Thanks for providing some additional configuration details [~jian659]. The
linked issue NIFI-8435 has extensive background on the {{PutKudu}} processor
and memory usage associated with the Apache Kudu client. That issue resulted
in the addition of the {{Kudu Client Worker Count}} processor property. Can
you provide the configured value for that property, and have you tried limiting
the worker count? Due the behavior of the underlying Netty library, PutKudu
will use a large number of threads and memory in its default configuration.
Reducing the worker count should help.
> PutKudu OutOfMemory Exception (GC overhead limit exceeded)
> ----------------------------------------------------------
>
> Key: NIFI-9767
> URL: https://issues.apache.org/jira/browse/NIFI-9767
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.15.0
> Environment: NiFi 1.15.0
> Reporter: jason
> Priority: Blocker
> Attachments: free_heap.png, pipeline.png, total_threads.png
>
>
> PutKudu 1.15.0 eats up all the heap memory. Must restart NiFi to free heap.
> We create twenty the same of pipelines for test, will out of memory on few
> hours.Pipeline as shown in pipeline.png
>
>
>
>
> This test scenario is to reproduce the OOM error (-Xms4G、-Xmx4G)
> In fact, found this issue because we have built some data pipelines including
> five putkudu processors(
> -Xms12G、-Xmx12G). It is found that the heap cannot be free and the number of
> threads continues to increase until OOM(After 8 hours). After analysis, it is
> found that it is a problem with the putkudu processor. We are confused why
> the heap is not freed automatically and there are more and more threads
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
