[
https://issues.apache.org/jira/browse/NIFI-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17510264#comment-17510264
]
Josef Zahner commented on NIFI-9820:
------------------------------------
Ah right, you are referring to a case where NiFi version is smaller dann
1.14.0.
Anyway, it's always a tradeoff, in our case we had a massive memory issue due
to the fact that the default was so high and would be too high with the new
value (default = no of CPUs) as well. In my point of view it would be better to
have worse performance per default, instead of having a crashing system like in
our case. If the performance is worse, you recognise this due to the fact the
the processor has a bad performance and the queues get filled. So you
understand relatively fast where the issue could be (somewhere within the
PutKudu processor) and it's an easy fix. On the other hand with the default
high, you are getting a memory warning and you don't see where it comes from.
So without deep analysis, it wouldn't be possible to find the culprit.
> Change PutKudu Property "Kudu Client Worker Count" Default Value
> ----------------------------------------------------------------
>
> Key: NIFI-9820
> URL: https://issues.apache.org/jira/browse/NIFI-9820
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 1.15.3
> Reporter: Josef Zahner
> Assignee: David Handermann
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The PutKudu processor property "Kudu Client Worker Count" has a suboptimal
> value. Please don't use the current "number of CPUs multiplied by 2"
> behaviour as it leads to a massive amount of workers in our case with
> physical servers. We have a 8-node cluster where each server has 64 CPUs. We
> have about 30 PutKudu processors configured -> a lot of worker threads per
> default just for kudu.
> We have changed the number of worker threads in our case to the number of
> concurrent tasks. I don't know, maybe it would be great to set it a bit
> higher than that, but to be honest, I don't exactly understand the impact. It
> looks still fast with the current config.
> *To sum it up, please set a low default value (eg. 4 or 8) for the property
> "Kudu Client Worker Count" and not a pseudo dynamic one for the PutKudu
> processor.*
> Btw. are there any suggestions how big the number should be?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
