Joseph Witt commented on NIFI-4905:

[~jzahner] ok - i strongly suspect there is custom code processors/validation 
methods that are causing this slow response time.  While we should definitely 
work to have such things be entirely asynchronous that isn't really feasible 
'right now'.  It is important to validate the processors before the show up in 
a shared context in large usage.


Now, I could be wrong and it might be something more 
fundamental/frameworky/standard processor-ish.  But I strongly recommend you 
validate the custom code modules validate methods first.  And you could easily 
add log statements in their validate methods to track time.


That said, we should do that in the framework logic someplace (log validate 
call durations)....  [~markap14] do you think this would fit within the 
processor diagnostics stuff you had the PR for recently?

> Cluster response time is slow
> -----------------------------
>                 Key: NIFI-4905
>                 URL: https://issues.apache.org/jira/browse/NIFI-4905
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core UI
>    Affects Versions: 1.5.0
>         Environment: Centos 7.3
> NiFi 1.5.0 Cluster with 8 physical nodes
>            Reporter: Josef Zahner
>            Priority: Major
>         Attachments: cpu_load_nifi.PNG, root_canvas.png
> we are actually working on a PoC with 8 nodes (HP BL460c Blades, 24 Cores, 
> 44GB RAM) in a NiFi 1.5.0 cluster. Our configuration has about 160 processors 
> and all of them are stopped. Even in the stopped state, we are constantly 
> getting the messages below for all nodes, not only for the primary node.
> {code:java}
> Response time from nifi2-07.xyz.ch:8080 was slow for each of the last 3 
> requests made. To see more information about timing, enable DEBUG logging for 
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator
> {code}
> If you are on the root canvas, you feel that it takes a few seconds until it 
> response after a refresh. We have already tuned the parameters below, but 
> without any luck. After restart of NiFi it is fine for a few minutes, but 
> then the messages return.
> {code:java}
> nifi.cluster.protocol.heartbeat.interval=15 sec
> nifi.cluster.node.protocol.threads=40
> nifi.cluster.node.protocol.max.threads=80
> nifi.cluster.node.connection.timeout=60 sec
> nifi.cluster.node.read.timeout=60 sec
> {code}
> The nodes have absolutely no load beside of NiFi. What surprised me was, that 
> when the UI refreshs (default every 30s) it produces about 20% cpu load on my 
> machine. And remember, it's a 24x2.9GHz blade server.
> !cpu_load_nifi.PNG!
> That's a picture of my root canvas:
> !root_canvas.png!
> Actually we can't work with the cluster under this circumstances because the 
> gui always gets so slow.
> Is this a bug or normal behavior? Do we have to many elements on the GUI or 
> what could cause this issue?
> Cheers

This message was sent by Atlassian JIRA

Reply via email to