Dan I'd add that when it doubt get a thread dump out. If ever the system seems to be behavior incorrectly run
bin/nifi.sh dump wait 30 seconds bin/nifi.sh dump And ideally send the full contents of the logs directory in a tar.gz tar czvf nifilogs.tar.gz logs Thanks Joe On Tue, Jan 30, 2018 at 8:43 AM, dan young <[email protected]> wrote: > Hello Koji, > > I don't see any OOM errors in the logs, I'll keep an eye on the avail. > thread count. Thank you. > > Regards, > Dan > > On Mon, Jan 29, 2018 at 10:49 PM Koji Kawamura <[email protected]> > wrote: >> >> Hi Dan, >> >> If all available Timer Driven Thread are being used (or hang >> unexpectedly for some reason), then no processor can be scheduled. >> The number at the left top the NiFi UI under the NiFi logo shows the >> number of threads currently working. >> If you see something more than 0, then I'd recommend to take some >> thread dumps to figure out what running thread is doing. >> >> Other than that, I've encountered unexpected behavior with a NiFi >> cluster if a node encountered OutOfMemory error. >> The cluster started to behave incorrectly as it can not replicate REST >> requests among nodes. I'd search any ERR logs in nifi-app.log. >> >> Thanks, >> Koji >> >> On Tue, Jan 30, 2018 at 1:10 PM, dan young <[email protected]> wrote: >> > Hello, >> > >> > We're running a secure 3 node 1.4 cluster. Has anyone seen any >> > behaviour >> > where the cluster just stops scheduling the running of flowfiles/tasks? >> > i.e. cron/timer, just don't run when they're suppose to. I've tried to >> > stop >> > and restart a processor that is say set to run ever 900sec, but nothing >> > happens. Then only thing I can do is to cycle through restarting each >> > node >> > in the cluster and then we're good for a few days....this is something >> > that >> > just started happening and has occurred twice in the last week or >> > so..... >> > Anything I should keep an eye out for or look for in the logs? >> > >> > Regards, >> > >> > Dan
