Great comments all. I agree with the architecture comment about push monitoring. I've been monitoring applications for more than 2 decades now, but sometimes you have to work around the limitations of the situation. It would be really nice if NiFi had this logic built-in, and frankly I'm surprised it is not yet. I can't be the only one who has had to deal with queues filling up, causing problems downstream. NiFi certainly knows that the queues fill up, they change color and execute back-pressure logic. If it would just do something simple like write a log/error message to a log file when this happens, I would be good. I have looked at the new metrics and reporting tasks but still haven't found the right thing to do to get notified when any queue in my instance fills up. Are there any examples of using them for a similar task you can share?
Thanks, Scott On Wed, Jul 21, 2021 at 11:29 AM [email protected] <[email protected]> wrote: > In general, it is a bad architecture to do monitoring via pull request. > You should always push. I recommend a look at the book "The Art of > Monitoring" by James Turnbull. > > I also recommend the very good articles by Pierre Villard on the subject > of NiFi monitoring at > https://pierrevillard.com/2017/05/11/monitoring-nifi-introduction/. > > Hope this helps. > > Mit freundlichen Grüßen / best regards > Kay-Uwe Moosheimer > > Am 21.07.2021 um 16:45 schrieb Andrew Grande <[email protected]>: > > > Can't you leverage some of the recent nifi features and basically run sql > queries over NiFi metrics directly as part of the flow? Then act on it with > a full flexibility of the flow. Kinda like a push design. > > Andrew > > On Tue, Jul 20, 2021, 2:31 PM scott <[email protected]> wrote: > >> Hi all, >> I'm trying to setup some monitoring of all queues in my NiFi instance, to >> catch before queues become full. One solution I am looking at is to use the >> API, but because I have a secure NiFi that uses LDAP, it seems to require a >> token that expires in 24 hours or so. I need this to be an automated >> solution, so that is not going to work. Has anyone else tackled this >> problem with a secure LDAP enabled cluster? >> >> Thanks, >> Scott >> >
