> That should certainly be doable. It just never seemed clear that JMX > was the way to go. I think the theory had always been that if we > expose these sorts of things through the REST API over which we have > clearly defined authentication/authorization schemes we could then > support a variety of clients (people, systems).
That's definitely true. What we do over on HBase is wire up our metrics to JMX and provide a REST endpoint in every process that presents all mbean data as JSON. We don't set up JMX remote access, but let users edit configs for launch scripts if RMI JMX access is desired. This has some minor advantages in my opinion: - We don't need to do anything special to expose JVM runtime data like detailed information on GC pools and times. - If it is desired for some reason, devs/admins then do have the option to set up the usual properties in the environment for RMI to JMX using the standard options for auth and such. In my experience some shops do want to work with JMX, usually when other systems that export operational data via JMX are already under management. On Thu, Feb 19, 2015 at 9:52 AM, Joe Witt <[email protected]> wrote: > Andrew, > > That should certainly be doable. It just never seemed clear that JMX > was the way to go. I think the theory had always been that if we > expose these sorts of things through the REST API over which we have > clearly defined authentication/authorization schemes we could then > support a variety of clients (people, systems). > > I'll have to read up more on the options for security around JMX and > the sort of client interactions that opens up. That said, it would > also be good to have a sense of tradeoffs of opening up further > avenues of client interaction beyond the REST API. Once we get our > REST API docs legit again (had to pull due to licensing concerns) then > that should help a lot too. > > Thanks > Joe > > > On Thu, Feb 19, 2015 at 12:42 PM, Andrew Purtell <[email protected]> > wrote: > > Would it be possible to instantiate a Metrics JmxReporter for each > process? > > Sounds like enabling metrics export via JMX could be an interesting JIRA > at > > any rate. > > > > I did a quick search of the User, Developer, and Admin guides and there > are > > no occurrences of the string "JMX" > > > > On Tue, Feb 17, 2015 at 10:25 PM, Otis Gospodnetic < > > [email protected]> wrote: > > > >> Hi Joe, > >> > >> Kafka devs describe their MBeans here - > >> https://kafka.apache.org/08/ops.html > >> - section 6.5, which is good for people who want to understand what each > >> metric means. > >> It sounds like the reporting task stuff is all about the push. I think > >> pull may be better for us, so if you can document the end point (I > assume > >> we are talking about an HTTP/REST API? That returns various metrics in > >> JSON?), that would be very very helpful (and not just to us). > >> > >> Thanks, > >> Otis > >> -- > >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management > >> Solr & Elasticsearch Support * http://sematext.com/ > >> > >> > >> On Wed, Feb 18, 2015 at 12:50 AM, Joe Witt <[email protected]> wrote: > >> > >> > Otis, > >> > > >> > We're quite light on documentation thus far for the Reporting Task > >> > API. But you can see a bit about it here: > >> > > >> > > >> > > >> > http://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html#reporting-tasks > >> > > >> > > >> > http://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html#developing-a-reporting-task > >> > > >> > The javadocs are published to Maven as well so that will do nicely to > >> > give a sense of what we expose from the Internals of NiFi to an > >> > implementation of a reporting task. > >> > > >> > But feel free to ask detailed questions and as many as you want. That > >> > will give us good information on the types of things we need to cover > >> > and in how much detail. > >> > > >> > Thanks > >> > Joe > >> > > >> > On Wed, Feb 18, 2015 at 12:46 AM, Otis Gospodnetic > >> > <[email protected]> wrote: > >> > > Hi Joe, > >> > > > >> > > On Wed, Feb 18, 2015 at 12:15 AM, Joe Witt <[email protected]> > wrote: > >> > > > >> > >> Otis, > >> > >> > >> > >> Glad you are liking it. > >> > >> > >> > >> We do provide support for both push and pull based metrics > >> > >> reporting/gathering. > >> > >> > >> > >> For NiFi being able to push out metrics: > >> > >> We do have an API for what we call "Reporting Tasks". For example > we > >> > >> have a Ganglia reporter today. > >> > >> > >> > > > >> > > So we are talking about the Yammer/Coda Hale's Metrics being used > here? > >> > > If so, are the metrics also available via JMX, too? > >> > > > >> > > For a client being able to pull metrics from NiFI: > >> > >> The REST API is the way to go. > >> > >> > >> > > > >> > > This sounds good. Are this API and the metrics it exposes > documented > >> > > anywhere? > >> > > I looked in the guides, but couldn't find anything. > >> > > > >> > > > >> > >> Do either of these sound like a path that can work for you? Let us > >> > >> know if you'd like more details. > >> > >> > >> > > > >> > > Thanks, > >> > > Otis > >> > > -- > >> > > Monitoring * Alerting * Anomaly Detection * Centralized Log > Management > >> > > Solr & Elasticsearch Support * http://sematext.com/ > >> > > > >> > > > >> > > > >> > > > >> > >> > >> > >> On Wed, Feb 18, 2015 at 12:01 AM, Otis Gospodnetic > >> > >> <[email protected]> wrote: > >> > >> > Hi, > >> > >> > > >> > >> > Nifi looks mightily impressive! > >> > >> > I see Nifi has built-in monitoring and metrics.... but is there > an > >> API > >> > >> that > >> > >> > would allow something like SPM to add support for Nifi > monitoring? > >> Or > >> > >> JMX? > >> > >> > > >> > >> > Thanks, > >> > >> > Otis > >> > >> > -- > >> > >> > Monitoring * Alerting * Anomaly Detection * Centralized Log > >> Management > >> > >> > Solr & Elasticsearch Support * http://sematext.com/ > >> > >> > >> > > >> > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
