Joe, other way round is to support it, but let the End User download and drop the binaries needed (not shipping them by default).
I am keen on working to get the JMX support and based on my reading of the code so far, it would be a Reporting Task (shall keep it generic around Metrics) and add initially enable it only for JMX and later as we have more clarity, we can add more functions. Shall I create a JIRA for this? On Fri, Feb 20, 2015 at 3:15 AM, Joe Witt <[email protected]> wrote: > yeah absolutely - i was just addressing the fact that there is a > licensing issue (in general) with that library. > > We can still totally do fun things with JMX, ganglia, etc.. Just > takes a little more work when the best libs aren't' readily useable. > > Thanks > Joe > > On Thu, Feb 19, 2015 at 4:42 PM, Andrew Purtell <[email protected]> wrote: >> Would it suffice to not support Ganglia reporting? Then you'd not be >> bringing in metrics-ganglia* and its transitive dependencies? >> >> * - And can actively exclude it in POMs if necessary >> >> >> >> On Thu, Feb 19, 2015 at 1:39 PM, Joe Witt <[email protected]> wrote: >> >>> Otis, >>> >>> We'll take a fresh look at JMX but if you want to help contribute in >>> this area that would be awesome as well. >>> >>> Regarding the Metric library and its licensing fitness: >>> ---------------------------- >>> >>> It's important to keep in mind that transitive dependencies and their >>> licenses are just as important as direct dependencies. If you start >>> here: >>> >>> https://github.com/dropwizard/metrics [yep good this one is ASLv2] >>> from that we want to use >>> >>> https://github.com/dropwizard/metrics/tree/master/metrics-ganglia [yep >>> good still ASLv2 from parent but let's look at deps] >>> specifically this one: info.ganglia.gmetric4j:gmetric4j >>> >>> https://github.com/ganglia/gmetric4j [great it is BSD licensed and >>> thats ok too as it is Category-A but lets check its deps] >>> specifically this one: org.acplt.remotetea:remotetea-oncrpc >>> >>> http://sourceforge.net/projects/remotetea/ [And this is where things >>> are problematic. This one is LGPLv2 which is Category-X - not allowed >>> for either source or binary dependency] >>> Upon further review the source does indeed depend on the remotetea >>> library so we can't simply ignore that dependency. Which means we can >>> use the gmetric4j library and thus can't use metrics-ganglia library. >>> >>> Now this is just from the ganglia tree. Other parts of the metrics >>> library might well be fine. We're getting around this problem today >>> by using the older metrics library and its older ganglia support which >>> doesn't have this dependency licensing issue. >>> >>> >>> Thanks >>> Joe >>> >>> On Thu, Feb 19, 2015 at 3:56 PM, Otis Gospodnetic >>> <[email protected]> wrote: >>> > +1 to everything that Andrew said. >>> > I think there is actually no licensing issue with Metric library - it's >>> > ASLv2 - https://github.com/dropwizard/metrics/blob/master/LICENSE >>> > >>> > re JMX and auth stuff, we've documented it for SPM users here: >>> > >>> https://sematext.atlassian.net/wiki/display/PUBSPM/SPM+Monitor+-+Standalone >>> > >>> > Otis >>> > -- >>> > Monitoring * Alerting * Anomaly Detection * Centralized Log Management >>> > Solr & Elasticsearch Support * http://sematext.com/ >>> > >>> > >>> > On Thu, Feb 19, 2015 at 1:12 PM, Andrew Purtell <[email protected]> >>> wrote: >>> > >>> >> > That should certainly be doable. It just never seemed clear that JMX >>> >> > was the way to go. I think the theory had always been that if we >>> >> > expose these sorts of things through the REST API over which we have >>> >> > clearly defined authentication/authorization schemes we could then >>> >> > support a variety of clients (people, systems). >>> >> >>> >> That's definitely true. >>> >> >>> >> What we do over on HBase is wire up our metrics to JMX and provide a >>> REST >>> >> endpoint in every process that presents all mbean data as JSON. We don't >>> >> set up JMX remote access, but let users edit configs for launch scripts >>> if >>> >> RMI JMX access is desired. This has some minor advantages in my opinion: >>> >> - We don't need to do anything special to expose JVM runtime data like >>> >> detailed information on GC pools and times. >>> >> - If it is desired for some reason, devs/admins then do have the option >>> to >>> >> set up the usual properties in the environment for RMI to JMX using the >>> >> standard options for auth and such. >>> >> >>> >> In my experience some shops do want to work with JMX, usually when other >>> >> systems that export operational data via JMX are already under >>> management. >>> >> >>> >> >>> >> >>> >> On Thu, Feb 19, 2015 at 9:52 AM, Joe Witt <[email protected]> wrote: >>> >> >>> >> > Andrew, >>> >> > >>> >> > That should certainly be doable. It just never seemed clear that JMX >>> >> > was the way to go. I think the theory had always been that if we >>> >> > expose these sorts of things through the REST API over which we have >>> >> > clearly defined authentication/authorization schemes we could then >>> >> > support a variety of clients (people, systems). >>> >> > >>> >> > I'll have to read up more on the options for security around JMX and >>> >> > the sort of client interactions that opens up. That said, it would >>> >> > also be good to have a sense of tradeoffs of opening up further >>> >> > avenues of client interaction beyond the REST API. Once we get our >>> >> > REST API docs legit again (had to pull due to licensing concerns) then >>> >> > that should help a lot too. >>> >> > >>> >> > Thanks >>> >> > Joe >>> >> > >>> >> > >>> >> > On Thu, Feb 19, 2015 at 12:42 PM, Andrew Purtell <[email protected] >>> > >>> >> > wrote: >>> >> > > Would it be possible to instantiate a Metrics JmxReporter for each >>> >> > process? >>> >> > > Sounds like enabling metrics export via JMX could be an interesting >>> >> JIRA >>> >> > at >>> >> > > any rate. >>> >> > > >>> >> > > I did a quick search of the User, Developer, and Admin guides and >>> there >>> >> > are >>> >> > > no occurrences of the string "JMX" >>> >> > > >>> >> > > On Tue, Feb 17, 2015 at 10:25 PM, Otis Gospodnetic < >>> >> > > [email protected]> wrote: >>> >> > > >>> >> > >> Hi Joe, >>> >> > >> >>> >> > >> Kafka devs describe their MBeans here - >>> >> > >> https://kafka.apache.org/08/ops.html >>> >> > >> - section 6.5, which is good for people who want to understand what >>> >> each >>> >> > >> metric means. >>> >> > >> It sounds like the reporting task stuff is all about the push. I >>> >> think >>> >> > >> pull may be better for us, so if you can document the end point (I >>> >> > assume >>> >> > >> we are talking about an HTTP/REST API? That returns various >>> metrics >>> >> in >>> >> > >> JSON?), that would be very very helpful (and not just to us). >>> >> > >> >>> >> > >> Thanks, >>> >> > >> Otis >>> >> > >> -- >>> >> > >> Monitoring * Alerting * Anomaly Detection * Centralized Log >>> Management >>> >> > >> Solr & Elasticsearch Support * http://sematext.com/ >>> >> > >> >>> >> > >> >>> >> > >> On Wed, Feb 18, 2015 at 12:50 AM, Joe Witt <[email protected]> >>> >> wrote: >>> >> > >> >>> >> > >> > Otis, >>> >> > >> > >>> >> > >> > We're quite light on documentation thus far for the Reporting >>> Task >>> >> > >> > API. But you can see a bit about it here: >>> >> > >> > >>> >> > >> > >>> >> > >> > >>> >> > >> >>> >> > >>> >> >>> http://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html#reporting-tasks >>> >> > >> > >>> >> > >> > >>> >> > >> >>> >> > >>> >> >>> http://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html#developing-a-reporting-task >>> >> > >> > >>> >> > >> > The javadocs are published to Maven as well so that will do >>> nicely >>> >> to >>> >> > >> > give a sense of what we expose from the Internals of NiFi to an >>> >> > >> > implementation of a reporting task. >>> >> > >> > >>> >> > >> > But feel free to ask detailed questions and as many as you want. >>> >> That >>> >> > >> > will give us good information on the types of things we need to >>> >> cover >>> >> > >> > and in how much detail. >>> >> > >> > >>> >> > >> > Thanks >>> >> > >> > Joe >>> >> > >> > >>> >> > >> > On Wed, Feb 18, 2015 at 12:46 AM, Otis Gospodnetic >>> >> > >> > <[email protected]> wrote: >>> >> > >> > > Hi Joe, >>> >> > >> > > >>> >> > >> > > On Wed, Feb 18, 2015 at 12:15 AM, Joe Witt <[email protected] >>> > >>> >> > wrote: >>> >> > >> > > >>> >> > >> > >> Otis, >>> >> > >> > >> >>> >> > >> > >> Glad you are liking it. >>> >> > >> > >> >>> >> > >> > >> We do provide support for both push and pull based metrics >>> >> > >> > >> reporting/gathering. >>> >> > >> > >> >>> >> > >> > >> For NiFi being able to push out metrics: >>> >> > >> > >> We do have an API for what we call "Reporting Tasks". For >>> >> example >>> >> > we >>> >> > >> > >> have a Ganglia reporter today. >>> >> > >> > >> >>> >> > >> > > >>> >> > >> > > So we are talking about the Yammer/Coda Hale's Metrics being >>> used >>> >> > here? >>> >> > >> > > If so, are the metrics also available via JMX, too? >>> >> > >> > > >>> >> > >> > > For a client being able to pull metrics from NiFI: >>> >> > >> > >> The REST API is the way to go. >>> >> > >> > >> >>> >> > >> > > >>> >> > >> > > This sounds good. Are this API and the metrics it exposes >>> >> > documented >>> >> > >> > > anywhere? >>> >> > >> > > I looked in the guides, but couldn't find anything. >>> >> > >> > > >>> >> > >> > > >>> >> > >> > >> Do either of these sound like a path that can work for you? >>> Let >>> >> us >>> >> > >> > >> know if you'd like more details. >>> >> > >> > >> >>> >> > >> > > >>> >> > >> > > Thanks, >>> >> > >> > > Otis >>> >> > >> > > -- >>> >> > >> > > Monitoring * Alerting * Anomaly Detection * Centralized Log >>> >> > Management >>> >> > >> > > Solr & Elasticsearch Support * http://sematext.com/ >>> >> > >> > > >>> >> > >> > > >>> >> > >> > > >>> >> > >> > > >>> >> > >> > >> >>> >> > >> > >> On Wed, Feb 18, 2015 at 12:01 AM, Otis Gospodnetic >>> >> > >> > >> <[email protected]> wrote: >>> >> > >> > >> > Hi, >>> >> > >> > >> > >>> >> > >> > >> > Nifi looks mightily impressive! >>> >> > >> > >> > I see Nifi has built-in monitoring and metrics.... but is >>> there >>> >> > an >>> >> > >> API >>> >> > >> > >> that >>> >> > >> > >> > would allow something like SPM to add support for Nifi >>> >> > monitoring? >>> >> > >> Or >>> >> > >> > >> JMX? >>> >> > >> > >> > >>> >> > >> > >> > Thanks, >>> >> > >> > >> > Otis >>> >> > >> > >> > -- >>> >> > >> > >> > Monitoring * Alerting * Anomaly Detection * Centralized Log >>> >> > >> Management >>> >> > >> > >> > Solr & Elasticsearch Support * http://sematext.com/ >>> >> > >> > >> >>> >> > >> > >>> >> > >> >>> >> > > >>> >> > > >>> >> > > >>> >> > > -- >>> >> > > Best regards, >>> >> > > >>> >> > > - Andy >>> >> > > >>> >> > > Problems worthy of attack prove their worth by hitting back. - Piet >>> >> Hein >>> >> > > (via Tom White) >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> Best regards, >>> >> >>> >> - Andy >>> >> >>> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein >>> >> (via Tom White) >>> >> >>> >> >> >> >> -- >> Best regards, >> >> - Andy >> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein >> (via Tom White) -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
