I like this idea. In addition to dropwizard, I think we need to rule out leveraging what is used in hadoop common. I don't know how generic it is but do know that there is a JMXServlet that can be used to expose the metrics in a REST-like way.
Leveraging hadoop common where possible is something that I like to consider more frequently and I think that we should have probably done that more from the start. I would definitely say to create the JIRA and continue the proposal there with high level design in attachments or wiki. On Fri, Dec 11, 2015 at 10:28 AM, sumit gupta <su...@apache.org> wrote: > Hi everyone, > > I wanted to start a discussion about adding the capability of > capturing metrics in Knox. The initial step could be something simple > like request processing metrics and exposing the numbers via JMX > and/or REST APIs. There is a bigger discussion around metering and > throttling that this could play into but maybe wise to avoid initially > (maybe...). > > One of the projects that I came across while looking into this is > https://dropwizard.github.io/metrics/3.1.0/. It seems to be popular > amongst other open source projects and has a lot of useful > functionality and seems quite stable (also has Apache 2.0 license). > > Another project that comes up in this space is http://prometheus.io/, > but I am strongly leaning towards dropwizard based on some initial > playing around that I’m doing. > > Anyhow, implementation aside, it would be good to discuss and agree > upon requirements. Things like: > > 1. What metrics are we interested in? > 2. Who are the potential consumers of this data? Dashboards? > 3. How do we want to expose the metrics? > 4. Do we want to capture metrics at a service level (e.g. All requests > made for WebHDFS)? > 5. Configuration (how do we want to turn this functionality on or off > or something more granular) > > > Of course the basic question is whether we would like to consider this > functionality at all. If we do, I’d be happy to file a JIRA and work > on it. > > Thoughts? > > Sumit >
