[
https://issues.apache.org/jira/browse/OOZIE-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated OOZIE-1817:
---------------------------------
Attachment: OOZIE-1817.patch
The patch adds a MetricsInstrumentationService and MetricsInstrumentation
class.
To maintain backwards compatibility, I wrote the MetricsInstrumentation to have
the same API as Instrumentation, but it's backed by the Codahale Metrics
instead of Oozie Instrumentation metrics (thanks to [~tucu00] for a bunch of
help with that!). I think it might make sense to remove Instrumentation in
Oozie 5 and refactor MetricsInstrumentation to do certain things in a better
way.
Some info:
- {{Instrumentation.Cron}} is backed by Metrics {{Timer}}
- {{Instrumentation.Variable}} is backed by Metrics {{Gauge}}
- Instrumentation counter is backed by Metrics {{Counter}}
- {{Instrumentation.Sampler}} is backed by Metrics {{Histogram}}
-- {{Histogram}} is actually more accurate than {{Sampler}} in most cases
- The Metrics json for the REST API is handled for us via the {{MetricsModule}}
class
-- The Metrics json has a little bit different format than Instrumentation
json. There are different categories there are different types of metrics.
Also, there is only one level, so the groups are not hierarchically shown.
- Metrics also has a way of automatically logging the metrics to Slf4j, among a
few other things; unfortunately, it doesn't have a log4j one. I copied and
slightly modified their {{Sl4jReporter}} class into {{XLogReporter}} which is
almost identical except it uses {{XLog}}.
-- The output format to oozie-instrumentation.log is different. each metric is
on its own log line because {{log.info(...)}} is called for each one instead of
generating a giant String and dumping that. We could change this fairly
easily, but this should use less memory.
- Instead of the three simple variables for JVM memory info, Metrics has a
built-in set of Gauges with a bunch of other metrics so I'm using that.
- Metrics also has some metrics for how often log messages are written, which
I've added
- The Metrics show up at {{/admin/metrics}} endpoint. When using
Instrumentation, this endpoint doesn't work and when using Metrics
{{/admin/instrumentation}} endpoint doesn't work.
- The Oozie Web UI replaces the "Instrumentation" tab with a similar "Metrics"
tab when using Metrics
- I also cleaned up a few cron issues.
> Oozie timers are not biased
> ---------------------------
>
> Key: OOZIE-1817
> URL: https://issues.apache.org/jira/browse/OOZIE-1817
> Project: Oozie
> Issue Type: Improvement
> Components: monitoring
> Affects Versions: 4.0.0
> Reporter: Gilad Wolff
> Assignee: Robert Kanter
> Attachments: OOZIE-1817.patch, OOZIE-1817.patch
>
>
> Oozie timers are not biased, that is, the statistical metrics they expose are
> over the run-time of the Oozie server instead of a window of time. This makes
> them not very useful especially after the server has been running for a while
> (codehale has very efficient and easy to use biased histograms that can be
> used instead).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
