[
https://issues.apache.org/jira/browse/HADOOP-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mostafa Elhemali updated HADOOP-9090:
-------------------------------------
Description:
Updated description based on feedback:
We have a need to publish metrics out of some short-living processes, which is
not really well-suited to the current metrics system implementation which
periodically publishes metrics asynchronously (a behavior that works great for
long-living processes). Of course I could write my own metrics system, but it
seems like such a waste to rewrite all the awesome code currently in the
MetricsSystemImpl and supporting classes.
The way this JIRA solves this problem is adding a new method
publishMetricsNow() to the MetricsSystemImpl() class, that does a synchronous
out-of-band push of the metrics from the sources to the sink. I also add a
method to MetricsSinkAdapter (putMetricsImmediate) to support that change.
was:
We have a need to publish metrics out of some short-living processes, which is
not really well-suited to the current metrics system implementation which
periodically publishes metrics asynchronously (a behavior that works great for
long-living processes). Of course I could write my own metrics system, but it
seems like such a waste to rewrite all the awesome code currently in the
MetricsSystemImpl and supporting classes.
The way I'm proposing to solve this is to:
1. Refactor the MetricsSystemImpl class into an abstract base MetricsSystemImpl
class (common configuration and other code) and a concrete
PeriodicPublishMetricsSystemImpl class (timer thread).
2. Refactor the MetricsSinkAdapter class into an abstract base
MetricsSinkAdapter class (common configuration and other code) and a concrete
AsyncMetricsSinkAdapter class (asynchronous publishing using the SinkQueue).
3. Derive a new simple class OnDemandPublishMetricsSystemImpl from
MetricsSystemImpl, that just exposes a synchronous publish() method to do all
the work.
4. Derive a SyncMetricsSinkAdapter class from MetricsSinkAdapter to just
synchronously push metrics to the underlying sink.
Does that sound reasonable? I'll attach the patch with all this coded up and
simple tests (could use some polish I guess, but wanted to get everyone's
opinion first). Notice that this is somewhat of a breaking change since
MetricsSystemImpl is public (although it's marked with
InterfaceAudience.Private); if the breaking change is a problem I could just
rename the refactored classes so that PeriodicPublishMetricsSystemImpl is still
called MetricsSystemImpl (and MetricsSystemImpl -> BaseMetricsSystemImpl).
> Refactor MetricsSystemImpl to allow for an on-demand publish system
> -------------------------------------------------------------------
>
> Key: HADOOP-9090
> URL: https://issues.apache.org/jira/browse/HADOOP-9090
> Project: Hadoop Common
> Issue Type: New Feature
> Components: metrics
> Reporter: Mostafa Elhemali
> Priority: Minor
> Attachments: HADOOP-9090.2.patch,
> HADOOP-9090.justEnhanceDefaultImpl.2.patch,
> HADOOP-9090.justEnhanceDefaultImpl.3.patch,
> HADOOP-9090.justEnhanceDefaultImpl.4.patch,
> HADOOP-9090.justEnhanceDefaultImpl.patch, HADOOP-9090.patch
>
>
> Updated description based on feedback:
> We have a need to publish metrics out of some short-living processes, which
> is not really well-suited to the current metrics system implementation which
> periodically publishes metrics asynchronously (a behavior that works great
> for long-living processes). Of course I could write my own metrics system,
> but it seems like such a waste to rewrite all the awesome code currently in
> the MetricsSystemImpl and supporting classes.
> The way this JIRA solves this problem is adding a new method
> publishMetricsNow() to the MetricsSystemImpl() class, that does a synchronous
> out-of-band push of the metrics from the sources to the sink. I also add a
> method to MetricsSinkAdapter (putMetricsImmediate) to support that change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
