[
https://issues.apache.org/jira/browse/EAGLE-75?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15044458#comment-15044458
]
Libin, Sun commented on EAGLE-75:
---------------------------------
After some further investigation, I find there are something we should notice
when using dropwizard metrics
Dropwizard Metric Framework launch a thread for metric reporting, and allow
user to pass a user define clock for the reporter, that's good feature if we
want to use message timestamp, especially when doing catching up
But one problem is when we consume hdfs audit log, different partition has
different clock(latest message time), and when data passed into our eagle
executor, the partition info lost, what we process is HDFSAuditLogObject, then
each user should has a clock
if user count is large, saying 1000, and we use 8 kafka spout, then there
should be 125 thread each spout for the single metric, which is too heavy
{code:title=metric.java|borderStyle=solid}
MetricRegistry registry = new MetricRegistry();
ConsoleReporter reporter = ConsoleReporter.forRegistry(registry)
.withClock(clock)
.build();
reporter.start(1, TimeUnit.SECONDS);
{code}
> Leverage dropwizard metrics for generating Eagle Topology and DataSource
> Metrics
> --------------------------------------------------------------------------------
>
> Key: EAGLE-75
> URL: https://issues.apache.org/jira/browse/EAGLE-75
> Project: Eagle
> Issue Type: Sub-task
> Affects Versions: 0.3.0
> Reporter: Libin, Sun
> Assignee: Libin, Sun
> Fix For: 0.3.0
>
>
> As a monitoring system, eagle need to report & record both the datasource
> data distribution metric & topology running status metrics to let user have
> better understanding of the system running status.
> Previously we added a simple metric framework to support user cases like
> [EAGLE-2](https://issues.apache.org/jira/browse/EAGLE-2) ,
> [EAGLE-24](https://issues.apache.org/jira/browse/EAGLE-24)
> Considering we will have a lot metric user cases to onboard later, we need a
> better metric framework to make it easier to support these cases, and after
> some investigation, we found [Dropwizard
> metrics](https://github.com/dropwizard/metrics) is a good one that can match
> our needs, So we want to integrate Dropwizard metrics with eagle
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
