[
https://issues.apache.org/jira/browse/SAMZA-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127185#comment-14127185
]
Yan Fang commented on SAMZA-349:
--------------------------------
{quote}
If you're willing for the metric to be approximate (which, in practice, is
usually fine), then you don't need to keep every single timing within your
aggregation period. It's sufficient to keep a randomly selected sample, for
example up to 1,000 values.
{quote}
This is good enough to give an approximate result. I think we can keep this as
an option as the default implementation. I am thinking having the option of
getting metric in a time interval, such, 5 mins/30 seconds, is helpful as well.
Prefer to leave this API there and users may want to use it. What do you think?
{quote}
In order to expire old values, you don't need to keep a timestamp for every
single value. If you want to aggregate over the last 5 minutes, a simple
approach is to keep a separate reservoir for every minute. To calculate the
metric, you can combine the samples from the last 5 minutely reservoirs. Once a
minute, you throw away the oldest reservoir which is no longer needed.
{quote}
Yes, this is true. Because we report the metrics every 1 min and I set the
timer to 5 mins as the default. But when the the metrics are reported in a
smaller interval, such as 30s or 10s, this implementation may have a problem.
It turns out that we still need to expire the old values based on timestamp.
Correct me if I understand it wrong.
> add timer in metrics
> --------------------
>
> Key: SAMZA-349
> URL: https://issues.apache.org/jira/browse/SAMZA-349
> Project: Samza
> Issue Type: Bug
> Reporter: Yan Fang
> Assignee: Yan Fang
> Attachments: SAMZA-349.1.patch, SAMZA-349.2.patch, SAMZA-349.3.patch,
> SAMZA-349.patch, SAMZA-349.patch
>
>
> If my understanding is correct, the metrics we provide are for every 60
> seconds and all counters will be reset every 60 seconds. Current the
> MetricsSnapshotReporter seems missing this implementation. It sends out the
> metrics every 60 seconds but does not reset the counter value.
> {code}
> registry.getGroup(group).foreach {
> case (name, metric) =>
> metric.visit(new MetricsVisitor {
> def counter(counter: Counter) = groupMsg.put(name,
> counter.getCount: java.lang.Long)
> def gauge[T](gauge: Gauge[T]) = groupMsg.put(name,
> gauge.getValue.asInstanceOf[Object])
> })
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
