[
https://issues.apache.org/jira/browse/SAMZA-540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371863#comment-14371863
]
Yan Fang commented on SAMZA-540:
--------------------------------
{quote}
Question here though, this latest offset, how does Samza know about this? is
this a parameter of SystemStreamPartitionMetadata?
{quote}
yes, this is the right place. But since we have not decided to add this
metadata, have not changed the API. That's why you can not see it. And that's
why I think it's better to postpone exposing this metric.
{quote}
I added a metrics registry and add gauge objects into this. Is this what you
meant? I also added a test but I have my doubts about it so your comments are
more than welcome.
{quote}
Yes, this is exact what I mean! Thank you. Just a few comments to improve the
patch a little:
1) could you create a new Class, OffsetManagerMetrics which extends
MetricsHelper? Refer to
[TaskInstanceMetrics|https://github.com/apache/samza/blob/master/samza-core/src/main/scala/org/apache/samza/container/TaskInstanceMetrics.scala].
This will make the code a little clear and have the same pattern as other
classes.
2) In the OffsetMaganerMetrics, you may want to create a map
systemStreamPartition -> Gauge. You can find the similar fashion for val
{code}lag = new ConcurrentHashMap[TopicAndPartition, Gauge[Long]] {code} in
[KafkaSystemConsumerMetrics|https://github.com/apache/samza/blob/master/samza-kafka/src/main/scala/org/apache/samza/system/kafka/KafkaSystemConsumerMetrics.scala]
. Then update the value of the Gauge.
3) when you update the Gauge's value, you use .set, not creating a new Gauge
(what you did in patch :) . Refer to
[here|https://github.com/apache/samza/blob/master/samza-kafka/src/main/scala/org/apache/samza/system/kafka/BrokerProxy.scala#L112]
to see how the Gauge value is updated.
After those changes, I think it's fine. Thank you again for your work.
> Expose latency related metrics in OffsetManager
> -----------------------------------------------
>
> Key: SAMZA-540
> URL: https://issues.apache.org/jira/browse/SAMZA-540
> Project: Samza
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 0.8.0
> Reporter: Yan Fang
> Labels: newbie
> Fix For: 0.10.0
>
> Attachments: SAMZA-540.1.patch
>
>
> Follow-up to SAMZA-503
> Expose checkpointed offset, max offset, and max offset - checkpointed offset
> metrics in OffsetManager.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
