[
https://issues.apache.org/jira/browse/ATLAS-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4330:
--------------------------------------
Labels: kafka metrics topic-lag (was: )
> Add Kafka topics lag information on metrics and log
> ---------------------------------------------------
>
> Key: ATLAS-4330
> URL: https://issues.apache.org/jira/browse/ATLAS-4330
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Affects Versions: 2.1.0
> Reporter: Sarath Subramanian
> Assignee: Sarath Subramanian
> Priority: Major
> Labels: kafka, metrics, topic-lag
> Fix For: 3.0.0, 2.2.0
>
> Attachments: image-2021-06-07-15-03-28-166.png
>
>
> In metrics page, in addition to other Notification details. Lag information
> of kafka consumer topics will be useful to check the current backed up
> messages in Atlas:
> !image-2021-06-07-15-03-28-166.png!
> It will also be useful to print the lag information periodically in atlas
> application.log on a frequency that can be made configurable (print lag every
> 1 hour, 5min or so).
> *atlas.notification.consumer.topic.report.frequency.seconds=60*
> {code:java}
> 2021-06-07 21:56:06,121 INFO - NotificationProcessingStats:
> topic=ATLAS_SPARK_HOOK-0, currentOffset=18, endOffset=18, lag=0
> 2021-06-07 21:56:09,105 INFO - NotificationProcessingStats:
> topic=ATLAS_HOOK-0, currentOffset=1118, endOffset=1118, lag=0
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
