[
https://issues.apache.org/jira/browse/GOBBLIN-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sudarshan Vasudevan closed GOBBLIN-1087.
----------------------------------------
> Track and report histogram of observed lag from Gobblin Kafka pipeline
> ----------------------------------------------------------------------
>
> Key: GOBBLIN-1087
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1087
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-kafka
> Affects Versions: 0.15.0
> Reporter: Sudarshan Vasudevan
> Assignee: Shirshanka Das
> Priority: Major
> Fix For: 0.15.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> In this PR, we instrument the KafkaExtractor to track the observed latency of
> Kafka consumer records processed by the pipeline. Here, observed latency is
> measured as the time difference between processing time of the record and the
> original creation time. The latency distribution is tracked in an
> HdrHistogram, which is serialized into a string when emitted as part of a
> GobblinTrackingEvent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
