[
https://issues.apache.org/jira/browse/KAFKA-14112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elkhan Eminov updated KAFKA-14112:
----------------------------------
Description:
The replication offset lag is the difference of the {*}l{*}ast {*}e{*}nd
{*}o{*}ffset of the source partition (LEO) the {*}l{*}ast {*}r{*}eplicated
_source_ {*}o{*}ffset (LRO).
The offset lag is a difference (LEO-LRO), and its constituents are calculated
at different points of time and place:
* LEO shall be calculated during source task's poll loop (ready to get it from
the consumer)
* LRO shall be kept in an in-memory "cache", that is updated during the task's
producer callback
The difference shall be calculated when the freshest LEO acquired in the poll
loop. The calculated amount shall be defined as a MirrorMaker metric.
This would describe to amount of "to be replicated" number of records for a
certain topic-partition.
was:
The offset lag is the difference of the last replicated record's (LRO) source
offset and the end offset of the source (LEO).
The offset lag is a difference (LRO-LEO), but its constituents calculated at
different points of time and place
* LEO shall be calculated during source task's poll loop (ready to get it from
the consumer)
* LRO shall be kept in an in-memory "cache", that is updated during the task's
producer callback
LRO is initialized when task is started, from the offset store. The difference
shall be calculated when the freshest LEO acquired
in the poll loop. The calculated amount shall be defined as a MirrorMaker
metric.
This would describe to amount of "to be replicated" number of records for a
certain topic-partition.
> Expose replication-offset-lag Mirror metric
> -------------------------------------------
>
> Key: KAFKA-14112
> URL: https://issues.apache.org/jira/browse/KAFKA-14112
> Project: Kafka
> Issue Type: Improvement
> Components: mirrormaker
> Reporter: Elkhan Eminov
> Assignee: Elkhan Eminov
> Priority: Minor
>
> The replication offset lag is the difference of the {*}l{*}ast {*}e{*}nd
> {*}o{*}ffset of the source partition (LEO) the {*}l{*}ast {*}r{*}eplicated
> _source_ {*}o{*}ffset (LRO).
> The offset lag is a difference (LEO-LRO), and its constituents are calculated
> at different points of time and place:
> * LEO shall be calculated during source task's poll loop (ready to get it
> from the consumer)
> * LRO shall be kept in an in-memory "cache", that is updated during the
> task's producer callback
> The difference shall be calculated when the freshest LEO acquired in the poll
> loop. The calculated amount shall be defined as a MirrorMaker metric.
> This would describe to amount of "to be replicated" number of records for a
> certain topic-partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
