[
https://issues.apache.org/jira/browse/HBASE-22660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-22660:
-----------------------------------
Summary: Probabilistic end to end tracking of cross cluster replication
latency (was: Precise end to end tracking of cross cluster replication latency)
> Probabilistic end to end tracking of cross cluster replication latency
> ----------------------------------------------------------------------
>
> Key: HBASE-22660
> URL: https://issues.apache.org/jira/browse/HBASE-22660
> Project: HBase
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Priority: Major
>
> ageOfLastShippedOp tracks replication latency forward from the point where a
> source process tailing a WAL has found an edit to ship. This is not an end to
> end measure.
> To achieve a holistic end to end measure we should have an active process
> that periodically injects sentinel values at commit time adjacent to the
> WALedits carrying application data at the source and records when they are
> finally processed at the sink, using a timestamp embedded in the sentinel to
> measure true end to end latency for the adjacent commit. This could be done
> for a configurable (and small) percentage of commits so would give a
> probabilistic measure with confidence controlled by sample rate. It should be
> done this way rather than by passively sampling cell timestamps because cell
> timestamps can be set by the user and may not correspond to wall clock time.
> We could introduce a new type of synthetic WALedit, a new global metric, and
> because the adjacent commit from which we build the sentinel contains table
> information we could track that too and add a per table metric.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
