virajjasani commented on a change in pull request #1894: URL: https://github.com/apache/hbase/pull/1894#discussion_r440367737
########## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ########## @@ -2629,6 +2629,91 @@ You can use the HBase Shell command `status 'replication'` to monitor the replic * `status 'replication', 'source'` -- prints the status for each replication source, sorted by hostname. * `status 'replication', 'sink'` -- prints the status for each replication sink, sorted by hostname. +==== Understanding the output + +The command output will vary according to the state of replication. For example right after a restart +and if destination peer is not reachable, no replication source threads would be running, +so no metrics would get displayed: + +---- +hbase01.home: +SOURCE: PeerID=1 +Normal Queue: 1 +No Reader/Shipper threads runnning yet. +SINK: TimeStampStarted=1591985197350, Waiting for OPs... +---- + +Under normal circumstances, a healthy, active-active replication deployment would +show the following: + +---- + hbase01.home: + SOURCE: PeerID=1 + Normal Queue: 1 + AgeOfLastShippedOp=0, TimeStampOfLastShippedOp=Fri Jun 12 18:49:23 BST 2020, SizeOfLogQueue=1, EditsReadFromLogQueue=1, OpsShippedToTarget=1, TimeStampOfNextToReplicate=Fri Jun 12 18:49:23 BST 2020, Replication Lag=0 + SINK: TimeStampStarted=1591983663458, AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Fri Jun 12 18:57:18 BST 2020 Review comment: Oh no, my bad. I know `ageOfLastShipped` metric, I just took it's example to state that we can't get similar metrics from destination cluster anyways. My question was, do we really have any metric (from replication viewpoint) available to us from destination cluster? (I don't know of any such metric so far, I hope we can't know), like cluster A knows it's own `ageOfLastShipped` and `ageOfLastApplied` but does it know cluster B's `ageOfLastShipped` and `ageOfLastApplied` if both clusters are each other's pairs? And by this command output, I was thinking what if we could display both clusters' metrics together (in case of active-active), but that might not be possible (and might not even be worth spending time) Something like this could be really fancy but nothing necessary (hbase01.home belongs to cluster A and hbase02.home belongs to cluster B, only if we have 2 way replication setup): ``` hbase01.home: SOURCE: PeerID=1 Normal Queue: 1 AgeOfLastShippedOp=0, TimeStampOfLastShippedOp=Fri Jun 12 18:49:23 BST 2020, SizeOfLogQueue=1, EditsReadFromLogQueue=1, OpsShippedToTarget=1, TimeStampOfNextToReplicate=Fri Jun 12 18:49:23 BST 2020, Replication Lag=0 SINK: TimeStampStarted=1591983663458, AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Fri Jun 12 18:57:18 BST 2020 hbase02.home: SOURCE: PeerID=1 Normal Queue: 1 AgeOfLastShippedOp=0, TimeStampOfLastShippedOp=Fri Jun 12 18:49:23 BST 2020, SizeOfLogQueue=1, EditsReadFromLogQueue=1, OpsShippedToTarget=1, TimeStampOfNextToReplicate=Fri Jun 12 18:49:23 BST 2020, Replication Lag=0 SINK: TimeStampStarted=1591983663458, AgeOfLastAppliedOp=0, TimeStampsOfLastAppliedOp=Fri Jun 12 18:57:18 BST 2020 ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org