[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16450: Release Note: New tool to dump existing replication peers, configurations and queues when using HBase Replication. hbase org.apache.hadoop.hbase.replication.regionserver.DumpReplicationQueues The tool provides two flags: --distributed This flag will poll each RS for information about the replication queues being processed on this RS. By default this is not enabled and the information about the replication queues and configuration will be obtained from ZooKeeper. --hdfs When --distributed is used, this flag will attempt to calculate the total size of the WAL files used by the replication queues. Since its possible that multiple peers can be configured this value can be overestimated. was: New tool to dump existing replication peers, configurations and queues when using HBase Replication. The tool provides two flags: --distributed This flag will poll each RS for information about the replication queues being processed on this RS. By default this is not enabled and the information about the replication queues and configuration will be obtained from ZooKeeper. --hdfs When --distributed is used, this flag will attempt to calculate the total size of the WAL files used by the replication queues. Since its possible that multiple peers can be configured this value can be overestimated. > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 1.3.0, 1.1.5, 1.2.2, 2.0.0 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez >Priority: Major > Fix For: 1.3.0, 0.98.24, 2.0.0 > > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.branch-1.002.patch, HBASE-16450.master.001.patch, > HBASE-16450.master.002.patch, HBASE-16450.master.003.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16450: --- Fix Version/s: (was: 1.4.0) > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Fix For: 2.0.0, 1.3.0, 0.98.24 > > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.branch-1.002.patch, HBASE-16450.master.001.patch, > HBASE-16450.master.002.patch, HBASE-16450.master.003.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16450: --- Fix Version/s: 0.98.24 > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.24 > > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.branch-1.002.patch, HBASE-16450.master.001.patch, > HBASE-16450.master.002.patch, HBASE-16450.master.003.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16450: -- Fix Version/s: 1.4.0 > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.branch-1.002.patch, HBASE-16450.master.001.patch, > HBASE-16450.master.002.patch, HBASE-16450.master.003.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.branch-1.002.patch, HBASE-16450.master.001.patch, > HBASE-16450.master.002.patch, HBASE-16450.master.003.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Release Note: New tool to dump existing replication peers, configurations and queues when using HBase Replication. The tool provides two flags: --distributed This flag will poll each RS for information about the replication queues being processed on this RS. By default this is not enabled and the information about the replication queues and configuration will be obtained from ZooKeeper. --hdfs When --distributed is used, this flag will attempt to calculate the total size of the WAL files used by the replication queues. Since its possible that multiple peers can be configured this value can be overestimated. > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.branch-1.002.patch, HBASE-16450.master.001.patch, > HBASE-16450.master.002.patch, HBASE-16450.master.003.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Fix Version/s: 1.3.0 2.0.0 > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.branch-1.002.patch, HBASE-16450.master.001.patch, > HBASE-16450.master.002.patch, HBASE-16450.master.003.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Attachment: HBASE-16450.branch-1.002.patch branch-1 patch attached, just some minor fixes due delta of methods between master and branch-1. Thanks [~mbertozzi] > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.branch-1.002.patch, HBASE-16450.master.001.patch, > HBASE-16450.master.002.patch, HBASE-16450.master.003.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Attachment: HBASE-16450.master.003.patch New patch with changes suggested by [~mbertozzi] > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.master.001.patch, HBASE-16450.master.002.patch, > HBASE-16450.master.003.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[
https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Esteban Gutierrez updated HBASE-16450:
--
Attachment: HBASE-16450.master.002.patch
attaching new patch that can be called externally from the hbase shell. (needs
another JIRA for that work)
[~stack] the output from the tool is something like this:
{code}
2016-08-22 16:29:38,930 INFO [main] zookeeper.ZooKeeper: Initiating client
connection, connectString=localhost:2181 sessionTimeout=9
watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@f51d56f
2016-08-22 16:29:38,931 INFO [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Opening socket connection to server
localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL
(unknown error)
2016-08-22 16:29:38,931 INFO [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Socket connection established to
localhost/0:0:0:0:0:0:0:1:2181, initiating session
2016-08-22 16:29:38,932 INFO [main-SendThread(localhost:2181)]
zookeeper.ClientCnxn: Session establishment complete on server
localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x156b3d6749f00ad, negotiated
timeout = 4
2016-08-22 16:29:38,938 INFO [main] regionserver.DumpReplicationQueues: Our
Quorum: localhost:2181
2016-08-22 16:29:39,087 INFO [main] regionserver.DumpReplicationQueues:
Replicated Tables: [{replicationType=1, columnFamilyName=c, tableName=x}]
2016-08-22 16:29:39,091 INFO [main] regionserver.DumpReplicationQueues: Peer:
nothing
State: ENABLED
Cluster Name: clusterKey=localhost:2181:/hbase,replicationEndpointImpl=null,
Peer Table CFs: null
2016-08-22 16:29:39,091 INFO [main] regionserver.DumpReplicationQueues: Found
[--distributed], will poll each RegionServer.
2016-08-22 16:29:39,123 INFO [main] replication.ReplicationPeersZKImpl: Added
new peer cluster=localhost:2181:/hbase
2016-08-22 16:29:39,137 INFO [main] regionserver.DumpReplicationQueues:
Dumping replication queue info for RegionServer:
[172.16.1.251,51236,1471895993290]
PeerID: nothing
Recovered: false
No dead RegionServers found in this queue.
Was deleted: false
Number of WALs in replication queue: 1
Replication position for
172.16.1.251%2C51236%2C1471895993290.1471906796781: 0 (not started or nothing
to replicate)
Total size of WALs on HDFS: 83
{code}
> Shell tool to dump replication queues
> --
>
> Key: HBASE-16450
> URL: https://issues.apache.org/jira/browse/HBASE-16450
> Project: HBase
> Issue Type: Improvement
> Components: Operability, Replication
>Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
> Attachments: HBASE-16450.branch-1.001.patch,
> HBASE-16450.master.001.patch, HBASE-16450.master.002.patch
>
>
> Currently there is no way to dump list of the configured queues and the
> replication queues when replication is enabled. Unfortunately the HBase
> master only offers an option to dump the whole content of the znodes but not
> details on the queues being processed on each RS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16450: -- Component/s: Operability > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.master.001.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Attachment: HBASE-16450.master.001.patch HBASE-16450.branch-1.001.patch > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.master.001.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Attachment: (was: HBASE-16450.v1.001.patch) > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.master.001.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Status: Open (was: Patch Available) > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 1.2.2, 1.1.5, 2.0.0, 1.3.0 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.master.001.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Status: Patch Available (was: Open) > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 1.2.2, 1.1.5, 2.0.0, 1.3.0 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.master.001.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Attachment: (was: HBASE-16450.master.001.patch) > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.master.001.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Status: Patch Available (was: Open) > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 1.2.2, 1.1.5, 2.0.0, 1.3.0 >Reporter: Esteban Gutierrez > Attachments: HBASE-16450.master.001.patch, HBASE-16450.v1.001.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-16450: -- Attachment: HBASE-16450.v1.001.patch HBASE-16450.master.001.patch > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez > Attachments: HBASE-16450.master.001.patch, HBASE-16450.v1.001.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
