[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14352401#comment-14352401 ] Hudson commented on HBASE-13135: FAILURE: Integrated in HBase-TRUNK #6222 (See [https://builds.apache.org/job/HBase-TRUNK/6222/]) HBASE-13135 Move replication ops mgmt stuff from Javadoc to Ref Guide (mstanleyjones: rev aed88cf382bf2a084e45c18c15dbf5026b22eba1) * hbase-server/src/main/javadoc/org/apache/hadoop/hbase/replication/package.html * src/main/asciidoc/_chapters/ops_mgt.adoc > Move replication ops mgmt stuff from Javadoc to Ref Guide > - > > Key: HBASE-13135 > URL: https://issues.apache.org/jira/browse/HBASE-13135 > Project: HBase > Issue Type: Bug > Components: documentation, Replication >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 2.0.0 > > Attachments: HBASE-13135-v1.patch, HBASE-13135-v2.patch, > HBASE-13135.patch > > > As per discussion with [~jmhsieh] and [~saint@gmail.com] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349168#comment-14349168
]
Jean-Daniel Cryans commented on HBASE-13135:
Good stuff Misty, a few more things while we're here.
I think we can remove this line since it's on by default, maybe say that the
operator should check it wasn't disabled:
bq. On the source cluster, enable replication by setting `hbase.replication` to
`true` in _hbase-site.xml_.
The following can be removed, we don't print this out at least since the
endpoint changes:
{quote}
Considering 1 rs, with ratio 0.1
Getting 1 rs from peer cluster # 0
Choosing peer 10.10.1.49:62020
{quote}
This is a more reliable thing to point out instead, coming from
ReplicationSource:
bq. LOG.info("Replicating "+clusterId + " -> " + peerClusterId);
> Move replication ops mgmt stuff from Javadoc to Ref Guide
> -
>
> Key: HBASE-13135
> URL: https://issues.apache.org/jira/browse/HBASE-13135
> Project: HBase
> Issue Type: Bug
> Components: documentation, Replication
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-13135-v1.patch, HBASE-13135-v2.patch,
> HBASE-13135.patch
>
>
> As per discussion with [~jmhsieh] and [~saint@gmail.com]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348446#comment-14348446
]
Hadoop QA commented on HBASE-13135:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12702736/HBASE-13135-v2.patch
against master branch at commit 74837d9b1f13b42415395e08ff168aeed0c21676.
ATTACHMENT ID: 12702736
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+Cluster replication documentation has been moved to the
link:http://hbase.apache.org/book.html#_cluster_replication[Cluster
Replication] section of the link:http://hbase.apache.org/book.html[Apache HBase
Reference Guide].
+. If both clusters use the same ZooKeeper cluster, you must use a different
`zookeeper.znode.parent`, because they cannot write in the same folder.
+. On the source cluster, enable replication by setting `hbase.replication` to
`true` in _hbase-site.xml_.
+. On the source cluster, in HBase Shell, add the destination cluster as a
peer, using the `add_peer` command.
+ * CLUSTER_KEY: composed using the following template, with appropriate
place-holders:
`hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent`
+ Disable a replication relationship. HBase will no longer send edits to that
peer cluster, but it still keeps track of all the new WALs that it will need to
replicate if and when it is re-enabled.
+ Disable and remove a replication relationship. HBase will no longer send
edits to that peer cluster or keep track of WALs.
+The `VerifyReplication` MapReduce job, which is included in HBase, performs a
systematic comparison of replicated data between two different clusters. Run
the VerifyReplication job on the master cluster, supplying it with the peer ID
and table name to use for validation. You can limit the verification further by
specifying a time range or specific families. The job's short name is
`verifyrep`. To run the job, use a command like the following:
+$ HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath`
"${HADOOP_HOME}/bin/hadoop" jar "${HBASE_HOME}/hbase-server-VERSION.jar"
verifyrep --starttime= --stoptime= --families=
+The `VerifyReplication` command prints out `GOODROWS` and `BADROWS` counters
to indicate rows that did and did not replicate correctly.
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13098//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13098//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13098//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13098//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13098//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13098//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13098//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13098//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13098//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs w
[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348298#comment-14348298 ] Ashish Singhi commented on HBASE-13135: --- lgtm > Move replication ops mgmt stuff from Javadoc to Ref Guide > - > > Key: HBASE-13135 > URL: https://issues.apache.org/jira/browse/HBASE-13135 > Project: HBase > Issue Type: Bug > Components: documentation, Replication >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-13135-v1.patch, HBASE-13135-v2.patch, > HBASE-13135.patch > > > As per discussion with [~jmhsieh] and [~saint@gmail.com] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348221#comment-14348221
]
Ashish Singhi commented on HBASE-13135:
---
bq. Are you sure? That means command substitution in Bash
Yes, this is what we want to achieve.
Here is the output of the command I tried.
{noformat}
ashish@host:~> HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath`
${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/lib/hbase-server-2.0.0-SNAPSHOT.jar
verifyrep
Usage: verifyrep [--starttime=X] [--stoptime=Y] [--families=A]
Options:
starttimebeginning of the time range
without endtime means from starttime to forever
endtime end of the time range
versions number of cell versions to verify
families comma-separated list of families to copy
Args:
peerid Id of the peer used for verification, must match the one given
for replication
tablenameName of the table to verify
Examples:
To verify the data replicated from TestTable for a 1 hour window with peer #5
$ bin/hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication
--starttime=1265875194289 --endtime=1265878794289 5 TestTable
{noformat}
Here is the code for {{117.11.2. ImportTsv Example}} section
{noformat}
For ImportTsv to use this imput file, the command line needs to look like this:
HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` ${HADOOP_HOME}/bin/hadoop
jar ${HBASE_HOME}/hbase-server-VERSION.jar importtsv
-Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2
-Dimporttsv.bulk.output=hdfs://storefileoutput datatsv hdfs://inputfile
{noformat}
Thanks
> Move replication ops mgmt stuff from Javadoc to Ref Guide
> -
>
> Key: HBASE-13135
> URL: https://issues.apache.org/jira/browse/HBASE-13135
> Project: HBase
> Issue Type: Bug
> Components: documentation, Replication
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-13135-v1.patch, HBASE-13135.patch
>
>
> As per discussion with [~jmhsieh] and [~saint@gmail.com]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348207#comment-14348207 ] Misty Stanley-Jones commented on HBASE-13135: - Are you sure? That means command substitution in Bash. Will it work to enclose in $() instead of back-ticks, as that is more POSIX compliant and also easier to show in Asciidoc (the backticks turn into mono text). > Move replication ops mgmt stuff from Javadoc to Ref Guide > - > > Key: HBASE-13135 > URL: https://issues.apache.org/jira/browse/HBASE-13135 > Project: HBase > Issue Type: Bug > Components: documentation, Replication >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-13135-v1.patch, HBASE-13135.patch > > > As per discussion with [~jmhsieh] and [~saint@gmail.com] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348185#comment-14348185
]
Ashish Singhi commented on HBASE-13135:
---
bq. "${HBASE_HOME}/bin/hbase classpath"
Should be enclosed within '*`*'. You can refer {{117.11.2. ImportTsv Example}}
section in the book.
> Move replication ops mgmt stuff from Javadoc to Ref Guide
> -
>
> Key: HBASE-13135
> URL: https://issues.apache.org/jira/browse/HBASE-13135
> Project: HBase
> Issue Type: Bug
> Components: documentation, Replication
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-13135-v1.patch, HBASE-13135.patch
>
>
> As per discussion with [~jmhsieh] and [~saint@gmail.com]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343406#comment-14343406 ] Jean-Daniel Cryans commented on HBASE-13135: Looking at the current doc, this line should be changed since at the end it includes a link that points back to the doc we're moving: bq. The following is a simplified procedure for configuring cluster replication. It may not cover every edge case. For more information, see the API documentation for replication. This whole section is very similar to the tail of the "Configuring Cluster Replication" section (merge or skip?): bq. + Verifying Replicated Data > Move replication ops mgmt stuff from Javadoc to Ref Guide > - > > Key: HBASE-13135 > URL: https://issues.apache.org/jira/browse/HBASE-13135 > Project: HBase > Issue Type: Bug > Components: documentation, Replication >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-13135.patch > > > As per discussion with [~jmhsieh] and [~saint@gmail.com] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342831#comment-14342831
]
Ashish Singhi commented on HBASE-13135:
---
bq. +$ hadoop jar /usr/lib/hbase/hbase.jar verifyrep --starttime=
Can we use like this ?
+$ HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath`
${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/hbase-server-VERSION.jar verifyrep
--starttime=
Thanks
> Move replication ops mgmt stuff from Javadoc to Ref Guide
> -
>
> Key: HBASE-13135
> URL: https://issues.apache.org/jira/browse/HBASE-13135
> Project: HBase
> Issue Type: Bug
> Components: documentation, Replication
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-13135.patch
>
>
> As per discussion with [~jmhsieh] and [~saint@gmail.com]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342813#comment-14342813 ] stack commented on HBASE-13135: --- +1 Looks great. Thanks [~misty] > Move replication ops mgmt stuff from Javadoc to Ref Guide > - > > Key: HBASE-13135 > URL: https://issues.apache.org/jira/browse/HBASE-13135 > Project: HBase > Issue Type: Bug > Components: documentation, Replication >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-13135.patch > > > As per discussion with [~jmhsieh] and [~saint@gmail.com] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide
[
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342776#comment-14342776
]
Hadoop QA commented on HBASE-13135:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12701775/HBASE-13135.patch
against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
ATTACHMENT ID: 12701775
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+Cluster replication documentation has been moved to the
link:http://hbase.apache.org/book.html#_cluster_replication[Cluster
Replication] section of the link:http://hbase.apache.org/book.html[Apache HBase
Reference Guide].
+NOTE: This material was previously available in the
link:http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/replication/package-summary.html#requirements[Replication
API documentation].
+Several HBase Shell commands are available to assist you in managing
replication relationships between clusters (also called peers).
+ * CLUSTER_KEY: composed using the following template, with appropriate
place-holders:
`hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent`
+ Disable a replication relationship. HBase will no longer send edits to that
peer cluster, but it still keeps track of all the new WALs that it will need to
replicate if and when it is re-enabled.
+ Disable and remove a replication relationship. HBase will no longer send
edits to that peer cluster or keep track of WALs.
+The `VerifyReplication` MapReduce job, which is included in HBase, performs a
systematic comparison of replicated data between two different clusters. Run
the VerifyReplication job on the master cluster, supplying it with the peer ID
and table name to use for validation. You can limit the verification further by
specifying a time range or specific families. The job's short name is
`verifyrep`. To run the job, use a command like the following:
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Checkstyle Errors
