[jira] [Commented] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[ https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332279#comment-15332279 ] churro morales commented on HBASE-15974: I think he is referring to cluster shutdown. How do we ensure that table holding the replication information gets shut down last such that we don't miss any logs to replicate. > Create a ReplicationQueuesClientHBaseImpl > - > > Key: HBASE-15974 > URL: https://issues.apache.org/jira/browse/HBASE-15974 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Fix For: 2.0.0 > > Attachments: HBASE-15974.patch > > > Currently ReplicationQueuesClient utilizes a ZooKeeper implementation > ReplicationQueuesClientZkImpl that attempts to read from the ZNode where > ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase > implementation for ReplicationQueuesClient. > The review is posted at https://reviews.apache.org/r/48521/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[ https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332169#comment-15332169 ] Elliott Clark commented on HBASE-15974: --- +1 > Create a ReplicationQueuesClientHBaseImpl > - > > Key: HBASE-15974 > URL: https://issues.apache.org/jira/browse/HBASE-15974 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15974.patch > > > Currently ReplicationQueuesClient utilizes a ZooKeeper implementation > ReplicationQueuesClientZkImpl that attempts to read from the ZNode where > ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase > implementation for ReplicationQueuesClient. > The review is posted at https://reviews.apache.org/r/48521/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[ https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332168#comment-15332168 ] Elliott Clark commented on HBASE-15974: --- The region should be re-assigned in seconds if a regionserver fails. > Create a ReplicationQueuesClientHBaseImpl > - > > Key: HBASE-15974 > URL: https://issues.apache.org/jira/browse/HBASE-15974 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15974.patch > > > Currently ReplicationQueuesClient utilizes a ZooKeeper implementation > ReplicationQueuesClientZkImpl that attempts to read from the ZNode where > ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase > implementation for ReplicationQueuesClient. > The review is posted at https://reviews.apache.org/r/48521/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[ https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331302#comment-15331302 ] Vincent Poon commented on HBASE-15974: -- I see. What about server shutdown? If RS hosting the Replication shuts down first, all other servers wait for 2 hours and then abort? > Create a ReplicationQueuesClientHBaseImpl > - > > Key: HBASE-15974 > URL: https://issues.apache.org/jira/browse/HBASE-15974 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15974.patch > > > Currently ReplicationQueuesClient utilizes a ZooKeeper implementation > ReplicationQueuesClientZkImpl that attempts to read from the ZNode where > ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase > implementation for ReplicationQueuesClient. > The review is posted at https://reviews.apache.org/r/48521/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[ https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330774#comment-15330774 ] Joseph commented on HBASE-15974: Hello Vincent, Yes, I think that this will add an extra dependency on the RS's hosting the Replication Table. During ReplicationSourceManager.init(), when we try to claim orphaned queues, ReplicationQueuesHBaseImpl will try to run a Scanner over the entire Replication Table to locate the orphaned queues. If a RS hosting a region of the Replication Table happens to be down during this time the Scanner operation will fail. We try to combat this situation by setting an extremely long retry period for Replication Table operations: HBASE-15937. As of now operations on the Replication Table have a retry time of 2 hours. If the region is still unavailable after this time, the RS making the request will abort. We are hoping that cluster startup does not take longer than 2 hours. The patch at HBASE-14190 should also help make sure that the Replication Table's initialization is prioritized during cluster startup. > Create a ReplicationQueuesClientHBaseImpl > - > > Key: HBASE-15974 > URL: https://issues.apache.org/jira/browse/HBASE-15974 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15974.patch > > > Currently ReplicationQueuesClient utilizes a ZooKeeper implementation > ReplicationQueuesClientZkImpl that attempts to read from the ZNode where > ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase > implementation for ReplicationQueuesClient. > The review is posted at https://reviews.apache.org/r/48521/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[ https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330578#comment-15330578 ] Vincent Poon commented on HBASE-15974: -- Does this require any change to cluster startup/shutdown order? ZK is always up during startup/shutdown, so the queues in ZK worked fine. Now it seems there's a dependency on the regionserver hosting the region with these queues? So, a rs starts up / shuts down -> rolls logs -> tries to update queue -> tries to write to other RS that may not be up yet (or may be down already) > Create a ReplicationQueuesClientHBaseImpl > - > > Key: HBASE-15974 > URL: https://issues.apache.org/jira/browse/HBASE-15974 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15974.patch > > > Currently ReplicationQueuesClient utilizes a ZooKeeper implementation > ReplicationQueuesClientZkImpl that attempts to read from the ZNode where > ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase > implementation for ReplicationQueuesClient. > The review is posted at https://reviews.apache.org/r/48521/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[
https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325221#comment-15325221
]
Hadoop QA commented on HBASE-15974:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
37s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 17s
{color} | {color:green} hbase-client-jdk1.8.0 with JDK v1.8.0 generated 0 new +
0 unchanged - 5 fixed = 0 total (was 5) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 21s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 47s
{color} | {color:green} hbase-server in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 6s
{color} | {color:green} hbase-client-jdk1.7.0_79 with JDK v1.7.0_79 generated 0
new + 0 unchanged - 5 fixed = 0 total (was 5) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} hbase-client in the patch passed with JDK v1.7.0_79.
{color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} hbase-server in the patch passed with JDK v1.7.0_79.
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 27s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 53s
{color} | {color:green} hbase-client-jdk1.7.0_79 with JDK v1.7.0_79 generated 0
new + 13 unchanged - 1 fixed = 13 total (was 14) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} hbase-client in
[jira] [Commented] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[
https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323679#comment-15323679
]
Hadoop QA commented on HBASE-15974:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 8s {color}
| {color:red} hbase-client-jdk1.8.0 with JDK v1.8.0 generated 4 new + 5
unchanged - 0 fixed = 9 total (was 5) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 0s {color}
| {color:red} hbase-client-jdk1.7.0_79 with JDK v1.7.0_79 generated 4 new + 5
unchanged - 0 fixed = 9 total (was 5) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 28s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 20s
{color} | {color:red} hbase-client generated 3 new + 0 unchanged - 0 fixed = 3
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 4m 0s
{color} | {color:red} hbase-client-jdk1.7.0_79 with JDK v1.7.0_79 generated 1
new + 14 unchanged - 0 fixed = 15 total (was 14) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 59s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 55s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color}
