[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375498#comment-16375498 ] Hudson commented on HBASE-20054: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4640 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4640/]) HBASE-20054 Forward port HBASE-18282 ReplicationLogCleaner can delete (tedyu: rev 9a7645360bad99cc0dc42ae73f6dc779cbbdf490) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.branch-2.v2.txt, 20054.v1.txt, 20054.v2.txt, > 20054.v3.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the new test to master branch. > After discussion, the retry from #1 above is going to be dropped - we abort > directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[
https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375268#comment-16375268
]
Hadoop QA commented on HBASE-20054:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
47s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
44s{color} | {color:red} hbase-server in branch-2 has 22 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
30s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}113m
51s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}151m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-20054 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911847/20054.branch-2.v2.txt
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8111e6ff2c82 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 4ddfdaffdc |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
|
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[
https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375264#comment-16375264
]
Hadoop QA commented on HBASE-20054:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
47s{color} | {color:red} hbase-server in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
56s{color} | {color:red} hbase-server: The patch generated 3 new + 1 unchanged
- 0 fixed = 4 total (was 1) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}106m
28s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}144m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20054 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911845/20054.v2.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b6c90ce6cc0a 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ad5cd50dfc |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| findbugs |
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375166#comment-16375166 ] Ben Lau commented on HBASE-20054: - Patches look good to me +1. > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.branch-2.v2.txt, 20054.v1.txt, 20054.v2.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the new test to master branch. > After discussion, the retry from #1 above is going to be dropped - we abort > directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375161#comment-16375161 ] Ben Lau commented on HBASE-20054: - Sounds good [~yuzhih...@gmail.com]. > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.v1.txt, 20054.v2.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the new test to master branch. > After discussion, the retry from #1 above is going to be dropped - we abort > directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375157#comment-16375157 ] Ted Yu commented on HBASE-20054: w.r.t. the retry in ReplicationSourceManager#adoptAbandonedQueues, I will put up patch for branch-2 which removes the indefinite retry. > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.v1.txt, 20054.v2.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the above to master branch -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375150#comment-16375150 ] Ben Lau commented on HBASE-20054: - [~yuzhih...@gmail.com] I guess it doesn't hurt to forward port the test improvement although the bug doesn't exist in master. For the other change though (queue adoption), I wonder. If master branch decided to abort the server if there is a replication error instead of doing a retry, maybe that is something we should go with instead. In other words, maybe we should backport the server abortion to branch-2, instead forward-porting the RPC retry to master. I also thought about this more and I wonder if indefinitely retrying is a good idea anyway. Maybe it would be better to have a limited # of retries and then abort. But it might be simpler to just abort immediately like in master-branch. Thoughts? > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.v1.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the above to master branch -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[
https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373660#comment-16373660
]
Hadoop QA commented on HBASE-20054:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
1s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
53s{color} | {color:red} hbase-server: The patch generated 3 new + 6 unchanged
- 0 fixed = 9 total (was 6) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
0s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m
58s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
1s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}105m
43s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}132m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20054 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12911611/20054.v1.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 056938704dfb 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d68f697f39 |
| maven |
[jira] [Commented] (HBASE-20054) Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException
[ https://issues.apache.org/jira/browse/HBASE-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373430#comment-16373430 ] Ted Yu commented on HBASE-20054: [~benlau]: Mind taking a look ? > Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet > replicated in case of KeeperException > > > Key: HBASE-20054 > URL: https://issues.apache.org/jira/browse/HBASE-20054 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0 > > Attachments: 20054.v1.txt > > > Some changes from HBASE-18282 were missing in master branch: > * The retry in ReplicationSourceManager#adoptAbandonedQueues > * new tests in TestLogsCleaner > This issue is to forward port the above to master branch -- This message was sent by Atlassian JIRA (v7.6.3#76005)
