[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527082#comment-16527082
]
Hadoop QA commented on HBASE-19753:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-19753 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905728/0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13446/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Miscellany of fixes for hbase-zookeeper tests to make them more robust
> --
>
> Key: HBASE-19753
> URL: https://issues.apache.org/jira/browse/HBASE-19753
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Attachments:
> 0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch,
> HBASE-19753.branch-2.001.patch, HBASE-19753.branch-2.002.patch,
> HBASE-19753.branch-2.003.patch, HBASE-19753.branch-2.004.patch,
> HBASE-19753.branch-2.005.patch, HBASE-19753.branch-2.006.patch,
> HBASE-19753.branch-2.007.patch, HBASE-19753.branch-2.008.patch, keepalive.diff
>
>
> On my cluster which slows zk, tests hbase-zookeeper rarely all pass.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527081#comment-16527081 ] Mike Drob commented on HBASE-19753: --- [~stack] - can we resolve this and open a new issue for the keepalive work? > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Major > Attachments: > 0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch, > HBASE-19753.branch-2.001.patch, HBASE-19753.branch-2.002.patch, > HBASE-19753.branch-2.003.patch, HBASE-19753.branch-2.004.patch, > HBASE-19753.branch-2.005.patch, HBASE-19753.branch-2.006.patch, > HBASE-19753.branch-2.007.patch, HBASE-19753.branch-2.008.patch, keepalive.diff > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323349#comment-16323349 ] Hudson commented on HBASE-19753: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4384 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4384/]) HBASE-19753 Miscellany of fixes for hbase-zookeeper tests to make them (stack: rev a7f9668c339eefecc58f25263c6e7809063a75f2) * (edit) hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKNodeTracker.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ReadOnlyZKClient.java * (edit) hbase-zookeeper/pom.xml * (edit) hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKMainServer.java * (edit) hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKMainServer.java * (edit) hbase-zookeeper/src/test/resources/log4j.properties * (add) hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperHelper.java * (edit) hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestReadOnlyZKClient.java > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: > 0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch, > HBASE-19753.branch-2.001.patch, HBASE-19753.branch-2.002.patch, > HBASE-19753.branch-2.003.patch, HBASE-19753.branch-2.004.patch, > HBASE-19753.branch-2.005.patch, HBASE-19753.branch-2.006.patch, > HBASE-19753.branch-2.007.patch, HBASE-19753.branch-2.008.patch, keepalive.diff > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322816#comment-16322816
]
Hadoop QA commented on HBASE-19753:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-19753 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.6.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905728/0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11024/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Miscellany of fixes for hbase-zookeeper tests to make them more robust
> --
>
> Key: HBASE-19753
> URL: https://issues.apache.org/jira/browse/HBASE-19753
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Attachments:
> 0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch,
> HBASE-19753.branch-2.001.patch, HBASE-19753.branch-2.002.patch,
> HBASE-19753.branch-2.003.patch, HBASE-19753.branch-2.004.patch,
> HBASE-19753.branch-2.005.patch, HBASE-19753.branch-2.006.patch,
> HBASE-19753.branch-2.007.patch, HBASE-19753.branch-2.008.patch, keepalive.diff
>
>
> On my cluster which slows zk, tests hbase-zookeeper rarely all pass.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322699#comment-16322699 ] stack commented on HBASE-19753: --- Ok. Compound patch is hanging on my dodgy hardware... its a hard hang so I'm going to punt on bundling the keepalive for now. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch, HBASE-19753.branch-2.007.patch, > HBASE-19753.branch-2.008.patch, keepalive.diff > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322568#comment-16322568
]
Hadoop QA commented on HBASE-19753:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
17s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
18s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
31s{color} | {color:red} hbase-client: The patch generated 4 new + 0 unchanged
- 0 fixed = 4 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 47s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
9s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
43s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905702/HBASE-19753.branch-2.008.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile xml |
| uname | Linux 39b4a165936b 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 814d08a2d6 |
| maven | version: Apache
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322505#comment-16322505 ] stack commented on HBASE-19753: --- Did some quick tests this morning. Replacing the wait-on-connect with the keepalive patch had hbase-zookeeper large tests timeout occasionally. I didn't dig too much. The keepalive looks like we need it. .008 wraps it in. I ran some tests this morning on .007, no timeout addendum, and got failures like below [ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 159.169 s <<< FAILURE! - in org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient [ERROR] testSessionExpire(org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient) Time elapsed: 110.726 s <<< ERROR! java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /test at org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient.testSessionExpire(TestReadOnlyZKClient.java:140) Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /test [ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 204.377 s <<< FAILURE! - in org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient [ERROR] testSessionExpire(org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient) Time elapsed: 110.708 s <<< ERROR! java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /test at org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient.testSessionExpire(TestReadOnlyZKClient.java:140) Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /test Let me try this combined .008. Running it on cluster machines too though am less and less inclined to believe results out of these machines. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch, HBASE-19753.branch-2.007.patch, > HBASE-19753.branch-2.008.patch, keepalive.diff > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322200#comment-16322200
]
Hadoop QA commented on HBASE-19753:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905669/keepalive.diff |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fbf45f1469eb 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 71a1192d67 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11017/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321986#comment-16321986
]
Hadoop QA commented on HBASE-19753:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
34s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
59s{color} | {color:red} hbase-client: The patch generated 3 new + 0 unchanged
- 0 fixed = 3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
26s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 27s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
50s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
25s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
0s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905641/HBASE-19753.branch-2.007.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile xml |
| uname | Linux d7cb01f46634 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 814d08a2d6 |
| maven | version: Apache
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321975#comment-16321975 ] stack commented on HBASE-19753: --- bq. Then how about modifying the UTs only as I described above? Increasing the keep alive time, so that we do not close the zookeeper too frequently. And also increasing the wait time in tests. I tried it. It'd still fail. The wait for connection in ROZKClient seems effective; it makes the real difference. Thanks for the review on v7. Let me wait on qa run to be sure. Thanks for help [~Apache9] > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch, HBASE-19753.branch-2.007.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321907#comment-16321907 ] Duo Zhang commented on HBASE-19753: --- The approach in v7 is fine. +1. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch, HBASE-19753.branch-2.007.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321898#comment-16321898 ] stack commented on HBASE-19753: --- .007 Change how we wait on connection in ROZKClient. Don't wait so long either. [~Apache9] would you +1 this one? Testing on a few machines, it keeps the robustness of the old patch and TestAdmin2 passes. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch, HBASE-19753.branch-2.007.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321897#comment-16321897 ] Duo Zhang commented on HBASE-19753: --- Then how about modifying the UTs only as I described above? Increasing the keep alive time, so that we do not close the zookeeper too frequently. And also increasing the wait time in tests. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch, HBASE-19753.branch-2.007.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321887#comment-16321887 ] stack commented on HBASE-19753: --- I can't commit this patch. As is it makes TestAdmin2 fail. It has a test that has us connect to a non-existent cluster. We just hang for ever trying to connect instead of giving up promptly. Here is an alternative. It waits a while on creation of zk in ROZKClient. If we fail to connect, just carry-on. The first usage will fail if we are not connected. I tried doing w/o the change in ROZKClient but tests in hbase-zk started failing w/o it. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321725#comment-16321725
]
Hadoop QA commented on HBASE-19753:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
31s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} hbase-client: The patch generated 1 new + 0 unchanged
- 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
56s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 22s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
37s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905609/HBASE-19753.branch-2.006.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile xml |
| uname | Linux 60d8924e3e15 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 0885fe161a |
| maven | version: Apache
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321703#comment-16321703 ] Duo Zhang commented on HBASE-19753: --- +1. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321702#comment-16321702 ] stack commented on HBASE-19753: --- Yeah, this patch has been running last couple of hours and all zk pass now on my dodgy machines. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321699#comment-16321699 ] stack commented on HBASE-19753: --- .006 fixes checkstyle. Will push this soon. Shout if have any other input [~Apache9] Thanks for the help. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321633#comment-16321633
]
Hadoop QA commented on HBASE-19753:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
28s{color} | {color:red} hbase-client: The patch generated 3 new + 0 unchanged
- 0 fixed = 3 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hbase-zookeeper: The patch generated 1 new + 3
unchanged - 0 fixed = 4 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 10s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
46s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905596/HBASE-19753.branch-2.005.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile xml |
| uname | Linux 536e23cc4955 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool |
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321603#comment-16321603 ] stack commented on HBASE-19753: --- .004 implements the [~Apache9] suggestion removing dupe'd code. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321598#comment-16321598
]
Hadoop QA commented on HBASE-19753:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
2s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
15s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 0s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
47s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
36s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905592/HBASE-19753.branch-2.003.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile xml |
| uname | Linux 5ec3f371abc1 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / d524768528 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d;
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321592#comment-16321592 ] stack commented on HBASE-19753: --- Running .003 on three machines in a loop, TestReadOnlyZKClient passes reliably. Running the hbase-zookeeper suite, they all pass on these machines now too. Let me make a patch w/ a helper. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321561#comment-16321561
]
Hadoop QA commented on HBASE-19753:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
28s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
24s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
11s{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 11s{color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} hbase-client: The patch generated 1 new + 0 unchanged
- 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 1m
30s{color} | {color:red} patch has 68 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
34s{color} | {color:red} The patch causes 68 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
40s{color} | {color:red} The patch causes 68 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
49s{color} | {color:red} The patch causes 68 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
9s{color} | {color:red} hbase-client in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 11s{color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
34s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905589/HBASE-19753.branch-2.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile xml |
| uname | Linux
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321552#comment-16321552 ] Duo Zhang commented on HBASE-19753: --- +1 on the approach of patch v3. But let's introduce a helper method for waiting the zk to be connected? The same code appears several times in the patch. Thanks. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321549#comment-16321549 ] stack commented on HBASE-19753: --- .002 adds wait to getZK in ReadOnlyZKClient. .003 Fixes a type. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321483#comment-16321483 ] Duo Zhang commented on HBASE-19753: --- I think add a wait until connected in getZk could help, the keep alive logic and getZk are in the same thread so it will not close the zk if you are waiting on the connecting. Thanks. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321471#comment-16321471 ] stack commented on HBASE-19753: --- This is on a linux machine: $ more /etc/issue Ubuntu 14.04.4 LTS \n \l Yeah, I need to figure why zk setup is slow -- some network config I need to figure. For now it is turning up some interesting issues. I'm looking at ReadOnlyZKClient#getZk... It returns a new ZooKeeper w/o waiting on connecting. But adding a wait until connected doesn't seem to help... At least in the case I describe above. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321467#comment-16321467
]
Duo Zhang commented on HBASE-19753:
---
{code}
conf.setInt(ReadOnlyZKClient.KEEPALIVE_MILLIS, 3000);
{code}
Up the 3000 here to 1, or even more.
And in testGetAndExists, also up the wait time.
You can try this to see if it works for you [~stack].
Thanks.
> Miscellany of fixes for hbase-zookeeper tests to make them more robust
> --
>
> Key: HBASE-19753
> URL: https://issues.apache.org/jira/browse/HBASE-19753
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-19753.branch-2.001.patch
>
>
> On my cluster which slows zk, tests hbase-zookeeper rarely all pass.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321450#comment-16321450 ] Duo Zhang commented on HBASE-19753: --- The problem is that, the keepalive time is too small, that it will close the connection to zk while we are still trying to connect to zk. The design philosophy here is that the keep alive time is usually much greater than the operation time. The default value is 1 minute. Here we want to test the keep alive mechanism so we set the keep alive time to 3 seconds, which is too small for your environment. You can try up it to see if it helps. And it is really strange that, why a zk connecting can take even more than 3 seconds? There must be something wrong. What's your OS sir? Linux or mac? Thanks. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321416#comment-16321416
]
stack commented on HBASE-19753:
---
Let me dump this info in here in case it helps...
I'm looking at flakies. It seems like TestHCM hangs on occasion because we
can't read clusterid using the ReadOnlyZKClient that is in the ZKAsyncRegistry.
The Master has written it (I have it dump the log when it thinks it wrote
it)... then later I see the ReadOnlyZKClient not being able to find it. The
Master startup fails because we are stuck at this spot. Here is a bit of log...
{code}
2018-01-10 15:43:06,512 INFO [M:0;ve0524:60789] master.HMaster(805): CLUSTERID
SET
2018-01-10 15:43:06,536 INFO [M:0;ve0524:60789]
balancer.BaseLoadBalancer(1037): slop=0.001, tablesOnMaster=false,
systemTablesOnMaster=false
2018-01-10 15:43:06,537 INFO [M:0;ve0524:60789]
balancer.StochasticLoadBalancer(214): Loaded config; maxSteps=100,
stepsPerRegion=800, maxRunningTime=3, isByTable=false, etc.
2018-01-10 15:43:06,538 DEBUG [M:0;ve0524:60789] zookeeper.ZKUtil(355):
master:60789-0x160e26d00200010, quorum=localhost:50723, baseZNode=/hbase Set
watcher on existing znode=/hbase/balancer
2018-01-10 15:43:06,539 DEBUG [M:0;ve0524:60789] zookeeper.ZKUtil(357):
master:60789-0x160e26d00200010, quorum=localhost:50723, baseZNode=/hbase Set
watcher on znode that does not yet exist, /hbase/normalizer
2018-01-10 15:43:06,541 DEBUG [M:0;ve0524:60789] zookeeper.ZKUtil(357):
master:60789-0x160e26d00200010, quorum=localhost:50723, baseZNode=/hbase Set
watcher on znode that does not yet exist, /hbase/switch/split
2018-01-10 15:43:06,541 DEBUG [M:0;ve0524:60789] zookeeper.ZKUtil(357):
master:60789-0x160e26d00200010, quorum=localhost:50723, baseZNode=/hbase Set
watcher on znode that does not yet exist, /hbase/switch/merge
2018-01-10 15:43:06,543 DEBUG [Time-limited test-EventThread]
zookeeper.ZKWatcher(460): regionserver:35280-0x160e26d00200011,
quorum=localhost:50723, baseZNode=/hbase Received ZooKeeper Event,
type=NodeCreated, state=SyncConnected, path=/hbase/running
2018-01-10 15:43:06,543 DEBUG [Time-limited test-EventThread]
zookeeper.ZKWatcher(460): regionserver:57007-0x160e26d00200012,
quorum=localhost:50723, baseZNode=/hbase Received ZooKeeper Event,
type=NodeCreated, state=SyncConnected, path=/hbase/running
2018-01-10 15:43:06,543 DEBUG [Time-limited test-EventThread]
zookeeper.ZKWatcher(460): master:60789-0x160e26d00200010,
quorum=localhost:50723, baseZNode=/hbase Received ZooKeeper Event,
type=NodeCreated, state=SyncConnected, path=/hbase/running
2018-01-10 15:43:06,546 INFO [M:0;ve0524:60789]
zookeeper.ReadOnlyZKClient(129): Start read only zookeeper connection
0x655063fa to localhost:50723, session timeout 9 ms, retries 30, retry
interval 1000 ms, keep alive 6 ms
2018-01-10 15:43:18,316 INFO [ReadOnlyZKClient]
zookeeper.ReadOnlyZKClient(305): 0x1cbc2aa9 no activities for 6 ms, close
active connection. Will reconnect next time when there are new requests.
2018-01-10 15:43:33,249 INFO [ReadOnlyZKClient]
zookeeper.ReadOnlyZKClient(305): 0x7dc94b7b no activities for 6 ms, close
active connection. Will reconnect next time when there are new requests.
Process Thread Dump: Thread dump because Master not active after 3 seconds
{code}
I added threadumping if Master can't start and Master seems stuck here:
{code}
Thread 1646 (M:0;ve0524:60789):
State: WAITING
Blocked count: 74
Waited count: 130
Waiting on java.util.concurrent.CompletableFuture$Signaller@2b7d5aee
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
org.apache.hadoop.hbase.client.ConnectionImplementation.retrieveClusterId(ConnectionImplementation.java:526)
org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:286)
org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:141)
org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:132)
org.apache.hadoop.hbase.client.ConnectionUtils.createShortCircuitConnection(ConnectionUtils.java:185)
org.apache.hadoop.hbase.regionserver.HRegionServer.createClusterConnection(HRegionServer.java:770)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321356#comment-16321356 ] Duo Zhang commented on HBASE-19753: --- Will take a look soon. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321348#comment-16321348 ] stack commented on HBASE-19753: --- This is pretty: 125 2018-01-10 15:23:26,648 WARN [ReadOnlyZKClient-SendThread(user-164-127-0-0.play-internet.pl:51573)] zookeeper.ClientCnxn$SendThread(1108): Client session timed out, have not heard from server in 45043ms for sessionid 0x0 > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321345#comment-16321345
]
stack commented on HBASE-19753:
---
Thanks for taking a look [~Apache9] Here is info.
Here is failure:
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient
[ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 250.187
s <<< FAILURE! - in org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient
[ERROR]
testSessionExpire(org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient) Time
elapsed: 156.005 s <<< ERROR!
java.util.concurrent.ExecutionException:
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode =
ConnectionLoss for /test
at
org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient.testSessionExpire(TestReadOnlyZKClient.java:145)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /test
{code}
119 2018-01-10 15:22:41,604 INFO [main] hbase.ResourceChecker(148): before:
zookeeper.TestReadOnlyZKClient#testSessionExpire Thread=13,
OpenFileDescriptor=112, MaxFileDescriptor=32000, SystemLoadAverage=67,
ProcessCount=833, AvailableMemoryMB=12065
120 2018-01-10 15:22:41,605 INFO [ReadOnlyZKClient] zookeeper.ZooKeeper(438):
Initiating client connection, connectString=localhost:51573
sessionTimeout=9
watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$11/1499902826@57a14636
121 2018-01-10 15:22:41,607 INFO
[ReadOnlyZKClient-SendThread(user-164-127-0-0.play-internet.pl:51573)]
zookeeper.ClientCnxn$SendThread(1032): Opening socket connection to server
user-164-127-0-0.play-internet.pl/164.127.0.0:51573. Will not attempt to
authenticate using SASL (unknown error)
122 2018-01-10 15:22:44,606 INFO [ReadOnlyZKClient]
zookeeper.ReadOnlyZKClient(305): 0x57f23557 no activities for 3000 ms, close
active connection. Will reconnect next time when there are new requests.
123 2018-01-10 15:22:44,606 DEBUG [ReadOnlyZKClient] zookeeper.ZooKeeper(673):
Closing session: 0x0
124 2018-01-10 15:22:44,606 DEBUG [ReadOnlyZKClient]
zookeeper.ClientCnxn(1370): Closing client for session: 0x0
125 2018-01-10 15:23:26,648 WARN
[ReadOnlyZKClient-SendThread(user-164-127-0-0.play-internet.pl:51573)]
zookeeper.ClientCnxn$SendThread(1108): Client session timed out, have not heard
from server in 45043ms for sessionid 0x0
126 2018-01-10 15:23:26,648 DEBUG
[ReadOnlyZKClient-SendThread(user-164-127-0-0.play-internet.pl:51573)]
zookeeper.ClientCnxn$SendThread(1146): An exception was thrown while closing
send thread for session 0x0 : Client session timed out, have not heard from
server in 45043ms for sessionid 0x0
127 2018-01-10 15:23:26,649 DEBUG
[ReadOnlyZKClient-SendThread(user-164-127-0-0.play-internet.pl:51573)]
zookeeper.ClientCnxnSocketNIO(203): Ignoring exception during shutdown input
128 java.net.SocketException: Socket is not connected
129 at sun.nio.ch.Net.translateToSocketException(Net.java:123)
130 at sun.nio.ch.Net.translateException(Net.java:157)
131 at sun.nio.ch.Net.translateException(Net.java:163)
132 at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:401)
133 at
org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:200)
134 at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246)
135 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1182)
136 Caused by: java.nio.channels.NotYetConnectedException
137 at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:782)
138 at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:399)
139 ... 3 more
140 2018-01-10 15:23:26,653 DEBUG
[ReadOnlyZKClient-SendThread(user-164-127-0-0.play-internet.pl:51573)]
zookeeper.ClientCnxnSocketNIO(210): Ignoring exception during shutdown output
141 java.net.SocketException: Socket is not connected
142 at sun.nio.ch.Net.translateToSocketException(Net.java:123)
143 at sun.nio.ch.Net.translateException(Net.java:157)
144 at sun.nio.ch.Net.translateException(Net.java:163)
145 at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:409)
146 at
org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:207)
147 at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246)
148 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1182)
149 Caused by: java.nio.channels.NotYetConnectedException
150 at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:799)
151 at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:407)
152 ... 3 more
153 2018-01-10 15:23:26,754 DEBUG [ReadOnlyZKClient]
zookeeper.ClientCnxn(1354): Disconnecting client for session: 0x0
154 2018-01-10
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321300#comment-16321300
]
Hadoop QA commented on HBASE-19753:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
17s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
13s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
34s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 1s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905556/HBASE-19753.branch-2.001.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile findbugs hbaseanti checkstyle |
| uname | Linux 0a33f57e072e 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / d524768528 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/10993/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/10993/testReport/ |
| modules | C: hbase-zookeeper U: hbase-zookeeper |
| Console output |
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321282#comment-16321282 ] Duo Zhang commented on HBASE-19753: --- I'm OK with the waiting until connected modification and increasing session timeout. The only concern is that we catch and ignore a exception in TestZKReadOnlyClient... > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321273#comment-16321273 ] Duo Zhang commented on HBASE-19753: --- Mind uploading the whole surefire output file for TestReadOnlyZKClient? It is a bit strange, the retry number is 30 IIRC and the retry interval is 1 second, but we fail in 5.84 seconds... Thanks. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321264#comment-16321264 ] stack commented on HBASE-19753: --- [~Apache9] you ok w/ my committing this. Got a good test run over in HBASE-19694 (I'd posted the patch here there by mistake). > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321257#comment-16321257 ] stack commented on HBASE-19753: --- Whoops. I'd actually put aside errors and meant to add them here. [~Apache9] Here are some of the failures I'd see: Here is what I'd get if we didn't wait on zk to connect: [INFO] Results: [INFO] [ERROR] Errors: [ERROR] org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient.org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient [ERROR] Run 1: TestReadOnlyZKClient.setUp:74 ยป ConnectionLoss KeeperErrorCode = ConnectionLos... [ERROR] Run 2: TestReadOnlyZKClient.tearDown:91 NullPointer [INFO] Same was happening in TestZKNodeTracker This is the error I'd get sometimes after session was closed: [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.579 s - in org.apache.hadoop.hbase.zookeeper.TestZKMulti [INFO] Running org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient [ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.84 s <<< FAILURE! - in org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient [ERROR] testSessionExpire(org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient) Time elapsed: 2.431 s <<< ERROR! java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /test at org.apache.hadoop.hbase.zookeeper.TestReadOnlyZKClient.testSessionExpire(TestReadOnlyZKClient.java:145) Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /test Other failures happened because default TestZKMainServer timeout was way too short at 1second. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321225#comment-16321225 ] Duo Zhang commented on HBASE-19753: --- Could you please post some logs about your failures sir? Where is the actual slow down happens? For zk server or zk client? > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
