ZooKeeper_branch34_jdk8 - Build # 1571 - Still Failing
See https://builds.apache.org/job/ZooKeeper_branch34_jdk8/1571/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 43.69 KB...] [junit] Running org.apache.zookeeper.test.RestoreCommittedLogTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.357 sec [junit] Running org.apache.zookeeper.test.SaslAuthDesignatedClientTest [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.847 sec [junit] Running org.apache.zookeeper.test.SaslAuthDesignatedServerTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.798 sec [junit] Running org.apache.zookeeper.test.SaslAuthFailDesignatedClientTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.173 sec [junit] Running org.apache.zookeeper.test.SaslAuthFailNotifyTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.746 sec [junit] Running org.apache.zookeeper.test.SaslAuthFailTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.87 sec [junit] Running org.apache.zookeeper.test.SaslAuthMissingClientConfigTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.683 sec [junit] Running org.apache.zookeeper.test.SaslClientTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec [junit] Running org.apache.zookeeper.test.SessionInvalidationTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.822 sec [junit] Running org.apache.zookeeper.test.SessionTest [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.411 sec [junit] Running org.apache.zookeeper.test.SessionTimeoutTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.091 sec [junit] Running org.apache.zookeeper.test.StandaloneTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.976 sec [junit] Running org.apache.zookeeper.test.StatTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.082 sec [junit] Running org.apache.zookeeper.test.StaticHostProviderTest [junit] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.76 sec [junit] Running org.apache.zookeeper.test.SyncCallTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.706 sec [junit] Running org.apache.zookeeper.test.TruncateTest [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.653 sec [junit] Running org.apache.zookeeper.test.UpgradeTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.02 sec [junit] Running org.apache.zookeeper.test.WatchedEventTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.086 sec [junit] Running org.apache.zookeeper.test.WatcherFuncTest [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.692 sec [junit] Running org.apache.zookeeper.test.WatcherTest [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.615 sec [junit] Running org.apache.zookeeper.test.ZkDatabaseCorruptionTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.474 sec [junit] Running org.apache.zookeeper.test.ZooKeeperQuotaTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.827 sec [junit] Running org.apache.jute.BinaryInputArchiveTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.077 sec fail.build.on.test.failure: BUILD FAILED /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/build.xml:1467: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk8/build.xml:1470: Tests failed! Total time: 42 minutes 54 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## 18 tests failed. FAILED: org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.testValidCredentials Error Message: waiting for server 0 being up Stack Trace: junit.framework.AssertionFailedError: waiting for server 0 being up at org.apache.zookeeper.server.quorum.auth.QuorumAuthTestBase.startQuorum(QuorumAuthTestBase.java:75) at org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.testValidCredentials(QuorumKerberosAuthTest.java:114) at
[GitHub] zookeeper issue #665: [ZOOKEEPER-3163] Use session map in the Netty to impro...
Github user maoling commented on the issue: https://github.com/apache/zookeeper/pull/665 @lvfangmin look at the branch3.4 is not consistency with master.Could we create a separate jira to backport this refactor to branch3.4 to keep them consistency? ---
ZooKeeper-trunk - Build # 244 - Failure
See https://builds.apache.org/job/ZooKeeper-trunk/244/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 198.38 KB...] [junit] Running org.apache.zookeeper.test.ServerCnxnTest in thread 3 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.498 sec, Thread: 3, Class: org.apache.zookeeper.test.ServerCnxnTest [junit] Running org.apache.zookeeper.test.SessionInvalidationTest in thread 3 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.957 sec, Thread: 3, Class: org.apache.zookeeper.test.SessionInvalidationTest [junit] Running org.apache.zookeeper.test.SessionTest in thread 3 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 116.315 sec, Thread: 4, Class: org.apache.zookeeper.test.RecoveryTest [junit] Running org.apache.zookeeper.test.SessionTimeoutTest in thread 4 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.387 sec, Thread: 4, Class: org.apache.zookeeper.test.SessionTimeoutTest [junit] Running org.apache.zookeeper.test.SessionTrackerCheckTest in thread 4 [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.086 sec, Thread: 4, Class: org.apache.zookeeper.test.SessionTrackerCheckTest [junit] Running org.apache.zookeeper.test.SessionUpgradeTest in thread 4 [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.157 sec, Thread: 3, Class: org.apache.zookeeper.test.SessionTest [junit] Running org.apache.zookeeper.test.StandaloneTest in thread 3 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.804 sec, Thread: 3, Class: org.apache.zookeeper.test.StandaloneTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 93.432 sec, Thread: 1, Class: org.apache.zookeeper.test.RestoreCommittedLogTest [junit] Running org.apache.zookeeper.test.StatTest in thread 1 [junit] Running org.apache.zookeeper.test.StaticHostProviderTest in thread 3 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.84 sec, Thread: 1, Class: org.apache.zookeeper.test.StatTest [junit] Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.076 sec, Thread: 3, Class: org.apache.zookeeper.test.StaticHostProviderTest [junit] Running org.apache.zookeeper.test.StringUtilTest in thread 1 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 sec, Thread: 1, Class: org.apache.zookeeper.test.StringUtilTest [junit] Running org.apache.zookeeper.test.SyncCallTest in thread 3 [junit] Running org.apache.zookeeper.test.TruncateTest in thread 1 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.038 sec, Thread: 3, Class: org.apache.zookeeper.test.SyncCallTest [junit] Running org.apache.zookeeper.test.WatchEventWhenAutoResetTest in thread 3 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.894 sec, Thread: 4, Class: org.apache.zookeeper.test.SessionUpgradeTest [junit] Running org.apache.zookeeper.test.WatchedEventTest in thread 4 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.143 sec, Thread: 4, Class: org.apache.zookeeper.test.WatchedEventTest [junit] Running org.apache.zookeeper.test.WatcherFuncTest in thread 4 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.02 sec, Thread: 3, Class: org.apache.zookeeper.test.WatchEventWhenAutoResetTest [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.175 sec, Thread: 1, Class: org.apache.zookeeper.test.TruncateTest [junit] Running org.apache.zookeeper.test.WatcherTest in thread 3 [junit] Running org.apache.zookeeper.test.X509AuthTest in thread 1 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.119 sec, Thread: 1, Class: org.apache.zookeeper.test.X509AuthTest [junit] Running org.apache.zookeeper.test.ZkDatabaseCorruptionTest in thread 1 [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.774 sec, Thread: 4, Class: org.apache.zookeeper.test.WatcherFuncTest [junit] Running org.apache.zookeeper.test.ZooKeeperQuotaTest in thread 4 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.558 sec, Thread: 4, Class: org.apache.zookeeper.test.ZooKeeperQuotaTest [junit] Running org.apache.jute.BinaryInputArchiveTest in thread 4 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.071 sec, Thread: 4, Class: org.apache.jute.BinaryInputArchiveTest [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.893 sec, Thread: 1, Class: org.apache.zookeeper.test.ZkDatabaseCorruptionTest [junit] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.401 sec,
improving tolerance to network failures
I am starting work on a project to improve the tolerance of Zookeeper to network failures and would like feedback on the idea. The problem is that with environments where link bonding is forbidden (they exist, trust me), Zookeeper is sensitive to the loss of a single switch or a few network links. This applies to client and server. Upon examination of the problem, I think that this could be mitigated by changing the logic that opens connections between servers to try one of several options. This should be a small change. I think that dynamic reconfiguration should be fine with this as well. On the client side, the situation is simpler, we can simply provide, either by configuration or from the server cluster, a list of all possible addresses and the client's current connection logic should work fine. One worry I have has to do with certificates on secure connection, but it seems that multiple certs would work the trick. I have started a collaborative document to work on the design approach. Once that is judged by the community to be sufficiently mature, I will move it to a JIRA. That document is at https://docs.google.com/document/d/1iGVwxeHp57qogwfdodCh9b32P2_kOQaJZ2GDo7j36fI/edit?usp=sharing The design document is currently open to the world for commenting so that anybody can suggest changes or ask questions. I will act as a bit of a moderator so that the document can remain completely open.
Failed: ZOOKEEPER- PreCommit Build #2486
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2486/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 85.60 MB...] [exec] == [exec] [exec] [exec] [exec] Error: No value specified for option "issue" [exec] Session logged out. Session was JSESSIONID=30D0C254241DE00452141CF9868CB2D1. [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD FAILED /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1859: exec returned: 1 Total time: 22 minutes 12 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ Recording test results Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ [description-setter] Description set: ZOOKEEPER-3140 Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ Adding one-line test results to commit status... Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ Setting status of 0efc8306760350d9ee4691d979416acf320fb108 to FAILURE with url https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2486/ and message: 'FAILURE 1811 tests run, 2 skipped, 0 failed.' Using context: Jenkins Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2486/ Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ Setting JDK_1_8_LATEST__HOME=/home/jenkins/tools/java/latest1.8 Setting MAVEN_3_LATEST__HOME=/home/jenkins/tools/maven/latest3/ ### ## FAILED TESTS (if any) ## All tests passed
[GitHub] zookeeper issue #628: ZOOKEEPER-3140: Allow Followers to host Observers
Github user asfgit commented on the issue: https://github.com/apache/zookeeper/pull/628 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2486/ ---
[GitHub] zookeeper issue #669: ZOOKEEPER-3152: Port ZK netty stack to netty4
Github user asfgit commented on the issue: https://github.com/apache/zookeeper/pull/669 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2484/ ---
[GitHub] zookeeper issue #669: ZOOKEEPER-3152: Port ZK netty stack to netty4
Github user asfgit commented on the issue: https://github.com/apache/zookeeper/pull/669 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2485/ ---
[GitHub] zookeeper issue #669: ZOOKEEPER-3152: Port ZK netty stack to netty4
Github user asfgit commented on the issue: https://github.com/apache/zookeeper/pull/669 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2483/ ---
[GitHub] zookeeper issue #669: ZOOKEEPER-3152: Port ZK netty stack to netty4
Github user ivmaykov commented on the issue: https://github.com/apache/zookeeper/pull/669 Cleaner Epoll/Nio selection code ---
[GitHub] zookeeper issue #669: ZOOKEEPER-3152: Port ZK netty stack to netty4
Github user ivmaykov commented on the issue: https://github.com/apache/zookeeper/pull/669 @eolivelli use `voidPromise()` to avoid allocations when writing to channel ---
[GitHub] zookeeper issue #669: ZOOKEEPER-3152: Port ZK netty stack to netty4
Github user asfgit commented on the issue: https://github.com/apache/zookeeper/pull/669 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2482/ ---
[GitHub] zookeeper issue #669: ZOOKEEPER-3152: Port ZK netty stack to netty4
Github user ivmaykov commented on the issue: https://github.com/apache/zookeeper/pull/669 @eolivelli changes you requested: - use Epoll if available - if (LOG.isDebugEnabled()) around complex LOG.debug() statements - use netty-all artifact Haven't looked into voidPromise() yet. ---
[jira] [Updated] (ZOOKEEPER-3029) Create pom.xml for jute and server
[
https://issues.apache.org/jira/browse/ZOOKEEPER-3029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Norbert Kalmar updated ZOOKEEPER-3029:
--
Summary: Create pom.xml for jute and server (was: Create pom.xml for jute,
server, client and common)
> Create pom.xml for jute and server
> --
>
> Key: ZOOKEEPER-3029
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3029
> Project: ZooKeeper
> Issue Type: Sub-task
> Components: build, scripts
>Affects Versions: 3.6.0
>Reporter: Norbert Kalmar
>Assignee: Norbert Kalmar
>Priority: Major
>
> After the directory structures has been created, it is time to create the pom
> files for all the modules, and create the build hierarchy.
> At first, ant should remain in place until we are sure maven works fine.
> jute, server, client and common should be priority first. docs is handled in
> a different jira, as it is also being migrated. Recipes and contrib will
> remain for last.
> The different modules will get their maven structure:
> {noformat}
> zookeeper-[something]
> | -src
> || -main
> ||| -java
> ||| \org...
> ||\resources
> || -test (unit tests only)
> ||| -java
> ||| \org...
> ||\ resources
> || - it (integration tests)
> |\pom.xml
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ZOOKEEPER-3029) Create pom.xml for jute and server
[
https://issues.apache.org/jira/browse/ZOOKEEPER-3029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Norbert Kalmar updated ZOOKEEPER-3029:
--
Description:
After the directory structures has been created, it is time to create the pom
files for all the modules, and create the build hierarchy.
At first, ant should remain in place until we are sure maven works fine.
jute and server should be priority first. docs is handled in a different jira,
as it is also being migrated. Recipes and contrib will remain for last.
The different modules will get their maven structure:
{noformat}
zookeeper-[something]
| -src
|| -main
||| -java
||| \org...
||\resources
|| -test (unit tests only)
||| -java
||| \org...
||\ resources
|| - it (integration tests)
|\pom.xml
{noformat}
was:
After the directory structures has been created, it is time to create the pom
files for all the modules, and create the build hierarchy.
At first, ant should remain in place until we are sure maven works fine.
jute, server, client and common should be priority first. docs is handled in a
different jira, as it is also being migrated. Recipes and contrib will remain
for last.
The different modules will get their maven structure:
{noformat}
zookeeper-[something]
| -src
|| -main
||| -java
||| \org...
||\resources
|| -test (unit tests only)
||| -java
||| \org...
||\ resources
|| - it (integration tests)
|\pom.xml
{noformat}
> Create pom.xml for jute and server
> --
>
> Key: ZOOKEEPER-3029
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3029
> Project: ZooKeeper
> Issue Type: Sub-task
> Components: build, scripts
>Affects Versions: 3.6.0
>Reporter: Norbert Kalmar
>Assignee: Norbert Kalmar
>Priority: Major
>
> After the directory structures has been created, it is time to create the pom
> files for all the modules, and create the build hierarchy.
> At first, ant should remain in place until we are sure maven works fine.
> jute and server should be priority first. docs is handled in a different
> jira, as it is also being migrated. Recipes and contrib will remain for last.
> The different modules will get their maven structure:
> {noformat}
> zookeeper-[something]
> | -src
> || -main
> ||| -java
> ||| \org...
> ||\resources
> || -test (unit tests only)
> ||| -java
> ||| \org...
> ||\ resources
> || - it (integration tests)
> |\pom.xml
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] zookeeper issue #665: [ZOOKEEPER-3163] Use session map in the Netty to impro...
Github user lvfangmin commented on the issue: https://github.com/apache/zookeeper/pull/665 Yes, it's similar to ZOOKEEPER-1669, which uses sessionMap to reduce the cost of close session, most of the code are identical as well. In 3.4, it has the removeCnxn method defined in NettyServerCnxnFactory while it keeps the same logic in NettyServerCnxn.close() in 3.6, I'll leave it as is for now, given that it's not quite relative to this Jira and this diff is already ready to land. ---
[GitHub] zookeeper issue #647: [ZOOKEEPER-3125] Fixing pzxid consistent issue when re...
Github user lvfangmin commented on the issue: https://github.com/apache/zookeeper/pull/647 @anmolnar what's your opinion with @hanm 's reply? ---
ZooKeeper_branch34_openjdk7 - Build # 2095 - Failure
See https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/2095/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 43.42 KB...] [junit] Running org.apache.zookeeper.test.SaslAuthFailDesignatedClientTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.442 sec [junit] Running org.apache.zookeeper.test.SaslAuthFailNotifyTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.613 sec [junit] Running org.apache.zookeeper.test.SaslAuthFailTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.714 sec [junit] Running org.apache.zookeeper.test.SaslAuthMissingClientConfigTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.545 sec [junit] Running org.apache.zookeeper.test.SaslClientTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.085 sec [junit] Running org.apache.zookeeper.test.SessionInvalidationTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.542 sec [junit] Running org.apache.zookeeper.test.SessionTest [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.301 sec [junit] Running org.apache.zookeeper.test.SessionTimeoutTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.786 sec [junit] Running org.apache.zookeeper.test.StandaloneTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.867 sec [junit] Running org.apache.zookeeper.test.StatTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.767 sec [junit] Running org.apache.zookeeper.test.StaticHostProviderTest [junit] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.648 sec [junit] Running org.apache.zookeeper.test.SyncCallTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.635 sec [junit] Running org.apache.zookeeper.test.TruncateTest [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.408 sec [junit] Running org.apache.zookeeper.test.UpgradeTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.227 sec [junit] Running org.apache.zookeeper.test.WatchedEventTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.085 sec [junit] Running org.apache.zookeeper.test.WatcherFuncTest [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.041 sec [junit] Running org.apache.zookeeper.test.WatcherTest [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.007 sec [junit] Running org.apache.zookeeper.test.ZkDatabaseCorruptionTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.501 sec [junit] Running org.apache.zookeeper.test.ZooKeeperQuotaTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.675 sec [junit] Running org.apache.jute.BinaryInputArchiveTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.086 sec fail.build.on.test.failure: BUILD FAILED /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk7/build.xml:1467: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk7/build.xml:1470: Tests failed! Total time: 32 minutes 52 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Setting OPENJDK_7_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-7-openjdk-amd64/ Recording test results Setting OPENJDK_7_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-7-openjdk-amd64/ Setting OPENJDK_7_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-7-openjdk-amd64/ Setting OPENJDK_7_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-7-openjdk-amd64/ Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Setting OPENJDK_7_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-7-openjdk-amd64/ Setting OPENJDK_7_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-7-openjdk-amd64/ ### ## FAILED TESTS (if any) ## 1 tests failed. FAILED: org.apache.zookeeper.SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure Error Message: Did not connect Stack Trace: java.util.concurrent.TimeoutException: Did not connect at org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(ClientBase.java:152) at org.apache.zookeeper.SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure(SaslAuthTest.java:174) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55)
Re: Decrease number of threads in Jenkins builds to reduce flakyness
Thanks Bogdan, so far so good. testNodeDataChanged is an old beast, I've a possible fix for that from @afine: https://github.com/apache/zookeeper/pull/300 Would be great if we could review it and get rid of this flaky. Andor On 10/20/18 06:41, Bogdan Kanivets wrote: > I think the argument for keeping concurrency is that it may manifest some > unknown problems with the code. > > Maybe a middle ground - move largest offenders into separate junit tag and > run them after rest of the test with threads=1. Hopefully this will make > life better for PRs. > > On the note of largest offenders, I've done 44 runs on aws r3.large with > various thread settings (1, 2, 4, 8). > Failure counts: > 1 testNextConfigAlreadyActive > 1 testNonExistingOpCode > 1 testRaceConditionBetweenLeaderAndAckRequestProcessor > 1 testWatcherDisconnectOnClose > 2 testDoubleElection > 5 testCurrentServersAreObserversInNextConfig > 5 testNormalFollowerRunWithDiff > 7 startSingleServerTest > 18 testNodeDataChanged > > Haven't seen testPurgeWhenLogRollingInProgress > or testManyChildWatchersAutoReset failing yet. > > > > On Thu, Oct 18, 2018 at 10:03 PM Michael Han wrote: > >> It's a good idea to reduce the concurrency of to eliminate flakyness. Looks >> like single threaded unit tests on trunk is pretty stable >> https://builds.apache.org/job/zookeeper-trunk-single-thread/ (some >> failures >> are due to C tests). The build time is longer, but not too bad (for >> pre-commit build, for nightly build, build time should not be a concern at >> all). >> >> >> On Mon, Oct 15, 2018 at 5:50 AM Andor Molnar >> wrote: >> >>> +1 >>> >>> >>> >>> On Mon, Oct 15, 2018 at 1:55 PM, Enrico Olivelli >>> wrote: >>> Il giorno lun 15 ott 2018 alle ore 12:46 Andor Molnar ha scritto: > Thank you guys. This is great help. > > I remember your efforts Bogdan, as far as I remember you observer >>> thread starvation in multiple runs on Apache Jenkins. Correct my if I’m wrong. > I’ve created an umbrella Jira to capture all flaky test fixing >> efforts here: > https://issues.apache.org/jira/browse/ZOOKEEPER-3170 < https://issues.apache.org/jira/browse/ZOOKEEPER-3170> > All previous flaky-related tickets have been converted to sub-tasks. Some of them might not be up-to-date, please consider reviewing them >> and close if possible. Additionally feel free to create new sub-tasks to capture your actual work. > I’ve already modified Trunk and branch-3.5 builds to run on 4 threads for testing initially. It resulted in slightly more stable tests: +1 I have assigned the umbrella issue to you Andor as you are driving this important task. is is ok ? thank you Enrico > Trunk (java 8) - failing 1/4 (since #229) - build time increased by 40-45% > Trunk (java 9) - failing 0/2 (since #993) - ~40% > Trunk (java 10) - failing 1/2 (since #280) - > branch-3.5 (java 8) - failing 0/4 (since #1153) - ~35-45% > > However the pattern is not big enough and results are inaccurate, so >> I need more builds. I also need to fix a bug in SSL to get java9/10 >> builds working on 3.5. > Please let me know if I should revert the changes. Precommit build is still running on 8 threads, but I’d like to change that one too. > Regards, > Andor > > > >> On 2018. Oct 15., at 9:31, Bogdan Kanivets wrote: >> Fangmin, >> >> Those are good ideas. >> >> FYI, I've stated running tests continuously in aws m1.xlarge. >> https://github.com/lavacat/zookeeper-tests-lab >> >> So far, I've done ~ 12 runs of trunk. Same common offenders as in >>> Flaky >> dash: testManyChildWatchersAutoReset, >> testPurgeWhenLogRollingInProgr ess >> I'll do some more runs, then try to come up with report. >> >> I'm using aws and not Apache Jenkins env because of better >> control/observability. >> >> >> >> >> On Sun, Oct 14, 2018 at 4:58 PM Fangmin Lv wrote: >>> Internally, we also did some works to reduce the flaky, here are >> the main >>> things we've done: >>> >>> * using retry rule to retry in case the zk client lost it's connection, >>> this could happen if the quorum tests is running on unstable environment >>> and the leader election happened. >>> * using random port instead of sequentially to avoid the port >> racing when >>> running tests concurrently >>> * changing tests to avoid using the same test path when creating/deleting >>> nodes >>> >>> These greatly reduced the flaky internally, we should try those if we're >>> seeing similar issues in the Jenkins. >>> >>> Fangmin >>> >>> On Sat, Oct 13, 2018 at 10:48 AM Bogdan Kanivets < >>> bkaniv...@gmail.com >>> wrote: >>>
Re：ZooKeeper two issues review
About the ZOOKEEPER-3167：good feature,just go ahead.follow the guideline I had provided,then we can start from there.look at the patch you had attached,some quick comments:0.if(key.contains(path)). contains???1.don't use Chinese annotations.2.notice the order of import and don't import *.3.don't include the irrelevance file4.add a unit case. Best regards maoling Beijing,China - 原始邮件 - 发件人：田毅群 收件人："dev@zookeeper.apache.org" 主题：ZooKeeper two issues review 日期：2018年10月22日 00点43分 Hi, all I proposed a Jira issue to commit ZooKeeper codes. I was asked to follow the new issue. So firstly I need to send an email to describe my two issues. First one: Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-3167. Purpose: add an API to get total count of recursive sub nodes of one node Description: 1. In production environment, there will be always a situation that there are a lot of recursive sub nodes of one node. We need to count total number of the node. Like this.（We want to get all the subnodes of nodeA.） 2. Now, we can only use API getChildren which returns the List of first level of sub nodes.(We can only get the nodeB list directly). We need to iterate every sub node to get recursive sub nodes. It will cost a lot of time. 3. In zookeeper server side, it uses Hasp to store node. The key of the map represents the path of the node. We can iterate the map get total number of all levels of sub nodes of one node. Second One: Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-3168 Purpose: Reduce session revalidation time after zxid roll over Description: 1. Sometimes Zookeeper cluster will receive a lot of connections from clients, sometimes connection number even exceeds 1W. When zxid rolls over, the clients will reconnect and revalidate the session. 2. In Zookeeper design structure, when follower server receives the session revalidation requests, it will send requests to leader server, which is designed to be responsible for session revalidation. When LearnerZooKeeperServer receives reconnection, it will send revalidation requests to LeaderZooKeeperServer. LeaderZooKeeperServer will face a lot of pressure. 3. In a short time, Leader will handle lots of requests. I use a tool to get the statistics, some clients need to wait over 20s. It is too long for some special clients, like ResourceManager. 4. I design a thought: when zxid rollover happens. Leader will record the accurate time. When reelection finishs, all servers will get the rollover time. When clients reconnect and revalidate session. All servers can judge it. So it can reduce a lots of pressure of cluster, all clients can will wait for less time. These are my two issues. Help to review the solution is right or not. Thank you a lot. 田毅群 技术产品中心 云平台 爱奇艺公司 QIYI.com, Inc. 地址：上海市长宁区临虹路365号爱奇艺创新大厦6层 邮编：201103 手机：＋86 157 2140 1256 邮箱：tianyi...@qiyi.com
ZooKeeper_branch35_jdk8 - Build # 1165 - Still Failing
See https://builds.apache.org/job/ZooKeeper_branch35_jdk8/1165/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 108.05 KB...] [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.136 sec, Thread: 1, Class: org.apache.zookeeper.server.quorum.ReconfigDuringLeaderSyncTest [junit] Running org.apache.zookeeper.server.quorum.ReconfigFailureCasesTest in thread 1 [junit] Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 123.548 sec, Thread: 3, Class: org.apache.zookeeper.server.quorum.QuorumRequestPipelineTest [junit] Running org.apache.zookeeper.server.quorum.ReconfigLegacyTest in thread 3 [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 70.286 sec, Thread: 1, Class: org.apache.zookeeper.server.quorum.ReconfigFailureCasesTest [junit] Running org.apache.zookeeper.server.quorum.ReconfigRecoveryTest in thread 1 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.467 sec, Thread: 3, Class: org.apache.zookeeper.server.quorum.ReconfigLegacyTest [junit] Running org.apache.zookeeper.server.quorum.ReconfigRollingRestartCompatibilityTest in thread 3 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.593 sec, Thread: 3, Class: org.apache.zookeeper.server.quorum.ReconfigRollingRestartCompatibilityTest [junit] Running org.apache.zookeeper.server.quorum.RemotePeerBeanTest in thread 3 [junit] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 215.603 sec, Thread: 4, Class: org.apache.zookeeper.server.quorum.QuorumPeerMainTest [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.846 sec, Thread: 3, Class: org.apache.zookeeper.server.quorum.RemotePeerBeanTest [junit] Running org.apache.zookeeper.server.quorum.StatCommandTest in thread 4 [junit] Running org.apache.zookeeper.server.quorum.StandaloneDisabledTest in thread 3 [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.357 sec, Thread: 4, Class: org.apache.zookeeper.server.quorum.StatCommandTest [junit] Running org.apache.zookeeper.server.quorum.StatResetCommandTest in thread 4 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.976 sec, Thread: 4, Class: org.apache.zookeeper.server.quorum.StatResetCommandTest [junit] Running org.apache.zookeeper.server.quorum.UnifiedServerSocketTest in thread 4 [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.427 sec, Thread: 4, Class: org.apache.zookeeper.server.quorum.UnifiedServerSocketTest [junit] Running org.apache.zookeeper.server.quorum.WatchLeakTest in thread 4 [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.894 sec, Thread: 4, Class: org.apache.zookeeper.server.quorum.WatchLeakTest [junit] Running org.apache.zookeeper.server.quorum.Zab1_0Test in thread 4 [junit] Tests run: 12, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 37.319 sec, Thread: 4, Class: org.apache.zookeeper.server.quorum.Zab1_0Test [junit] Test org.apache.zookeeper.server.quorum.Zab1_0Test FAILED [junit] Running org.apache.zookeeper.server.quorum.auth.MiniKdcTest in thread 4 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.136 sec, Thread: 4, Class: org.apache.zookeeper.server.quorum.auth.MiniKdcTest [junit] Running org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest in thread 4 [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 98.036 sec, Thread: 3, Class: org.apache.zookeeper.server.quorum.StandaloneDisabledTest [junit] Running org.apache.zookeeper.server.quorum.auth.QuorumDigestAuthTest in thread 3 [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 40.537 sec, Thread: 4, Class: org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest [junit] Running org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest in thread 4 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.12 sec, Thread: 4, Class: org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest [junit] Running org.apache.zookeeper.server.quorum.auth.QuorumKerberosHostBasedAuthTest in thread 4 [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.153 sec, Thread: 3, Class: org.apache.zookeeper.server.quorum.auth.QuorumDigestAuthTest [junit] Running org.apache.zookeeper.server.util.SerializeUtilsTest in thread 3 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.091 sec, Thread: 3, Class: org.apache.zookeeper.server.util.SerializeUtilsTest [junit] Running org.apache.zookeeper.server.util.VerifyingFileFactoryTest in thread 3 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time
ZooKeeper_branch34_openjdk8 - Build # 96 - Failure
See https://builds.apache.org/job/ZooKeeper_branch34_openjdk8/96/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 71.60 KB...] [junit] Running org.apache.zookeeper.test.SaslAuthFailDesignatedClientTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.508 sec [junit] Running org.apache.zookeeper.test.SaslAuthFailNotifyTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.581 sec [junit] Running org.apache.zookeeper.test.SaslAuthFailTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.742 sec [junit] Running org.apache.zookeeper.test.SaslAuthMissingClientConfigTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.646 sec [junit] Running org.apache.zookeeper.test.SaslClientTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.101 sec [junit] Running org.apache.zookeeper.test.SessionInvalidationTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.754 sec [junit] Running org.apache.zookeeper.test.SessionTest [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.372 sec [junit] Running org.apache.zookeeper.test.SessionTimeoutTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.855 sec [junit] Running org.apache.zookeeper.test.StandaloneTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.964 sec [junit] Running org.apache.zookeeper.test.StatTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.918 sec [junit] Running org.apache.zookeeper.test.StaticHostProviderTest [junit] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.555 sec [junit] Running org.apache.zookeeper.test.SyncCallTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.683 sec [junit] Running org.apache.zookeeper.test.TruncateTest [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.518 sec [junit] Running org.apache.zookeeper.test.UpgradeTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.921 sec [junit] Running org.apache.zookeeper.test.WatchedEventTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109 sec [junit] Running org.apache.zookeeper.test.WatcherFuncTest [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.34 sec [junit] Running org.apache.zookeeper.test.WatcherTest [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.423 sec [junit] Running org.apache.zookeeper.test.ZkDatabaseCorruptionTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.559 sec [junit] Running org.apache.zookeeper.test.ZooKeeperQuotaTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.735 sec [junit] Running org.apache.jute.BinaryInputArchiveTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.075 sec fail.build.on.test.failure: BUILD FAILED /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk8/build.xml:1467: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk8/build.xml:1470: Tests failed! Total time: 40 minutes 3 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Setting OPENJDK_8_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-8-openjdk-amd64/ Recording test results Setting OPENJDK_8_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-8-openjdk-amd64/ Setting OPENJDK_8_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-8-openjdk-amd64/ Setting OPENJDK_8_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-8-openjdk-amd64/ Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Setting OPENJDK_8_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-8-openjdk-amd64/ Setting OPENJDK_8_ON_UBUNTU_ONLY__HOME=/usr/lib/jvm/java-8-openjdk-amd64/ ### ## FAILED TESTS (if any) ## 1 tests failed. FAILED: org.apache.zookeeper.server.SessionTrackerTest.testAddSessionAfterSessionExpiry Error Message: Should throw session expiry exception as the session has expired and closed Stack Trace: junit.framework.AssertionFailedError: Should throw session expiry exception as the session has expired and closed at org.apache.zookeeper.server.SessionTrackerTest.testAddSessionAfterSessionExpiry(SessionTrackerTest.java:74) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55) at
