[GitHub] zookeeper issue #229: ZOOKEEPER-2759: Flaky test: org.apache.zookeeper.serve...
Github user afine commented on the issue: https://github.com/apache/zookeeper/pull/229 @hanm thank you for the through review. you are right the original test is bogus. I wanted to write the test in a way that verifies behavior for both peers so I used mockito to spy on the sender worker maps. Let me know what you think. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (ZOOKEEPER-2759) Flaky test: org.apache.zookeeper.server.quorum.QuorumCnxManagerTest.testNoAuthLearnerConnectToAuthRequiredServerWithHigherSid
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985967#comment-15985967 ] ASF GitHub Bot commented on ZOOKEEPER-2759: --- Github user afine commented on the issue: https://github.com/apache/zookeeper/pull/229 @hanm thank you for the through review. you are right the original test is bogus. I wanted to write the test in a way that verifies behavior for both peers so I used mockito to spy on the sender worker maps. Let me know what you think. > Flaky test: > org.apache.zookeeper.server.quorum.QuorumCnxManagerTest.testNoAuthLearnerConnectToAuthRequiredServerWithHigherSid > - > > Key: ZOOKEEPER-2759 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2759 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.10 >Reporter: Abraham Fine >Assignee: Abraham Fine > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2662) Export a metric for txn log sync times
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985814#comment-15985814 ] Edward Ribeiro commented on ZOOKEEPER-2662: --- Hey [~ckulkarni], I didn't have any spare cycle to spend on ZK issues, but got to resume it now. Thanks for bringing this up. Yup, I think this unit test is enough, so thank you **very much** for the test case. :) [~hanm], would you mind to take a look at this PR, please? > Export a metric for txn log sync times > -- > > Key: ZOOKEEPER-2662 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2662 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Edward Ribeiro > Fix For: 3.6.0 > > Attachments: ZOOKEEPER-2662.diff > > > In FileTxnLog there is code that records the amount of time required to fsync > the txn log in order to warn if that time exceeds a configurable threshold. > This information should also be exported as a metric available by JMX so an > important aspect of quorum performance can be monitored. > ZooKeeperServerMXBean carries some global latency information for the server > process already, seems like a good place to put it if not an entirely new > bean for the TxnLog. After ZOOKEEPER-2310 might want to collect the same > information for snapshots. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2662) Export a metric for txn log sync times
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro updated ZOOKEEPER-2662: -- Fix Version/s: 3.6.0 > Export a metric for txn log sync times > -- > > Key: ZOOKEEPER-2662 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2662 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Edward Ribeiro > Fix For: 3.6.0 > > Attachments: ZOOKEEPER-2662.diff > > > In FileTxnLog there is code that records the amount of time required to fsync > the txn log in order to warn if that time exceeds a configurable threshold. > This information should also be exported as a metric available by JMX so an > important aspect of quorum performance can be monitored. > ZooKeeperServerMXBean carries some global latency information for the server > process already, seems like a good place to put it if not an entirely new > bean for the TxnLog. After ZOOKEEPER-2310 might want to collect the same > information for snapshots. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2662) Export a metric for txn log sync times
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985788#comment-15985788 ] ASF GitHub Bot commented on ZOOKEEPER-2662: --- GitHub user eribeiro opened a pull request: https://github.com/apache/zookeeper/pull/241 ZOOKEEPER-2662: Export a metric for txn log sync times You can merge this pull request into a Git repository by running: $ git pull https://github.com/eribeiro/zookeeper ZOOKEEPER-2662 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/241.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #241 commit 9349c6192a5832def17a35bb6f4769c6aa917a54 Author: Edward RibeiroDate: 2017-04-27T00:21:18Z ZOOKEEPER-2662: Export a metric for txn log sync times > Export a metric for txn log sync times > -- > > Key: ZOOKEEPER-2662 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2662 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Edward Ribeiro > Attachments: ZOOKEEPER-2662.diff > > > In FileTxnLog there is code that records the amount of time required to fsync > the txn log in order to warn if that time exceeds a configurable threshold. > This information should also be exported as a metric available by JMX so an > important aspect of quorum performance can be monitored. > ZooKeeperServerMXBean carries some global latency information for the server > process already, seems like a good place to put it if not an entirely new > bean for the TxnLog. After ZOOKEEPER-2310 might want to collect the same > information for snapshots. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] zookeeper pull request #241: ZOOKEEPER-2662: Export a metric for txn log syn...
GitHub user eribeiro opened a pull request: https://github.com/apache/zookeeper/pull/241 ZOOKEEPER-2662: Export a metric for txn log sync times You can merge this pull request into a Git repository by running: $ git pull https://github.com/eribeiro/zookeeper ZOOKEEPER-2662 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/241.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #241 commit 9349c6192a5832def17a35bb6f4769c6aa917a54 Author: Edward RibeiroDate: 2017-04-27T00:21:18Z ZOOKEEPER-2662: Export a metric for txn log sync times --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (ZOOKEEPER-2755) Allow to subclass ClientCnxnSocketNetty and NettyServerCnxn in order to use Netty Local transport
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985779#comment-15985779 ] ASF GitHub Bot commented on ZOOKEEPER-2755: --- Github user afine commented on a diff in the pull request: https://github.com/apache/zookeeper/pull/227#discussion_r113590962 --- Diff: src/java/test/org/apache/zookeeper/test/NettyLocalSuiteTest.java --- @@ -0,0 +1,35 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.zookeeper.test; + +import org.junit.runners.Suite; + +/** + * Run tests with: Netty Client against Netty server + */ +@Suite.SuiteClasses({ --- End diff -- > But IMHO a new system property/run time flag will add the need to run always twice the full suite at every build/release. Maybe. We have a good degree of flakyness in our testing and wondering if using this type of change for our precommit hook will increase performance and stability (especially if we can use something similar for server<->server). > At this moment I think it is not the time to add an official "local transport" netty connection factory. Agreed. > I can't find how to enable Netty for server-to-server communications It is currently not possible. All communications there are handled by old fashioned sockets. I was incorrect in my other comment, although I still think my point is valid. Is there a way to have similar functionality with old fashioned java sockets? I don't think this change makes too much sense for testing unless we can take zk off the OS network stack entirely. > Allow to subclass ClientCnxnSocketNetty and NettyServerCnxn in order to use > Netty Local transport > - > > Key: ZOOKEEPER-2755 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2755 > Project: ZooKeeper > Issue Type: New Feature > Components: java client, server >Affects Versions: 3.5.2 >Reporter: Enrico Olivelli > > ClientCnxnSocketNetty and NettyServerCnxn use explicitly InetSocketAddress > class to work with network addresses. > We can do a little refactoring to use only SocketAddress and make it possible > to create subclasses of ClientCnxnSocketNetty and NettyServerCnxn which > leverage built-in Netty 'local' channels. > Such Netty local channels do not create real sockets and so allow a simple > ZooKeeper server + ZooKeeper client to be run on the same JVM without binding > to real TCP endpoints. > Usecases: > Ability to run concurrently on the same machine tests of projects which use > ZooKeeper (usually in unit tests the server and the client run inside the > same JVM) without dealing with random ports and in general using less network > resources > Run simplified (standalone, all processes in the same JVM) versions of > applications which need a working ZooKeeper ensemble to run. > Note: > Embedding ZooKeeper server + client on the same JVM has many risks and in > general I think we should encourage users to do so, so I in this patch I will > not provide official implementations of ClientCnxnSocketNetty and > NettyServerCnxn. There will be implementations only inside the test packages, > in order to test that most of the features are working with custom socket > factories and in particular with the 'LocalAddress' specific subclass of > SocketAddress. > Note: > the 'Local' sockets feature will be available on Netty 4 too -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] zookeeper pull request #227: ZOOKEEPER-2755 Allow to subclass ClientCnxnSock...
Github user afine commented on a diff in the pull request: https://github.com/apache/zookeeper/pull/227#discussion_r113590962 --- Diff: src/java/test/org/apache/zookeeper/test/NettyLocalSuiteTest.java --- @@ -0,0 +1,35 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.zookeeper.test; + +import org.junit.runners.Suite; + +/** + * Run tests with: Netty Client against Netty server + */ +@Suite.SuiteClasses({ --- End diff -- > But IMHO a new system property/run time flag will add the need to run always twice the full suite at every build/release. Maybe. We have a good degree of flakyness in our testing and wondering if using this type of change for our precommit hook will increase performance and stability (especially if we can use something similar for server<->server). > At this moment I think it is not the time to add an official "local transport" netty connection factory. Agreed. > I can't find how to enable Netty for server-to-server communications It is currently not possible. All communications there are handled by old fashioned sockets. I was incorrect in my other comment, although I still think my point is valid. Is there a way to have similar functionality with old fashioned java sockets? I don't think this change makes too much sense for testing unless we can take zk off the OS network stack entirely. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Failed: ZOOKEEPER- PreCommit Build #641
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/641/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 69.33 MB...] [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] -1 core tests. The patch failed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/641//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/641//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/641//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 0eb5690833205864bde160bead71edcb8197f2d3 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: ‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess’ and ‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess’ are the same file BUILD FAILED /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1642: exec returned: 1 Total time: 13 minutes 34 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Compressed 577.80 KB of artifacts by 27.7% relative to #640 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2697 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## 1 tests failed. FAILED: org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentServersAreObserversInNextConfig Error Message: waiting for server 4 being up Stack Trace: junit.framework.AssertionFailedError: waiting for server 4 being up at org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentServersAreObserversInNextConfig(ReconfigRecoveryTest.java:217) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
[jira] [Commented] (ZOOKEEPER-2755) Allow to subclass ClientCnxnSocketNetty and NettyServerCnxn in order to use Netty Local transport
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985630#comment-15985630 ] Hadoop QA commented on ZOOKEEPER-2755: -- +1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/640//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/640//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/640//console This message is automatically generated. > Allow to subclass ClientCnxnSocketNetty and NettyServerCnxn in order to use > Netty Local transport > - > > Key: ZOOKEEPER-2755 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2755 > Project: ZooKeeper > Issue Type: New Feature > Components: java client, server >Affects Versions: 3.5.2 >Reporter: Enrico Olivelli > > ClientCnxnSocketNetty and NettyServerCnxn use explicitly InetSocketAddress > class to work with network addresses. > We can do a little refactoring to use only SocketAddress and make it possible > to create subclasses of ClientCnxnSocketNetty and NettyServerCnxn which > leverage built-in Netty 'local' channels. > Such Netty local channels do not create real sockets and so allow a simple > ZooKeeper server + ZooKeeper client to be run on the same JVM without binding > to real TCP endpoints. > Usecases: > Ability to run concurrently on the same machine tests of projects which use > ZooKeeper (usually in unit tests the server and the client run inside the > same JVM) without dealing with random ports and in general using less network > resources > Run simplified (standalone, all processes in the same JVM) versions of > applications which need a working ZooKeeper ensemble to run. > Note: > Embedding ZooKeeper server + client on the same JVM has many risks and in > general I think we should encourage users to do so, so I in this patch I will > not provide official implementations of ClientCnxnSocketNetty and > NettyServerCnxn. There will be implementations only inside the test packages, > in order to test that most of the features are working with custom socket > factories and in particular with the 'LocalAddress' specific subclass of > SocketAddress. > Note: > the 'Local' sockets feature will be available on Netty 4 too -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2755) Allow to subclass ClientCnxnSocketNetty and NettyServerCnxn in order to use Netty Local transport
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985097#comment-15985097 ] Hadoop QA commented on ZOOKEEPER-2755: -- -1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/638//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/638//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/638//console This message is automatically generated. > Allow to subclass ClientCnxnSocketNetty and NettyServerCnxn in order to use > Netty Local transport > - > > Key: ZOOKEEPER-2755 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2755 > Project: ZooKeeper > Issue Type: New Feature > Components: java client, server >Affects Versions: 3.5.2 >Reporter: Enrico Olivelli > > ClientCnxnSocketNetty and NettyServerCnxn use explicitly InetSocketAddress > class to work with network addresses. > We can do a little refactoring to use only SocketAddress and make it possible > to create subclasses of ClientCnxnSocketNetty and NettyServerCnxn which > leverage built-in Netty 'local' channels. > Such Netty local channels do not create real sockets and so allow a simple > ZooKeeper server + ZooKeeper client to be run on the same JVM without binding > to real TCP endpoints. > Usecases: > Ability to run concurrently on the same machine tests of projects which use > ZooKeeper (usually in unit tests the server and the client run inside the > same JVM) without dealing with random ports and in general using less network > resources > Run simplified (standalone, all processes in the same JVM) versions of > applications which need a working ZooKeeper ensemble to run. > Note: > Embedding ZooKeeper server + client on the same JVM has many risks and in > general I think we should encourage users to do so, so I in this patch I will > not provide official implementations of ClientCnxnSocketNetty and > NettyServerCnxn. There will be implementations only inside the test packages, > in order to test that most of the features are working with custom socket > factories and in particular with the 'LocalAddress' specific subclass of > SocketAddress. > Note: > the 'Local' sockets feature will be available on Netty 4 too -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2697) Handle graceful stop of ZookKeeper client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985098#comment-15985098 ] Hadoop QA commented on ZOOKEEPER-2697: -- -1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/639//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/639//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/639//console This message is automatically generated. > Handle graceful stop of ZookKeeper client > - > > Key: ZOOKEEPER-2697 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2697 > Project: ZooKeeper > Issue Type: Improvement > Components: java client >Affects Versions: 3.4.9 >Reporter: Enrico Olivelli >Priority: Critical > > As seen in ZOOKEEPER-1394 I would like to have the "close" which waits for > all background activities to finish. > In tests the method "testableWaitForShutdown" is used. > We can add a new ZooKeeper.close(int tineout) method which will act as > testableWaitForShutdown, joining all support threads. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2662) Export a metric for txn log sync times
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985094#comment-15985094 ] Andrew Purtell commented on ZOOKEEPER-2662: --- The test looks good to me [~ckulkarni]. > Export a metric for txn log sync times > -- > > Key: ZOOKEEPER-2662 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2662 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Edward Ribeiro > Attachments: ZOOKEEPER-2662.diff > > > In FileTxnLog there is code that records the amount of time required to fsync > the txn log in order to warn if that time exceeds a configurable threshold. > This information should also be exported as a metric available by JMX so an > important aspect of quorum performance can be monitored. > ZooKeeperServerMXBean carries some global latency information for the server > process already, seems like a good place to put it if not an entirely new > bean for the TxnLog. After ZOOKEEPER-2310 might want to collect the same > information for snapshots. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Success: ZOOKEEPER- PreCommit Build #636
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/636/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 69.73 MB...] [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 11 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/636//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/636//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/636//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 10497c140a2ebed6ed6ac1faa1358967f381a510 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD SUCCESSFUL Total time: 18 minutes 33 seconds Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2755 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Success Sending email for trigger: Success Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (ZOOKEEPER-2697) Handle graceful stop of ZookKeeper client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984735#comment-15984735 ] Hadoop QA commented on ZOOKEEPER-2697: -- +1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/637//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/637//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/637//console This message is automatically generated. > Handle graceful stop of ZookKeeper client > - > > Key: ZOOKEEPER-2697 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2697 > Project: ZooKeeper > Issue Type: Improvement > Components: java client >Affects Versions: 3.4.9 >Reporter: Enrico Olivelli >Priority: Critical > > As seen in ZOOKEEPER-1394 I would like to have the "close" which waits for > all background activities to finish. > In tests the method "testableWaitForShutdown" is used. > We can add a new ZooKeeper.close(int tineout) method which will act as > testableWaitForShutdown, joining all support threads. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Success: ZOOKEEPER- PreCommit Build #637
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/637/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 68.65 MB...] [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 6 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/637//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/637//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/637//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 86166b7ae492aeab63dabf43c049c17094ede996 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: ‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess’ and ‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess’ are the same file BUILD SUCCESSFUL Total time: 18 minutes 31 seconds Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2697 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Success Sending email for trigger: Success Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## All tests passed
[GitHub] zookeeper issue #222: ZOOKEEPER-2697 Handle graceful stop of ZookKeeper clie...
Github user eolivelli commented on the issue: https://github.com/apache/zookeeper/pull/222 @hanm This patch only adds a new method, with no changes to the rest of the library and it does not break binary compatibility. So IMHO It can be safely added to 3.5.x branch to. But I am a newbie of ZK project rules. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (ZOOKEEPER-2697) Handle graceful stop of ZookKeeper client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984535#comment-15984535 ] ASF GitHub Bot commented on ZOOKEEPER-2697: --- Github user eolivelli commented on the issue: https://github.com/apache/zookeeper/pull/222 @hanm This patch only adds a new method, with no changes to the rest of the library and it does not break binary compatibility. So IMHO It can be safely added to 3.5.x branch to. But I am a newbie of ZK project rules. > Handle graceful stop of ZookKeeper client > - > > Key: ZOOKEEPER-2697 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2697 > Project: ZooKeeper > Issue Type: Improvement > Components: java client >Affects Versions: 3.4.9 >Reporter: Enrico Olivelli >Priority: Critical > > As seen in ZOOKEEPER-1394 I would like to have the "close" which waits for > all background activities to finish. > In tests the method "testableWaitForShutdown" is used. > We can add a new ZooKeeper.close(int tineout) method which will act as > testableWaitForShutdown, joining all support threads. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] zookeeper pull request #222: ZOOKEEPER-2697 Handle graceful stop of ZookKeep...
Github user eolivelli commented on a diff in the pull request: https://github.com/apache/zookeeper/pull/222#discussion_r113415006 --- Diff: src/java/main/org/apache/zookeeper/ZooKeeper.java --- @@ -1325,6 +1325,23 @@ public synchronized void close() throws InterruptedException { } /** + * Close this client object as the {@link #close() } method. + * This method will wait for internal resources to be released. + * + * @param waitForShutdownTimeoutMs timeout (in milliseconds) to wait for resources to be released. + * Use zero or a negative value to skip the wait + * @throws InterruptedException + * @return true if waitForShutdownTimeout is greater than zero and all of the resources have been released + */ +public boolean close(int waitForShutdownTimeoutMs) throws InterruptedException { --- End diff -- synchronization is in the real close() method. IMHO If I will add the synch here I think we will introduce unpredictable side effects --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (ZOOKEEPER-2697) Handle graceful stop of ZookKeeper client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984530#comment-15984530 ] ASF GitHub Bot commented on ZOOKEEPER-2697: --- Github user eolivelli commented on a diff in the pull request: https://github.com/apache/zookeeper/pull/222#discussion_r113414762 --- Diff: src/java/main/org/apache/zookeeper/ZooKeeper.java --- @@ -1325,6 +1325,23 @@ public synchronized void close() throws InterruptedException { } /** + * Close this client object as the {@link #close() } method. + * This method will wait for internal resources to be released. + * + * @param waitForShutdownTimeoutMs timeout (in milliseconds) to wait for resources to be released. + * Use zero or a negative value to skip the wait + * @throws InterruptedException + * @return true if waitForShutdownTimeout is greater than zero and all of the resources have been released + */ +public boolean close(int waitForShutdownTimeoutMs) throws InterruptedException { +close(); +if (waitForShutdownTimeoutMs > 0) { +return testableWaitForShutdown(waitForShutdownTimeoutMs); --- End diff -- @hanm I did not change the testableWaitForShutdown method in order not to break binary compatibilty with existing subclasses. For instance in my projects I use that method in order to achieve the purpose of this patches, If it is not a problem for other users I will change the signature > Handle graceful stop of ZookKeeper client > - > > Key: ZOOKEEPER-2697 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2697 > Project: ZooKeeper > Issue Type: Improvement > Components: java client >Affects Versions: 3.4.9 >Reporter: Enrico Olivelli >Priority: Critical > > As seen in ZOOKEEPER-1394 I would like to have the "close" which waits for > all background activities to finish. > In tests the method "testableWaitForShutdown" is used. > We can add a new ZooKeeper.close(int tineout) method which will act as > testableWaitForShutdown, joining all support threads. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Failed: ZOOKEEPER- PreCommit Build #635
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/635/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 68.00 MB...] [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] -1 core tests. The patch failed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/635//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/635//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/635//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 6446613479b76805c12da428b979c12f9622e0af logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD FAILED /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1642: exec returned: 1 Total time: 13 minutes 21 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Compressed 577.76 KB of artifacts by 55.4% relative to #630 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2763 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## 1 tests failed. FAILED: org.apache.zookeeper.server.ToStringTest.testJuteToString Error Message: expected not same Stack Trace: junit.framework.AssertionFailedError: expected not same at org.apache.zookeeper.server.ToStringTest.testJuteToString(ToStringTest.java:36) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
[jira] [Commented] (ZOOKEEPER-2763) Utils.toCsvBuffer() omits leading 0 for bytes < 0x10
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984255#comment-15984255 ] Hadoop QA commented on ZOOKEEPER-2763: -- -1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/634//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/634//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/634//console This message is automatically generated. > Utils.toCsvBuffer() omits leading 0 for bytes < 0x10 > > > Key: ZOOKEEPER-2763 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2763 > Project: ZooKeeper > Issue Type: Bug > Components: jute >Affects Versions: 3.5.2 >Reporter: Brandon Berg >Assignee: Alburt Hoffman >Priority: Minor > > org.apache.jute.Utils.toCsvBuffer(), which converts a byte array to a string > containing the hex representation of that byte array, omits the leading zero > for any byte less than 0x10, due to its use of Integer.toHexString, which has > the same behavior. > https://github.com/apache/zookeeper/blob/master/src/java/main/org/apache/jute/Utils.java#L234 > One consequence of this is that the hex strings printed by > ClientCnxn.Packet.toString(), used in the debug logging for > ClientCnxn.readResponse(), cannot be parsed to determine the result of a > Zookeeper request from client debug logs. > Utils.toXmlBuffer() appears to have the same issue. > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Failed: ZOOKEEPER- PreCommit Build #634
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/634/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 68.22 MB...] [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] -1 core tests. The patch failed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/634//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/634//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/634//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file [exec] 1f470f1a28469ce983888c1bfbf42d54f008bdd4 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] BUILD FAILED /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1642: exec returned: 1 Total time: 13 minutes 33 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Compressed 577.81 KB of artifacts by 55.4% relative to #630 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2763 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## 2 tests failed. FAILED: org.apache.zookeeper.test.AsyncHammerTest.testHammer Error Message: null Stack Trace: junit.framework.AssertionFailedError at org.apache.zookeeper.test.AsyncHammerTest.testHammer(AsyncHammerTest.java:186) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) FAILED: org.apache.zookeeper.server.ToStringTest.testJuteToString Error Message: expected not same Stack Trace: junit.framework.AssertionFailedError: expected not same at org.apache.zookeeper.server.ToStringTest.testJuteToString(ToStringTest.java:36) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)