ZooKeeper-trunk-WinVS2008 - Build # 2482 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk-WinVS2008/2482/
###
## LAST 60 LINES OF THE CONSOLE
###
Started by timer
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
[EnvInject] - Loading node environment variables.
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
Building remotely on windows-2012-2 (Windows) in workspace
f:\jenkins\jenkins-slave\workspace\ZooKeeper-trunk-WinVS2008
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url git://git.apache.org/zookeeper.git # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
Fetching upstream changes from git://git.apache.org/zookeeper.git
> git --version # timeout=10
> git fetch --tags --progress git://git.apache.org/zookeeper.git
> +refs/heads/*:refs/remotes/origin/*
> git rev-parse "refs/remotes/origin/master^{commit}" # timeout=10
> git rev-parse "refs/remotes/origin/origin/master^{commit}" # timeout=10
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
Checking out Revision 86577c9d81bd40067bc084e49a7172884ac9ae49
(refs/remotes/origin/master)
Commit message: "ZOOKEEPER-2880: Rename README.txt to README.md"
> git config core.sparsecheckout # timeout=10
> git checkout -f 86577c9d81bd40067bc084e49a7172884ac9ae49
> git rev-list 86577c9d81bd40067bc084e49a7172884ac9ae49 # timeout=10
No emails were triggered.
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
[ZooKeeper-trunk-WinVS2008] $ cmd.exe /C
"F:\jenkins\tools\ant\latest\bin\ant.bat -Dtest.output=yes
-Dtest.junit.output.format=xml clean compile_jute && exit %%ERRORLEVEL%%"
java.lang.UnsupportedClassVersionError: org/apache/tools/ant/launch/Launcher :
Unsupported major.minor version 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
Exception in thread "main" Build step 'Invoke Ant' marked build as failure
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
###
## FAILED TESTS (if any)
##
No tests ran.
ZooKeeper-trunk - Build # 3520 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk/3520/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 66.22 MB...]
[junit] 2017-09-01 23:27:44,470 [myid:127.0.0.1:24813] - INFO
[main-SendThread(127.0.0.1:24813):ClientCnxn$SendThread@1113] - Opening socket
connection to server 127.0.0.1/127.0.0.1:24813. Will not attempt to
authenticate using SASL (unknown error)
[junit] 2017-09-01 23:27:44,471 [myid:] - INFO [New I/O boss
#15288:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
[junit] java.net.ConnectException: Connection refused:
127.0.0.1/127.0.0.1:24813
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2017-09-01 23:27:44,490 [myid:] - WARN [New I/O boss
#15288:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id:
0xe2e3d6d1] EXCEPTION: java.net.ConnectException: Connection refused:
127.0.0.1/127.0.0.1:24813
[junit] java.net.ConnectException: Connection refused:
127.0.0.1/127.0.0.1:24813
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2017-09-01 23:27:44,490 [myid:] - INFO [New I/O boss
#15288:ClientCnxnSocketNetty@208] - channel is told closing
[junit] 2017-09-01 23:27:44,490 [myid:127.0.0.1:24813] - INFO
[main-SendThread(127.0.0.1:24813):ClientCnxn$SendThread@1231] - channel for
sessionid 0x20669722f8b is lost, closing socket connection and attempting
reconnect
[junit] Running org.apache.zookeeper.server.quorum.Zab1_0Test in thread 3
[junit] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0
sec, Thread: 3, Class: org.apache.zookeeper.server.quorum.Zab1_0Test
[junit] Test org.apache.zookeeper.server.quorum.Zab1_0Test FAILED (timeout)
fail.build.on.test.failure:
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1339: The
following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1220: The
following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1224: Tests
failed!
Total time: 16 minutes 43 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/too
[jira] [Commented] (ZOOKEEPER-2809) Unnecessary stack-trace in server when the client disconnect unexpectedly
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151154#comment-16151154
]
ASF GitHub Bot commented on ZOOKEEPER-2809:
---
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/355#discussion_r136665164
--- Diff: src/java/main/org/apache/zookeeper/server/NIOServerCnxn.java ---
@@ -374,14 +373,12 @@ void doIO(SelectionKey k) throws InterruptedException
{
// expecting close to log session closure
close();
} catch (EndOfStreamException e) {
-LOG.warn("caught end of stream exception",e); // tell user why
-
+LOG.warn(e.getMessage());
--- End diff --
Do we want to log the whole stack trace if debug is enabled here?
> Unnecessary stack-trace in server when the client disconnect unexpectedly
> -
>
> Key: ZOOKEEPER-2809
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2809
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.4.8
>Reporter: Paul Millar
>Assignee: Mark Fenes
>Priority: Minor
> Fix For: 3.5.0
>
>
> In ZK 3.4.x, if the client disconnects unexpectedly then the server logs this
> with a stack-trace (see
> src/java/main/org/apache/zookeeper/server/NIOServerCnxn.java:356).
> This is unfortunate as we are using an embedded ZK server in our project (in
> a test environment) and we consider all stack-traces as bugs.
> I noticed that ZK 3.5 and later no longer log a stack-trace. This change is
> due to commit 6206b495 (in branch-3.5), which adds ZOOKEEPER-1504 and seems
> to fix this issue almost as a side-effect; a similar change in master has the
> same effect.
> I was wondering if the change in how EndOfStreamException is logged (i.e.,
> logging the message without a stack-trace) could be back-ported to 3.4
> branch, so could be included in the next 3.4 release.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] zookeeper pull request #355: ZOOKEEPER-2809: Unnecessary stack-trace in serv...
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/355#discussion_r136665164
--- Diff: src/java/main/org/apache/zookeeper/server/NIOServerCnxn.java ---
@@ -374,14 +373,12 @@ void doIO(SelectionKey k) throws InterruptedException
{
// expecting close to log session closure
close();
} catch (EndOfStreamException e) {
-LOG.warn("caught end of stream exception",e); // tell user why
-
+LOG.warn(e.getMessage());
--- End diff --
Do we want to log the whole stack trace if debug is enabled here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zookeeper pull request #348: ZOOKEEPER-2319:UnresolvedAddressException cause...
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/348#discussion_r136663144
--- Diff:
src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java ---
@@ -621,7 +621,7 @@ public Listener() {
@Override
public void run() {
int numRetries = 0;
-InetSocketAddress addr;
+InetSocketAddress addr = null;
--- End diff --
@maoling I think a test case would be great because catching that exception
does "change behavior".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (ZOOKEEPER-2319) UnresolvedAddressException cause the QuorumCnxManager.Listener exit
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151129#comment-16151129
]
ASF GitHub Bot commented on ZOOKEEPER-2319:
---
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/348#discussion_r136663144
--- Diff:
src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java ---
@@ -621,7 +621,7 @@ public Listener() {
@Override
public void run() {
int numRetries = 0;
-InetSocketAddress addr;
+InetSocketAddress addr = null;
--- End diff --
@maoling I think a test case would be great because catching that exception
does "change behavior".
> UnresolvedAddressException cause the QuorumCnxManager.Listener exit
> ---
>
> Key: ZOOKEEPER-2319
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2319
> Project: ZooKeeper
> Issue Type: Bug
>Affects Versions: 3.4.6
>Reporter: Zhaohui Yu
> Fix For: 3.5.4, 3.6.0
>
>
> Given three nodes, the leader on 2, but some issue with this machine, so I
> shutdown this machine, and change the host name to another machine.
> Then I start the node in the new machine, but the new node can not join.
> I found the the 1 and 3's Listener thread exit.
> With the code of Listener's run method:
> I think we should catch UnresolvedAddressException to avoid the Listener exit.
> {noformat}
> @Override
> public void run() {
>
> while((!shutdown) && (numRetries < 3)){
> try {
>// bind and accept
> receiveConnection(client);
>
> } catch (IOException e) {
>
> }
> }
> //
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] zookeeper pull request #338: ZOOKEEPER-1260:Audit logging in ZooKeeper serve...
Github user afine commented on a diff in the pull request: https://github.com/apache/zookeeper/pull/338#discussion_r136662232 --- Diff: src/docs/src/documentation/content/xdocs/zookeeperAuditLogs.xml --- @@ -0,0 +1,205 @@ + + +http://www.oasis-open.org/docbook/xml/simple/1.0/sdocbook.dtd";> + + ZooKeeper Audit Logging + + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. You may + obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0";>http://www.apache.org/licenses/LICENSE-2.0. + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an "AS IS" + BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied. See the License for the specific language governing permissions + and limitations under the License. + + + +This document contains information about Audit Logs in ZooKeeper. + + + +ZooKeeper Audit Logs +Apache ZooKeeper supports audit logs form version 3.5.4. By default audit logs are disabled. To enable audit +logs configure audit.enable=true in conf/zoo.cfg. Audit logs are not logged on all the ZooKeeper servers, but logged +only on the servers where client is connected as depicted in bellow figure. --- End diff -- I agree the figure is great! My point was that you can just say "as depicted below". --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (ZOOKEEPER-1260) Audit logging in ZooKeeper servers.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151120#comment-16151120 ] ASF GitHub Bot commented on ZOOKEEPER-1260: --- Github user afine commented on a diff in the pull request: https://github.com/apache/zookeeper/pull/338#discussion_r136662232 --- Diff: src/docs/src/documentation/content/xdocs/zookeeperAuditLogs.xml --- @@ -0,0 +1,205 @@ + + +http://www.oasis-open.org/docbook/xml/simple/1.0/sdocbook.dtd";> + + ZooKeeper Audit Logging + + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. You may + obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0";>http://www.apache.org/licenses/LICENSE-2.0. + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an "AS IS" + BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied. See the License for the specific language governing permissions + and limitations under the License. + + + +This document contains information about Audit Logs in ZooKeeper. + + + +ZooKeeper Audit Logs +Apache ZooKeeper supports audit logs form version 3.5.4. By default audit logs are disabled. To enable audit +logs configure audit.enable=true in conf/zoo.cfg. Audit logs are not logged on all the ZooKeeper servers, but logged +only on the servers where client is connected as depicted in bellow figure. --- End diff -- I agree the figure is great! My point was that you can just say "as depicted below". > Audit logging in ZooKeeper servers. > --- > > Key: ZOOKEEPER-1260 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1260 > Project: ZooKeeper > Issue Type: New Feature > Components: server >Reporter: Mahadev konar >Assignee: Mohammad Arshad > Fix For: 3.5.4, 3.6.0 > > Attachments: ZOOKEEPER-1260-01.patch, zookeeperAuditLogs.pdf > > > Lots of users have had questions on debugging which client changed what znode > and what updates went through a znode. We should add audit logging as in > Hadoop (look at Namenode Audit logging) to log which client changed what in > the zookeeper servers. This could just be a log4j audit logger. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-1260) Audit logging in ZooKeeper servers.
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151104#comment-16151104
]
ASF GitHub Bot commented on ZOOKEEPER-1260:
---
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/338#discussion_r136660599
--- Diff: src/java/main/org/apache/zookeeper/server/ServerCnxnFactory.java
---
@@ -54,6 +54,7 @@
*/
static final ByteBuffer closeConn = ByteBuffer.allocate(0);
+private static String loginUser = System.getProperty("user.name",
"");
--- End diff --
Perhaps this can be renamed? Maybe "serverUser" or something to clarify
this is not the same thing as the user "logged in" on the client. Or at least a
comment.
> Audit logging in ZooKeeper servers.
> ---
>
> Key: ZOOKEEPER-1260
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1260
> Project: ZooKeeper
> Issue Type: New Feature
> Components: server
>Reporter: Mahadev konar
>Assignee: Mohammad Arshad
> Fix For: 3.5.4, 3.6.0
>
> Attachments: ZOOKEEPER-1260-01.patch, zookeeperAuditLogs.pdf
>
>
> Lots of users have had questions on debugging which client changed what znode
> and what updates went through a znode. We should add audit logging as in
> Hadoop (look at Namenode Audit logging) to log which client changed what in
> the zookeeper servers. This could just be a log4j audit logger.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] zookeeper pull request #338: ZOOKEEPER-1260:Audit logging in ZooKeeper serve...
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/338#discussion_r136660599
--- Diff: src/java/main/org/apache/zookeeper/server/ServerCnxnFactory.java
---
@@ -54,6 +54,7 @@
*/
static final ByteBuffer closeConn = ByteBuffer.allocate(0);
+private static String loginUser = System.getProperty("user.name",
"");
--- End diff --
Perhaps this can be renamed? Maybe "serverUser" or something to clarify
this is not the same thing as the user "logged in" on the client. Or at least a
comment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Updated] (ZOOKEEPER-2888) Reconfig Command Isolates One of the Nodes when All Ports Change
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cesar Stuardo updated ZOOKEEPER-2888: - Description: When we run our Distributed system Model Checking (DMCK) in ZooKeeper v3.5.3 by following the workload (complete details attached): 1. start a 5 node cluster (all nodes know each other). 2. wait for the cluster to reach a steady state. 3. issue reconfig command which does not add or remove nodes but changes all the ports of the existing cluster (no role change either). We observer that in some situations, one of the followers my end up isolated since the other nodes change their ports and end up setting up new connections. The consequence is similar to the one at [ZK-2865|https://issues.apache.org/jira/browse/ZOOKEEPER-2865?jql=] but the scenario is different. We provide further details in the attached document. was: When we run our Distributed system Model Checking (DMCK) in ZooKeeper v3.5.3 by following the workload (complete details attached): 1. start a 5 node cluster (all nodes know each other). 2. wait for the cluster to reach a steady state. 3. issue reconfig command which does not add or remove nodes but changes all the ports of the existing cluster (no role change either). We observer that in some situations, one of the followers my end up isolated since the other nodes change their ports and end up setting up new connections. The consequence is similar to the one at [ZK-2865|https://issues.apache.org/jira/browse/ZOOKEEPER-2865?jql=] but the scenario is different. We provide further details in the attached document. > Reconfig Command Isolates One of the Nodes when All Ports Change > > > Key: ZOOKEEPER-2888 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2888 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.5.3 >Reporter: Cesar Stuardo > Attachments: ZK-2888.pdf > > > When we run our Distributed system Model Checking (DMCK) in ZooKeeper v3.5.3 > by following the workload (complete details attached): > 1. start a 5 node cluster (all nodes know each other). > 2. wait for the cluster to reach a steady state. > 3. issue reconfig command which does not add or remove nodes but changes all > the ports of the existing cluster (no role change either). > We observer that in some situations, one of the followers my end up isolated > since the other nodes change their ports and end up setting up new > connections. The consequence is similar to the one at > [ZK-2865|https://issues.apache.org/jira/browse/ZOOKEEPER-2865?jql=] but the > scenario is different. > We provide further details in the attached document. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ZOOKEEPER-2888) Reconfig Command Isolates One of the Nodes when All Ports Change
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cesar Stuardo updated ZOOKEEPER-2888: - Summary: Reconfig Command Isolates One of the Nodes when All Ports Change (was: Reconfig Command Isolates One of the Nodes when all ports change) > Reconfig Command Isolates One of the Nodes when All Ports Change > > > Key: ZOOKEEPER-2888 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2888 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.5.3 >Reporter: Cesar Stuardo > Attachments: ZK-2888.pdf > > > When we run our Distributed system Model Checking (DMCK) in ZooKeeper v3.5.3 > by following the workload (complete details attached): > 1. start a 5 node cluster (all nodes know each other). > 2. wait for the cluster to reach a steady state. > 3. issue reconfig command which does not add or remove nodes but changes all > the ports of the existing cluster (no role change either). > We observer that in some situations, one of the followers my end up isolated > since the other nodes change their ports and end up setting up new > connections. The consequence is similar to the one at > [ZK-2865|https://issues.apache.org/jira/browse/ZOOKEEPER-2865?jql=] but the > scenario is different. > We provide further details in the attached document. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ZOOKEEPER-2888) Reconfig Command Isolates One of the Nodes when all ports change
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cesar Stuardo updated ZOOKEEPER-2888: - Summary: Reconfig Command Isolates One of the Nodes when all ports change (was: Reconfig Causes Inconsistent Configuration file among the nodes) > Reconfig Command Isolates One of the Nodes when all ports change > > > Key: ZOOKEEPER-2888 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2888 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.5.3 >Reporter: Cesar Stuardo > Attachments: ZK-2888.pdf > > > When we run our Distributed system Model Checking (DMCK) in ZooKeeper v3.5.3 > by following the workload (complete details attached): > 1. start a 5 node cluster (all nodes know each other). > 2. wait for the cluster to reach a steady state. > 3. issue reconfig command which does not add or remove nodes but changes all > the ports of the existing cluster (no role change either). > We observer that in some situations, one of the followers my end up isolated > since the other nodes change their ports and end up setting up new > connections. The consequence is similar to the one at > [ZK-2865|https://issues.apache.org/jira/browse/ZOOKEEPER-2865?jql=] but the > scenario is different. > We provide further details in the attached document. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ZOOKEEPER-2888) Reconfig Causes Inconsistent Configuration file among the nodes
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cesar Stuardo updated ZOOKEEPER-2888: - Attachment: ZK-2888.pdf > Reconfig Causes Inconsistent Configuration file among the nodes > --- > > Key: ZOOKEEPER-2888 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2888 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.5.3 >Reporter: Cesar Stuardo > Attachments: ZK-2888.pdf > > > When we run our Distributed system Model Checking (DMCK) in ZooKeeper v3.5.3 > by following the workload (complete details attached): > 1. start a 5 node cluster (all nodes know each other). > 2. wait for the cluster to reach a steady state. > 3. issue reconfig command which does not add or remove nodes but changes all > the ports of the existing cluster (no role change either). > We observer that in some situations, one of the followers my end up isolated > since the other nodes change their ports and end up setting up new > connections. The consequence is similar to the one at > [ZK-2865|https://issues.apache.org/jira/browse/ZOOKEEPER-2865?jql=] but the > scenario is different. > We provide further details in the attached document. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ZOOKEEPER-2888) Reconfig Causes Inconsistent Configuration file among the nodes
Cesar Stuardo created ZOOKEEPER-2888: Summary: Reconfig Causes Inconsistent Configuration file among the nodes Key: ZOOKEEPER-2888 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2888 Project: ZooKeeper Issue Type: Bug Components: quorum Affects Versions: 3.5.3 Reporter: Cesar Stuardo When we run our Distributed system Model Checking (DMCK) in ZooKeeper v3.5.3 by following the workload (complete details attached): 1. start a 5 node cluster (all nodes know each other). 2. wait for the cluster to reach a steady state. 3. issue reconfig command which does not add or remove nodes but changes all the ports of the existing cluster (no role change either). We observer that in some situations, one of the followers my end up isolated since the other nodes change their ports and end up setting up new connections. The consequence is similar to the one at [ZK-2865|https://issues.apache.org/jira/browse/ZOOKEEPER-2865?jql=] but the scenario is different. We provide further details in the attached document. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2865) Reconfig Causes Inconsistent Configuration file among the nodes
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151051#comment-16151051 ] Cesar Stuardo commented on ZOOKEEPER-2865: -- Hello Alexander, In your first comment, you state that But what's required is for the cluster to be able to recover from this state - the server that didn't get the commit in your scenario should find out about the new config and eventually join the cluster. If that doesn't happen then that potentially is a bug, but its not clear from the description here. What do you mean by this? In our scenario, the node wont be able to recover since the nodes that it knows at startup are not listening in the same ports anymore, thus wont get updated. The only solution is admin intervention. > Reconfig Causes Inconsistent Configuration file among the nodes > --- > > Key: ZOOKEEPER-2865 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2865 > Project: ZooKeeper > Issue Type: Improvement > Components: documentation >Affects Versions: 3.5.3 >Reporter: Jeffrey F. Lukman >Assignee: Alexander Shraer >Priority: Trivial > Fix For: 3.5.4, 3.6.0 > > Attachments: ZK-2865.pdf > > > When we run our Distributed system Model Checking (DMCK) in ZooKeeper v3.5.3 > by following the workload in ZK-2778: > - initially start 2 ZooKeeper nodes > - start 3 new nodes > - do a reconfiguration (the complete reconfiguration is attached in the > document) > We think our DMCK found this following bug: > - while one of the just joined nodes has not received the latest > configuration update > (called as node X), the initial leader node closed its port, > therefore causing the node X to be isolated. > For complete information of the bug, please see the document that is attached. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Success: ZOOKEEPER- PreCommit Build #990
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/990/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 73.28 MB...] [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +0 tests included. The patch appears to be a documentation patch that doesn't require tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/990//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/990//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/990//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 9e8e12e875948147c0b7534481fb4e0670d52dc3 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD SUCCESSFUL Total time: 20 minutes 15 seconds Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2887 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Success Sending email for trigger: Success Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (ZOOKEEPER-2887) define dependency versions in build.xml to be easily overridden in build.properties
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151041#comment-16151041 ] Hadoop QA commented on ZOOKEEPER-2887: -- +1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/990//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/990//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/990//console This message is automatically generated. > define dependency versions in build.xml to be easily overridden in > build.properties > --- > > Key: ZOOKEEPER-2887 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2887 > Project: ZooKeeper > Issue Type: Improvement > Components: build >Reporter: Tamas Penzes >Assignee: Tamas Penzes > > Dependency versions are defined in ivy.xml, which is suboptimal since it is > hard to override them from a script. > If we defined the versions in the main build.xml (just as we do with > audience-annotations.version) and use variables in ivy.xml then we could > easily override the versions with creating a build.properties file, which > mechanism is already built in. > This way the dependency versions could be replaced by sed or any simple > command line tool. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ZOOKEEPER-2778) Potential server deadlock between follower sync with leader and follower receiving external connection requests.
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151039#comment-16151039
]
Cesar Stuardo edited comment on ZOOKEEPER-2778 at 9/1/17 7:16 PM:
--
Hello,
We are computer science PhD students building a Distributed model checking
tool. We have been able to reproduce this issue too, when the QuorumPeer thread
is racing with the Listener thread and gets into deadlock by requesting
QCNXManager lock while holding QV_LOCK (on the other side, Listener thread
holds QCNXManager lock and requests QV_LOCK). We also see this potential issue
with the WorkerSender thread while performing toSend -> connectOne (one
argument, requesting QCNXManager_LOCK) -> connectOne (two arguments, requesting
QCNXManager_LOCK) -> initiateConnection -> getElectionAddress (requesting
QV_LOCK) which can also race with the QuorumPeer thread for the same locks.
was (Author: castuardo):
Hello,
We are computer science PhD students building a Distributed model checking
tool. We have been able to reproduce this issue too, when the QuorumPeer thread
is racing with the Listener thread and gets into deadlock by requesting
QCNXManager lock while holding QV_LOCK (on the other side, Listener thread
holds QCNXManager lock and requests QV_LOCK). We also see this potential issue
with the WorkerSender thread while performing toSend -> connectOne (one
argument, requesting QCNXManager_LOCK) -> connectAll -> connectOne (two
arguments, requesting QCNXManager_LOCK) -> initiateConnection ->
getElectionAddress (requesting QV_LOCK) which can also race with the QuorumPeer
thread for the same locks.
> Potential server deadlock between follower sync with leader and follower
> receiving external connection requests.
>
>
> Key: ZOOKEEPER-2778
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2778
> Project: ZooKeeper
> Issue Type: Bug
> Components: quorum
>Affects Versions: 3.5.3
>Reporter: Michael Han
>Assignee: Michael Han
>Priority: Critical
>
> It's possible to have a deadlock during recovery phase.
> Found this issue by analyzing thread dumps of "flaky" ReconfigRecoveryTest
> [1]. . Here is a sample thread dump that illustrates the state of the
> execution:
> {noformat}
> [junit] java.lang.Thread.State: BLOCKED
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.getElectionAddress(QuorumPeer.java:686)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:265)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:445)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.receiveConnection(QuorumCnxManager.java:369)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:642)
> [junit]
> [junit] java.lang.Thread.State: BLOCKED
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:472)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.connectNewPeers(QuorumPeer.java:1438)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.setLastSeenQuorumVerifier(QuorumPeer.java:1471)
> [junit] at
> org.apache.zookeeper.server.quorum.Learner.syncWithLeader(Learner.java:520)
> [junit] at
> org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:88)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1133)
> {noformat}
> The dead lock happens between the quorum peer thread which running the
> follower that doing sync with leader work, and the listener of the qcm of the
> same quorum peer that doing the receiving connection work. Basically to
> finish sync with leader, the follower needs to synchronize on both QV_LOCK
> and the qmc object it owns; while in the receiver thread to finish setup an
> incoming connection the thread needs to synchronize on both the qcm object
> the quorum peer owns, and the same QV_LOCK. It's easy to see the problem here
> is the order of acquiring two locks are different, thus depends on timing /
> actual execution order, two threads might end up acquiring one lock while
> holding another.
> [1]
> org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentServersAreObserversInNextConfig
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ZOOKEEPER-2778) Potential server deadlock between follower sync with leader and follower receiving external connection requests.
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151039#comment-16151039
]
Cesar Stuardo edited comment on ZOOKEEPER-2778 at 9/1/17 7:16 PM:
--
Hello,
We are computer science PhD students building a Distributed model checking
tool. We have been able to reproduce this issue too, when the QuorumPeer thread
is racing with the Listener thread and gets into deadlock by requesting
QCNXManager lock while holding QV_LOCK (on the other side, Listener thread
holds QCNXManager lock and requests QV_LOCK). We also see this potential issue
with the WorkerSender thread while performing toSend -> connectOne (one
argument, requesting QCNXManager_LOCK) -> connectAll -> connectOne (two
arguments, requesting QCNXManager_LOCK) -> initiateConnection ->
getElectionAddress (requesting QV_LOCK) which can also race with the QuorumPeer
thread for the same locks.
was (Author: castuardo):
Hello,
We are computer science PhD students building a Distributed model checking
tool. We have been able to reproduce this issue too, when the QuorumPeer thread
is racing with the Listener thread and gets into deadlock by requesting
QCNXManager lock while holding QV_LOCK (on the other side, Listener thread
holds QCNXManager lock and requests QV_LOCK). We also see this potential issue
with the WorkerSender thread while performing toSend -> connectOne (one
argument, requesting QCNXManager_LOCK) -> connectOne (two arguments, requesting
QCNXManager_LOCK) -> initiateConnection -> getElectionAddress (requesting
QV_LOCK) which can also race with the QuorumPeer thread for the same locks.
> Potential server deadlock between follower sync with leader and follower
> receiving external connection requests.
>
>
> Key: ZOOKEEPER-2778
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2778
> Project: ZooKeeper
> Issue Type: Bug
> Components: quorum
>Affects Versions: 3.5.3
>Reporter: Michael Han
>Assignee: Michael Han
>Priority: Critical
>
> It's possible to have a deadlock during recovery phase.
> Found this issue by analyzing thread dumps of "flaky" ReconfigRecoveryTest
> [1]. . Here is a sample thread dump that illustrates the state of the
> execution:
> {noformat}
> [junit] java.lang.Thread.State: BLOCKED
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.getElectionAddress(QuorumPeer.java:686)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:265)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:445)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.receiveConnection(QuorumCnxManager.java:369)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:642)
> [junit]
> [junit] java.lang.Thread.State: BLOCKED
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:472)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.connectNewPeers(QuorumPeer.java:1438)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.setLastSeenQuorumVerifier(QuorumPeer.java:1471)
> [junit] at
> org.apache.zookeeper.server.quorum.Learner.syncWithLeader(Learner.java:520)
> [junit] at
> org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:88)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1133)
> {noformat}
> The dead lock happens between the quorum peer thread which running the
> follower that doing sync with leader work, and the listener of the qcm of the
> same quorum peer that doing the receiving connection work. Basically to
> finish sync with leader, the follower needs to synchronize on both QV_LOCK
> and the qmc object it owns; while in the receiver thread to finish setup an
> incoming connection the thread needs to synchronize on both the qcm object
> the quorum peer owns, and the same QV_LOCK. It's easy to see the problem here
> is the order of acquiring two locks are different, thus depends on timing /
> actual execution order, two threads might end up acquiring one lock while
> holding another.
> [1]
> org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentServersAreObserversInNextConfig
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2778) Potential server deadlock between follower sync with leader and follower receiving external connection requests.
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151039#comment-16151039
]
Cesar Stuardo commented on ZOOKEEPER-2778:
--
Hello,
We are computer science PhD students building a Distributed model checking
tool. We have been able to reproduce this issue too, when the QuorumPeer thread
is racing with the Listener thread and gets into deadlock by requesting
QCNXManager lock while holding QV_LOCK (on the other side, Listener thread
holds QCNXManager lock and requests QV_LOCK). We also see this potential issue
with the WorkerSender thread while performing toSend -> connectOne (one
argument, requesting QCNXManager_LOCK) -> connectOne (two arguments, requesting
QCNXManager_LOCK) -> initiateConnection -> getElectionAddress (requesting
QV_LOCK) which can also race with the QuorumPeer thrad for the same locks.
> Potential server deadlock between follower sync with leader and follower
> receiving external connection requests.
>
>
> Key: ZOOKEEPER-2778
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2778
> Project: ZooKeeper
> Issue Type: Bug
> Components: quorum
>Affects Versions: 3.5.3
>Reporter: Michael Han
>Assignee: Michael Han
>Priority: Critical
>
> It's possible to have a deadlock during recovery phase.
> Found this issue by analyzing thread dumps of "flaky" ReconfigRecoveryTest
> [1]. . Here is a sample thread dump that illustrates the state of the
> execution:
> {noformat}
> [junit] java.lang.Thread.State: BLOCKED
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.getElectionAddress(QuorumPeer.java:686)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:265)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:445)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.receiveConnection(QuorumCnxManager.java:369)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:642)
> [junit]
> [junit] java.lang.Thread.State: BLOCKED
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:472)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.connectNewPeers(QuorumPeer.java:1438)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.setLastSeenQuorumVerifier(QuorumPeer.java:1471)
> [junit] at
> org.apache.zookeeper.server.quorum.Learner.syncWithLeader(Learner.java:520)
> [junit] at
> org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:88)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1133)
> {noformat}
> The dead lock happens between the quorum peer thread which running the
> follower that doing sync with leader work, and the listener of the qcm of the
> same quorum peer that doing the receiving connection work. Basically to
> finish sync with leader, the follower needs to synchronize on both QV_LOCK
> and the qmc object it owns; while in the receiver thread to finish setup an
> incoming connection the thread needs to synchronize on both the qcm object
> the quorum peer owns, and the same QV_LOCK. It's easy to see the problem here
> is the order of acquiring two locks are different, thus depends on timing /
> actual execution order, two threads might end up acquiring one lock while
> holding another.
> [1]
> org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentServersAreObserversInNextConfig
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ZOOKEEPER-2778) Potential server deadlock between follower sync with leader and follower receiving external connection requests.
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151039#comment-16151039
]
Cesar Stuardo edited comment on ZOOKEEPER-2778 at 9/1/17 7:15 PM:
--
Hello,
We are computer science PhD students building a Distributed model checking
tool. We have been able to reproduce this issue too, when the QuorumPeer thread
is racing with the Listener thread and gets into deadlock by requesting
QCNXManager lock while holding QV_LOCK (on the other side, Listener thread
holds QCNXManager lock and requests QV_LOCK). We also see this potential issue
with the WorkerSender thread while performing toSend -> connectOne (one
argument, requesting QCNXManager_LOCK) -> connectOne (two arguments, requesting
QCNXManager_LOCK) -> initiateConnection -> getElectionAddress (requesting
QV_LOCK) which can also race with the QuorumPeer thread for the same locks.
was (Author: castuardo):
Hello,
We are computer science PhD students building a Distributed model checking
tool. We have been able to reproduce this issue too, when the QuorumPeer thread
is racing with the Listener thread and gets into deadlock by requesting
QCNXManager lock while holding QV_LOCK (on the other side, Listener thread
holds QCNXManager lock and requests QV_LOCK). We also see this potential issue
with the WorkerSender thread while performing toSend -> connectOne (one
argument, requesting QCNXManager_LOCK) -> connectOne (two arguments, requesting
QCNXManager_LOCK) -> initiateConnection -> getElectionAddress (requesting
QV_LOCK) which can also race with the QuorumPeer thrad for the same locks.
> Potential server deadlock between follower sync with leader and follower
> receiving external connection requests.
>
>
> Key: ZOOKEEPER-2778
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2778
> Project: ZooKeeper
> Issue Type: Bug
> Components: quorum
>Affects Versions: 3.5.3
>Reporter: Michael Han
>Assignee: Michael Han
>Priority: Critical
>
> It's possible to have a deadlock during recovery phase.
> Found this issue by analyzing thread dumps of "flaky" ReconfigRecoveryTest
> [1]. . Here is a sample thread dump that illustrates the state of the
> execution:
> {noformat}
> [junit] java.lang.Thread.State: BLOCKED
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.getElectionAddress(QuorumPeer.java:686)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:265)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:445)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.receiveConnection(QuorumCnxManager.java:369)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:642)
> [junit]
> [junit] java.lang.Thread.State: BLOCKED
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:472)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.connectNewPeers(QuorumPeer.java:1438)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.setLastSeenQuorumVerifier(QuorumPeer.java:1471)
> [junit] at
> org.apache.zookeeper.server.quorum.Learner.syncWithLeader(Learner.java:520)
> [junit] at
> org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:88)
> [junit] at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1133)
> {noformat}
> The dead lock happens between the quorum peer thread which running the
> follower that doing sync with leader work, and the listener of the qcm of the
> same quorum peer that doing the receiving connection work. Basically to
> finish sync with leader, the follower needs to synchronize on both QV_LOCK
> and the qmc object it owns; while in the receiver thread to finish setup an
> incoming connection the thread needs to synchronize on both the qcm object
> the quorum peer owns, and the same QV_LOCK. It's easy to see the problem here
> is the order of acquiring two locks are different, thus depends on timing /
> actual execution order, two threads might end up acquiring one lock while
> holding another.
> [1]
> org.apache.zookeeper.server.quorum.ReconfigRecoveryTest.testCurrentServersAreObserversInNextConfig
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2887) define dependency versions in build.xml to be easily overridden in build.properties
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151019#comment-16151019 ] ASF GitHub Bot commented on ZOOKEEPER-2887: --- GitHub user tamaashu opened a pull request: https://github.com/apache/zookeeper/pull/357 ZOOKEEPER-2887: define dependency versions in build.xml to be easily … …overridden in build.properties If the dependency versions are defined in build.xml they can be easily overridden by re-defining them in build.properties This process can be useful to avoid classpath clashes among different Hadoop components You can merge this pull request into a Git repository by running: $ git pull https://github.com/tamaashu/zookeeper ZOOKEEPER-2887 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/357.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #357 commit 6cf315fdd6e7df2d091a5cd3a9753cb40d4357fe Author: Tamas Penzes Date: 2017-09-01T18:50:23Z ZOOKEEPER-2887: define dependency versions in build.xml to be easily overridden in build.properties If the dependency versions are defined in build.xml they can be easily overridden by re-defining them in build.properties This process can be useful to avoid classpath clashes among different Hadoop components > define dependency versions in build.xml to be easily overridden in > build.properties > --- > > Key: ZOOKEEPER-2887 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2887 > Project: ZooKeeper > Issue Type: Improvement > Components: build >Reporter: Tamas Penzes >Assignee: Tamas Penzes > > Dependency versions are defined in ivy.xml, which is suboptimal since it is > hard to override them from a script. > If we defined the versions in the main build.xml (just as we do with > audience-annotations.version) and use variables in ivy.xml then we could > easily override the versions with creating a build.properties file, which > mechanism is already built in. > This way the dependency versions could be replaced by sed or any simple > command line tool. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] zookeeper pull request #357: ZOOKEEPER-2887: define dependency versions in b...
GitHub user tamaashu opened a pull request: https://github.com/apache/zookeeper/pull/357 ZOOKEEPER-2887: define dependency versions in build.xml to be easily ⦠â¦overridden in build.properties If the dependency versions are defined in build.xml they can be easily overridden by re-defining them in build.properties This process can be useful to avoid classpath clashes among different Hadoop components You can merge this pull request into a Git repository by running: $ git pull https://github.com/tamaashu/zookeeper ZOOKEEPER-2887 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/357.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #357 commit 6cf315fdd6e7df2d091a5cd3a9753cb40d4357fe Author: Tamas Penzes Date: 2017-09-01T18:50:23Z ZOOKEEPER-2887: define dependency versions in build.xml to be easily overridden in build.properties If the dependency versions are defined in build.xml they can be easily overridden by re-defining them in build.properties This process can be useful to avoid classpath clashes among different Hadoop components --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (ZOOKEEPER-2887) define dependency versions in build.xml to be easily overridden in build.properties
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas Penzes updated ZOOKEEPER-2887: Summary: define dependency versions in build.xml to be easily overridden in build.properties (was: define dependency versions in build.xml to be easily overridden) > define dependency versions in build.xml to be easily overridden in > build.properties > --- > > Key: ZOOKEEPER-2887 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2887 > Project: ZooKeeper > Issue Type: Improvement > Components: build >Reporter: Tamas Penzes >Assignee: Tamas Penzes > > Dependency versions are defined in ivy.xml, which is suboptimal since it is > hard to override them from a script. > If we defined the versions in the main build.xml (just as we do with > audience-annotations.version) and use variables in ivy.xml then we could > easily override the versions with creating a build.properties file, which > mechanism is already built in. > This way the dependency versions could be replaced by sed or any simple > command line tool. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-1416) Persistent Recursive Watch
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151001#comment-16151001
]
ASF GitHub Bot commented on ZOOKEEPER-1416:
---
Github user skamille commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/136#discussion_r136637514
--- Diff: src/java/main/org/apache/zookeeper/server/WatchManager.java ---
@@ -40,50 +40,110 @@
class WatchManager {
private static final Logger LOG =
LoggerFactory.getLogger(WatchManager.class);
-private final HashMap> watchTable =
-new HashMap>();
+enum Type {
+STANDARD() {
+@Override
+boolean isPersistent() {
+return false;
+}
+
+@Override
+boolean isRecursive() {
+return false;
+}
+},
+PERSISTENT() {
+@Override
+boolean isPersistent() {
+return true;
+}
+
+@Override
+boolean isRecursive() {
+return false;
+}
+},
+PERSISTENT_RECURSIVE() {
+@Override
+boolean isPersistent() {
+return true;
+}
+
+@Override
+boolean isRecursive() {
+return true;
+}
+}
+;
+
+abstract boolean isPersistent();
+abstract boolean isRecursive();
+}
+
+private final Map> watchTable =
+new HashMap<>();
+
+private final Map> watch2Paths =
+new HashMap<>();
-private final HashMap> watch2Paths =
-new HashMap>();
+private int recursiveWatchQty = 0;// guarded by sync
+
+// visible for testing
+synchronized int getRecursiveWatchQty() {
+return recursiveWatchQty;
+}
synchronized int size(){
int result = 0;
-for(Set watches : watchTable.values()) {
+for(Map watches : watchTable.values()) {
result += watches.size();
}
return result;
}
-synchronized void addWatch(String path, Watcher watcher) {
-HashSet list = watchTable.get(path);
+synchronized void addWatch(String path, Watcher watcher,
WatchManager.Type type) {
+Map list = watchTable.get(path);
if (list == null) {
// don't waste memory if there are few watches on a node
// rehash when the 4th entry is added, doubling size thereafter
// seems like a good compromise
-list = new HashSet(4);
+list = new HashMap<>(4);
--- End diff --
Does the assumption still hold re: memory management now that we have a
hashmap instead of a hashset?
> Persistent Recursive Watch
> --
>
> Key: ZOOKEEPER-1416
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1416
> Project: ZooKeeper
> Issue Type: Improvement
> Components: c client, documentation, java client, server
>Reporter: Phillip Liu
>Assignee: Jordan Zimmerman
> Attachments: ZOOKEEPER-1416.patch, ZOOKEEPER-1416.patch
>
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> h4. The Problem
> A ZooKeeper Watch can be placed on a single znode and when the znode changes
> a Watch event is sent to the client. If there are thousands of znodes being
> watched, when a client (re)connect, it would have to send thousands of watch
> requests. At Facebook, we have this problem storing information for thousands
> of db shards. Consequently a naming service that consumes the db shard
> definition issues thousands of watch requests each time the service starts
> and changes client watcher.
> h4. Proposed Solution
> We add the notion of a Persistent Recursive Watch in ZooKeeper. Persistent
> means no Watch reset is necessary after a watch-fire. Recursive means the
> Watch applies to the node and descendant nodes. A Persistent Recursive Watch
> behaves as follows:
> # Recursive Watch supports all Watch semantics: CHILDREN, DATA, and EXISTS.
> # CHILDREN and DATA Recursive Watches can be placed on any znode.
> # EXISTS Recursive Watches can be placed on any path.
> # A Recursive Watch behaves like a auto-watch registrar on the server side.
> Setting a Recursive Watch means to set watches on all descendant znodes.
> # When a watch on a descendant fires, no subsequent event is fired until a
> corresponding getData(..) on the znode is called, then Recursive Watch
> automical
[GitHub] zookeeper pull request #136: [ZOOKEEPER-1416] Persistent Recursive Watch
Github user skamille commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/136#discussion_r136637514
--- Diff: src/java/main/org/apache/zookeeper/server/WatchManager.java ---
@@ -40,50 +40,110 @@
class WatchManager {
private static final Logger LOG =
LoggerFactory.getLogger(WatchManager.class);
-private final HashMap> watchTable =
-new HashMap>();
+enum Type {
+STANDARD() {
+@Override
+boolean isPersistent() {
+return false;
+}
+
+@Override
+boolean isRecursive() {
+return false;
+}
+},
+PERSISTENT() {
+@Override
+boolean isPersistent() {
+return true;
+}
+
+@Override
+boolean isRecursive() {
+return false;
+}
+},
+PERSISTENT_RECURSIVE() {
+@Override
+boolean isPersistent() {
+return true;
+}
+
+@Override
+boolean isRecursive() {
+return true;
+}
+}
+;
+
+abstract boolean isPersistent();
+abstract boolean isRecursive();
+}
+
+private final Map> watchTable =
+new HashMap<>();
+
+private final Map> watch2Paths =
+new HashMap<>();
-private final HashMap> watch2Paths =
-new HashMap>();
+private int recursiveWatchQty = 0;// guarded by sync
+
+// visible for testing
+synchronized int getRecursiveWatchQty() {
+return recursiveWatchQty;
+}
synchronized int size(){
int result = 0;
-for(Set watches : watchTable.values()) {
+for(Map watches : watchTable.values()) {
result += watches.size();
}
return result;
}
-synchronized void addWatch(String path, Watcher watcher) {
-HashSet list = watchTable.get(path);
+synchronized void addWatch(String path, Watcher watcher,
WatchManager.Type type) {
+Map list = watchTable.get(path);
if (list == null) {
// don't waste memory if there are few watches on a node
// rehash when the 4th entry is added, doubling size thereafter
// seems like a good compromise
-list = new HashSet(4);
+list = new HashMap<>(4);
--- End diff --
Does the assumption still hold re: memory management now that we have a
hashmap instead of a hashset?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Updated] (ZOOKEEPER-2887) define dependency versions in build.xml to be easily overridden
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas Penzes updated ZOOKEEPER-2887: Description: Dependency versions are defined in ivy.xml, which is suboptimal since it is hard to override them from a script. If we defined the versions in the main build.xml (just as we do with audience-annotations.version) and use variables in ivy.xml then we could easily override the versions with creating a build.properties file, which mechanism is already built in. This way the dependency versions could be replaced by sed or any simple command line tool. was: Dependency versions are defined in ivy.xml, which is suboptimal since it is hard to override them from a script. If we defined the versions in the main build.xml and use variables in ivy.xml then we could easily override the versions with creating a build.properties file, which mechanism is already built in. This way the dependency versions could be replaced by sed or any simple command line tool. > define dependency versions in build.xml to be easily overridden > --- > > Key: ZOOKEEPER-2887 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2887 > Project: ZooKeeper > Issue Type: Improvement > Components: build >Reporter: Tamas Penzes >Assignee: Tamas Penzes > > Dependency versions are defined in ivy.xml, which is suboptimal since it is > hard to override them from a script. > If we defined the versions in the main build.xml (just as we do with > audience-annotations.version) and use variables in ivy.xml then we could > easily override the versions with creating a build.properties file, which > mechanism is already built in. > This way the dependency versions could be replaced by sed or any simple > command line tool. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ZOOKEEPER-2887) define dependency versions in build.xml to be easily overridden
Tamas Penzes created ZOOKEEPER-2887: --- Summary: define dependency versions in build.xml to be easily overridden Key: ZOOKEEPER-2887 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2887 Project: ZooKeeper Issue Type: Improvement Components: build Reporter: Tamas Penzes Assignee: Tamas Penzes Dependency versions are defined in ivy.xml, which is suboptimal since it is hard to override them from a script. If we defined the versions in the main build.xml and use variables in ivy.xml then we could easily override the versions with creating a build.properties file, which mechanism is already built in. This way the dependency versions could be replaced by sed or any simple command line tool. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ZOOKEEPER-1363) Categorise unit tests by 'test-commit', 'full-test' etc
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Fenes reassigned ZOOKEEPER-1363: - Assignee: Mark Fenes > Categorise unit tests by 'test-commit', 'full-test' etc > --- > > Key: ZOOKEEPER-1363 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1363 > Project: ZooKeeper > Issue Type: Improvement > Components: build, tests >Reporter: Henry Robinson >Assignee: Mark Fenes > Labels: newbie > > As discussed on the list, it would be good to split the Java test suite into > categories so that it's easy to run a small set of unit tests against a > patch, and to leave Jenkins to run the full suite of stress tests etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Success: ZOOKEEPER- PreCommit Build #989
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/989/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 72.87 MB...] [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 74 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/989//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/989//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/989//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 5ae3a0ff66845986bd6dbfe2a2f263c6ebd620f8 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: ‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess’ and ‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess’ are the same file BUILD SUCCESSFUL Total time: 17 minutes 57 seconds Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2630 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Success Sending email for trigger: Success Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (ZOOKEEPER-2630) Use interface type instead of implementation type when appropriate.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150490#comment-16150490 ] Hadoop QA commented on ZOOKEEPER-2630: -- +1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 74 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/989//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/989//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/989//console This message is automatically generated. > Use interface type instead of implementation type when appropriate. > --- > > Key: ZOOKEEPER-2630 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2630 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Michael Han >Assignee: Tamas Penzes >Priority: Trivial > Labels: newbie, refactoring > > There are a couple of places in code base where we declare a field / variable > as implementation type (i.e. HashMap, HashSet) instead of interface type > (i.e. Map, Set), while in other places we do the opposite by declaring as > interface type. A quick check indicates that most if not all of these places > could be updated so we have a consistent style over the code base (prefer > using interface type), which is also a good coding style to stick per best > practice. > See more info on https://github.com/apache/zookeeper/pull/102 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2809) Unnecessary stack-trace in server when the client disconnect unexpectedly
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150470#comment-16150470 ] Hadoop QA commented on ZOOKEEPER-2809: -- +1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/987//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/987//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/987//console This message is automatically generated. > Unnecessary stack-trace in server when the client disconnect unexpectedly > - > > Key: ZOOKEEPER-2809 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2809 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.4.8 >Reporter: Paul Millar >Assignee: Mark Fenes >Priority: Minor > Fix For: 3.5.0 > > > In ZK 3.4.x, if the client disconnects unexpectedly then the server logs this > with a stack-trace (see > src/java/main/org/apache/zookeeper/server/NIOServerCnxn.java:356). > This is unfortunate as we are using an embedded ZK server in our project (in > a test environment) and we consider all stack-traces as bugs. > I noticed that ZK 3.5 and later no longer log a stack-trace. This change is > due to commit 6206b495 (in branch-3.5), which adds ZOOKEEPER-1504 and seems > to fix this issue almost as a side-effect; a similar change in master has the > same effect. > I was wondering if the change in how EndOfStreamException is logged (i.e., > logging the message without a stack-trace) could be back-ported to 3.4 > branch, so could be included in the next 3.4 release. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Success: ZOOKEEPER- PreCommit Build #987
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/987/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 33.30 MB...] [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +0 tests included. The patch appears to be a documentation patch that doesn't require tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/987//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/987//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/987//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 690d086ec2389f45438634d49f3c762326d487b8 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD SUCCESSFUL Total time: 54 minutes 20 seconds Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2809 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Success Sending email for trigger: Success Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (ZOOKEEPER-2630) Use interface type instead of implementation type when appropriate.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150467#comment-16150467 ] ASF GitHub Bot commented on ZOOKEEPER-2630: --- Github user tamaashu commented on the issue: https://github.com/apache/zookeeper/pull/354 fixed the issues with the imports > Use interface type instead of implementation type when appropriate. > --- > > Key: ZOOKEEPER-2630 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2630 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Michael Han >Assignee: Tamas Penzes >Priority: Trivial > Labels: newbie, refactoring > > There are a couple of places in code base where we declare a field / variable > as implementation type (i.e. HashMap, HashSet) instead of interface type > (i.e. Map, Set), while in other places we do the opposite by declaring as > interface type. A quick check indicates that most if not all of these places > could be updated so we have a consistent style over the code base (prefer > using interface type), which is also a good coding style to stick per best > practice. > See more info on https://github.com/apache/zookeeper/pull/102 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] zookeeper issue #354: ZOOKEEPER-2630: Use interface type instead of implemen...
Github user tamaashu commented on the issue: https://github.com/apache/zookeeper/pull/354 fixed the issues with the imports --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Success: ZOOKEEPER- PreCommit Build #988
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/988/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 71.22 MB...] [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +0 tests included. The patch appears to be a documentation patch that doesn't require tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/988//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/988//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/988//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 9ec44764a00203e8a578fed21231fe08a46d0fc8 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD SUCCESSFUL Total time: 17 minutes 36 seconds Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2572 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Success Sending email for trigger: Success Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (ZOOKEEPER-2572) Potential resource leak in FileTxnLog.truncate
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150449#comment-16150449
]
Hadoop QA commented on ZOOKEEPER-2572:
--
+1 overall. GitHub Pull Request Build
+1 @author. The patch does not contain any @author tags.
+0 tests included. The patch appears to be a documentation patch that
doesn't require tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/988//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/988//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/988//console
This message is automatically generated.
> Potential resource leak in FileTxnLog.truncate
> --
>
> Key: ZOOKEEPER-2572
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2572
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.4.9, 3.5.2
>Reporter: Michael Han
>Assignee: gaoshu
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> In FileTxnLog.truncate, we have:
> {code}
> public boolean truncate(long zxid) throws IOException {
> FileTxnIterator itr = null;
> try {
> itr = new FileTxnIterator(this.logDir, zxid);
> PositionInputStream input = itr.inputStream;
> if(input == null) {
> throw new IOException("No log files found to truncate! This
> could " +
> "happen if you still have snapshots from an old setup
> or " +
> "log files were deleted accidentally or dataLogDir
> was changed in zoo.cfg.");
> }
> long pos = input.getPosition();
> // now, truncate at the current position
> RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
> raf.setLength(pos);
> raf.close();
> while(itr.goToNextLog()) {
> if (!itr.logFile.delete()) {
> LOG.warn("Unable to truncate {}", itr.logFile);
> }
> }
> } finally {
> close(itr);
> }
> return true;
> }
> {code}
> {{raf}} here can be potentially in a state of not closed after leaving the
> method, if there is an (IO) exception thrown from setLength.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (ZOOKEEPER-2572) Potential resource leak in FileTxnLog.truncate
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
gaoshu reassigned ZOOKEEPER-2572:
-
Assignee: gaoshu
> Potential resource leak in FileTxnLog.truncate
> --
>
> Key: ZOOKEEPER-2572
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2572
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.4.9, 3.5.2
>Reporter: Michael Han
>Assignee: gaoshu
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> In FileTxnLog.truncate, we have:
> {code}
> public boolean truncate(long zxid) throws IOException {
> FileTxnIterator itr = null;
> try {
> itr = new FileTxnIterator(this.logDir, zxid);
> PositionInputStream input = itr.inputStream;
> if(input == null) {
> throw new IOException("No log files found to truncate! This
> could " +
> "happen if you still have snapshots from an old setup
> or " +
> "log files were deleted accidentally or dataLogDir
> was changed in zoo.cfg.");
> }
> long pos = input.getPosition();
> // now, truncate at the current position
> RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
> raf.setLength(pos);
> raf.close();
> while(itr.goToNextLog()) {
> if (!itr.logFile.delete()) {
> LOG.warn("Unable to truncate {}", itr.logFile);
> }
> }
> } finally {
> close(itr);
> }
> return true;
> }
> {code}
> {{raf}} here can be potentially in a state of not closed after leaving the
> method, if there is an (IO) exception thrown from setLength.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2572) Potential resource leak in FileTxnLog.truncate
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150427#comment-16150427
]
ASF GitHub Bot commented on ZOOKEEPER-2572:
---
GitHub user bitgaoshu opened a pull request:
https://github.com/apache/zookeeper/pull/356
ZOOKEEPER-2572: Fix potential resource leak in FileTxnLog.truncate
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bitgaoshu/zookeeper fix/ZOOKEEPER-2572
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zookeeper/pull/356.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #356
commit 4c21eb8bc56ed4d54dac8d1a16f054f08f2efc2e
Author: bitgaoshu
Date: 2017-08-31T12:24:40Z
ZOOKEEPER-2572: Fix potential resource leak in FileTxnLog.truncate
> Potential resource leak in FileTxnLog.truncate
> --
>
> Key: ZOOKEEPER-2572
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2572
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.4.9, 3.5.2
>Reporter: Michael Han
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> In FileTxnLog.truncate, we have:
> {code}
> public boolean truncate(long zxid) throws IOException {
> FileTxnIterator itr = null;
> try {
> itr = new FileTxnIterator(this.logDir, zxid);
> PositionInputStream input = itr.inputStream;
> if(input == null) {
> throw new IOException("No log files found to truncate! This
> could " +
> "happen if you still have snapshots from an old setup
> or " +
> "log files were deleted accidentally or dataLogDir
> was changed in zoo.cfg.");
> }
> long pos = input.getPosition();
> // now, truncate at the current position
> RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
> raf.setLength(pos);
> raf.close();
> while(itr.goToNextLog()) {
> if (!itr.logFile.delete()) {
> LOG.warn("Unable to truncate {}", itr.logFile);
> }
> }
> } finally {
> close(itr);
> }
> return true;
> }
> {code}
> {{raf}} here can be potentially in a state of not closed after leaving the
> method, if there is an (IO) exception thrown from setLength.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] zookeeper pull request #356: ZOOKEEPER-2572: Fix potential resource leak in ...
GitHub user bitgaoshu opened a pull request: https://github.com/apache/zookeeper/pull/356 ZOOKEEPER-2572: Fix potential resource leak in FileTxnLog.truncate You can merge this pull request into a Git repository by running: $ git pull https://github.com/bitgaoshu/zookeeper fix/ZOOKEEPER-2572 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/356.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #356 commit 4c21eb8bc56ed4d54dac8d1a16f054f08f2efc2e Author: bitgaoshu Date: 2017-08-31T12:24:40Z ZOOKEEPER-2572: Fix potential resource leak in FileTxnLog.truncate --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (ZOOKEEPER-2809) Unnecessary stack-trace in server when the client disconnect unexpectedly
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150411#comment-16150411 ] ASF GitHub Bot commented on ZOOKEEPER-2809: --- GitHub user mfenes opened a pull request: https://github.com/apache/zookeeper/pull/355 ZOOKEEPER-2809: Unnecessary stack-trace in server when the client dis… Unnecessary stack-trace in server when the client disconnects unexpectedly. Backport from master, branch-3.5 to branch-3.4. Removes unnecessary stack traces from the catch blocks of method doIO in NIOServerCnxn. For EndOfStreamException stack trace is replaced with logging only the message and also contains the removal of stack traces for exceptions CancelledKeyException and IOException as per commit 6206b495 referenced in the ticket. This change is necessary as there are projects which consider all stack traces as bugs. For CancelledKeyException and IOException developers are still able to see stack traces at log level Debug. This change is in sync with master and branch-3.5. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mfenes/zookeeper ZOOKEEPER-2809 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/355.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #355 commit 81cd3cc42d85371bd24e427bedab1740695be819 Author: Mark Fenes Date: 2017-08-31T12:11:09Z ZOOKEEPER-2809: Unnecessary stack-trace in server when the client disconnect unexpectedly > Unnecessary stack-trace in server when the client disconnect unexpectedly > - > > Key: ZOOKEEPER-2809 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2809 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.4.8 >Reporter: Paul Millar >Assignee: Mark Fenes >Priority: Minor > Fix For: 3.5.0 > > > In ZK 3.4.x, if the client disconnects unexpectedly then the server logs this > with a stack-trace (see > src/java/main/org/apache/zookeeper/server/NIOServerCnxn.java:356). > This is unfortunate as we are using an embedded ZK server in our project (in > a test environment) and we consider all stack-traces as bugs. > I noticed that ZK 3.5 and later no longer log a stack-trace. This change is > due to commit 6206b495 (in branch-3.5), which adds ZOOKEEPER-1504 and seems > to fix this issue almost as a side-effect; a similar change in master has the > same effect. > I was wondering if the change in how EndOfStreamException is logged (i.e., > logging the message without a stack-trace) could be back-ported to 3.4 > branch, so could be included in the next 3.4 release. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] zookeeper pull request #355: ZOOKEEPER-2809: Unnecessary stack-trace in serv...
GitHub user mfenes opened a pull request: https://github.com/apache/zookeeper/pull/355 ZOOKEEPER-2809: Unnecessary stack-trace in server when the client dis⦠Unnecessary stack-trace in server when the client disconnects unexpectedly. Backport from master, branch-3.5 to branch-3.4. Removes unnecessary stack traces from the catch blocks of method doIO in NIOServerCnxn. For EndOfStreamException stack trace is replaced with logging only the message and also contains the removal of stack traces for exceptions CancelledKeyException and IOException as per commit 6206b495 referenced in the ticket. This change is necessary as there are projects which consider all stack traces as bugs. For CancelledKeyException and IOException developers are still able to see stack traces at log level Debug. This change is in sync with master and branch-3.5. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mfenes/zookeeper ZOOKEEPER-2809 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/355.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #355 commit 81cd3cc42d85371bd24e427bedab1740695be819 Author: Mark Fenes Date: 2017-08-31T12:11:09Z ZOOKEEPER-2809: Unnecessary stack-trace in server when the client disconnect unexpectedly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (ZOOKEEPER-2630) Use interface type instead of implementation type when appropriate.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150406#comment-16150406 ] ASF GitHub Bot commented on ZOOKEEPER-2630: --- Github user tamaashu commented on a diff in the pull request: https://github.com/apache/zookeeper/pull/354#discussion_r136557551 --- Diff: src/java/main/org/apache/jute/compiler/JRecord.java --- @@ -21,9 +21,7 @@ import java.io.File; import java.io.FileWriter; import java.io.IOException; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.Iterator; +import java.util.*; --- End diff -- working on the fix in my next commit > Use interface type instead of implementation type when appropriate. > --- > > Key: ZOOKEEPER-2630 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2630 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Michael Han >Assignee: Tamas Penzes >Priority: Trivial > Labels: newbie, refactoring > > There are a couple of places in code base where we declare a field / variable > as implementation type (i.e. HashMap, HashSet) instead of interface type > (i.e. Map, Set), while in other places we do the opposite by declaring as > interface type. A quick check indicates that most if not all of these places > could be updated so we have a consistent style over the code base (prefer > using interface type), which is also a good coding style to stick per best > practice. > See more info on https://github.com/apache/zookeeper/pull/102 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
ZooKeeper-trunk-jdk8 - Build # 1187 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk-jdk8/1187/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 65.87 MB...]
[junit] java.net.ConnectException: Connection refused:
127.0.0.1/127.0.0.1:19350
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit] at java.lang.Thread.run(Thread.java:748)
[junit] 2017-09-01 11:56:38,664 [myid:] - INFO [New I/O boss
#2601:ClientCnxnSocketNetty@208] - channel is told closing
[junit] 2017-09-01 11:56:38,664 [myid:127.0.0.1:19350] - INFO
[main-SendThread(127.0.0.1:19350):ClientCnxn$SendThread@1231] - channel for
sessionid 0x105b6b7e4be0001 is lost, closing socket connection and attempting
reconnect
[junit] 2017-09-01 11:56:38,741 [myid:127.0.0.1:19365] - INFO
[main-SendThread(127.0.0.1:19365):ClientCnxn$SendThread@1113] - Opening socket
connection to server 127.0.0.1/127.0.0.1:19365. Will not attempt to
authenticate using SASL (unknown error)
[junit] 2017-09-01 11:56:38,742 [myid:] - INFO [New I/O boss
#3791:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
[junit] java.net.ConnectException: Connection refused:
127.0.0.1/127.0.0.1:19365
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit] at java.lang.Thread.run(Thread.java:748)
[junit] 2017-09-01 11:56:38,742 [myid:] - WARN [New I/O boss
#3791:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id:
0xde073599] EXCEPTION: java.net.ConnectException: Connection refused:
127.0.0.1/127.0.0.1:19365
[junit] java.net.ConnectException: Connection refused:
127.0.0.1/127.0.0.1:19365
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit] at java.lang.Thread.run(Thread.java:748)
[junit] 2017-09-01 11:56:38,743 [myid:]
[GitHub] zookeeper pull request #354: ZOOKEEPER-2630: Use interface type instead of i...
Github user tamaashu commented on a diff in the pull request: https://github.com/apache/zookeeper/pull/354#discussion_r136557551 --- Diff: src/java/main/org/apache/jute/compiler/JRecord.java --- @@ -21,9 +21,7 @@ import java.io.File; import java.io.FileWriter; import java.io.IOException; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.Iterator; +import java.util.*; --- End diff -- working on the fix in my next commit --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (ZOOKEEPER-2572) Potential resource leak in FileTxnLog.truncate
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150402#comment-16150402
]
gaoshu commented on ZOOKEEPER-2572:
---
In addition, I think the {{itr.logFile.delete() }} can be replaced by
{{Files.delete(itr.logFile.toPath());}}. If failed, Files.delete can throw an
exception, which this is useful for error reporting and to diagnosed why a file
cannot be deleted. Then the function will return void if successful, or throw
an exception. Then we can log the detailed error and rethrow it.
> Potential resource leak in FileTxnLog.truncate
> --
>
> Key: ZOOKEEPER-2572
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2572
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.4.9, 3.5.2
>Reporter: Michael Han
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> In FileTxnLog.truncate, we have:
> {code}
> public boolean truncate(long zxid) throws IOException {
> FileTxnIterator itr = null;
> try {
> itr = new FileTxnIterator(this.logDir, zxid);
> PositionInputStream input = itr.inputStream;
> if(input == null) {
> throw new IOException("No log files found to truncate! This
> could " +
> "happen if you still have snapshots from an old setup
> or " +
> "log files were deleted accidentally or dataLogDir
> was changed in zoo.cfg.");
> }
> long pos = input.getPosition();
> // now, truncate at the current position
> RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
> raf.setLength(pos);
> raf.close();
> while(itr.goToNextLog()) {
> if (!itr.logFile.delete()) {
> LOG.warn("Unable to truncate {}", itr.logFile);
> }
> }
> } finally {
> close(itr);
> }
> return true;
> }
> {code}
> {{raf}} here can be potentially in a state of not closed after leaving the
> method, if there is an (IO) exception thrown from setLength.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ZOOKEEPER-2572) Potential resource leak in FileTxnLog.truncate
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150402#comment-16150402
]
gaoshu edited comment on ZOOKEEPER-2572 at 9/1/17 11:54 AM:
In addition, I think the {{itr.logFile.delete()}} can be replaced by
{{Files.delete(itr.logFile.toPath())}}. If failed, Files.delete can throw an
exception, which this is useful for error reporting and to diagnosed why a file
cannot be deleted. Then the function will return void if successful, or throw
an exception. Then we can log the detailed error and rethrow it. But the
Files.delete is available for jdk1.7 +
was (Author: gaoshu):
In addition, I think the {{itr.logFile.delete()}} can be replaced by
{{Files.delete(itr.logFile.toPath())}}. If failed, Files.delete can throw an
exception, which this is useful for error reporting and to diagnosed why a file
cannot be deleted. Then the function will return void if successful, or throw
an exception. Then we can log the detailed error and rethrow it.
> Potential resource leak in FileTxnLog.truncate
> --
>
> Key: ZOOKEEPER-2572
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2572
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.4.9, 3.5.2
>Reporter: Michael Han
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> In FileTxnLog.truncate, we have:
> {code}
> public boolean truncate(long zxid) throws IOException {
> FileTxnIterator itr = null;
> try {
> itr = new FileTxnIterator(this.logDir, zxid);
> PositionInputStream input = itr.inputStream;
> if(input == null) {
> throw new IOException("No log files found to truncate! This
> could " +
> "happen if you still have snapshots from an old setup
> or " +
> "log files were deleted accidentally or dataLogDir
> was changed in zoo.cfg.");
> }
> long pos = input.getPosition();
> // now, truncate at the current position
> RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
> raf.setLength(pos);
> raf.close();
> while(itr.goToNextLog()) {
> if (!itr.logFile.delete()) {
> LOG.warn("Unable to truncate {}", itr.logFile);
> }
> }
> } finally {
> close(itr);
> }
> return true;
> }
> {code}
> {{raf}} here can be potentially in a state of not closed after leaving the
> method, if there is an (IO) exception thrown from setLength.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (ZOOKEEPER-2572) Potential resource leak in FileTxnLog.truncate
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150402#comment-16150402
]
gaoshu edited comment on ZOOKEEPER-2572 at 9/1/17 11:52 AM:
In addition, I think the {{itr.logFile.delete()}} can be replaced by
{{Files.delete(itr.logFile.toPath())}}. If failed, Files.delete can throw an
exception, which this is useful for error reporting and to diagnosed why a file
cannot be deleted. Then the function will return void if successful, or throw
an exception. Then we can log the detailed error and rethrow it.
was (Author: gaoshu):
In addition, I think the {{itr.logFile.delete() }} can be replaced by
{{Files.delete(itr.logFile.toPath());}}. If failed, Files.delete can throw an
exception, which this is useful for error reporting and to diagnosed why a file
cannot be deleted. Then the function will return void if successful, or throw
an exception. Then we can log the detailed error and rethrow it.
> Potential resource leak in FileTxnLog.truncate
> --
>
> Key: ZOOKEEPER-2572
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2572
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.4.9, 3.5.2
>Reporter: Michael Han
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> In FileTxnLog.truncate, we have:
> {code}
> public boolean truncate(long zxid) throws IOException {
> FileTxnIterator itr = null;
> try {
> itr = new FileTxnIterator(this.logDir, zxid);
> PositionInputStream input = itr.inputStream;
> if(input == null) {
> throw new IOException("No log files found to truncate! This
> could " +
> "happen if you still have snapshots from an old setup
> or " +
> "log files were deleted accidentally or dataLogDir
> was changed in zoo.cfg.");
> }
> long pos = input.getPosition();
> // now, truncate at the current position
> RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
> raf.setLength(pos);
> raf.close();
> while(itr.goToNextLog()) {
> if (!itr.logFile.delete()) {
> LOG.warn("Unable to truncate {}", itr.logFile);
> }
> }
> } finally {
> close(itr);
> }
> return true;
> }
> {code}
> {{raf}} here can be potentially in a state of not closed after leaving the
> method, if there is an (IO) exception thrown from setLength.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Issue Comment Deleted] (ZOOKEEPER-2572) Potential resource leak in FileTxnLog.truncate
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
gaoshu updated ZOOKEEPER-2572:
--
Comment: was deleted
(was: IMO, the exception happened in FileTxnLog.truncate can be got caught in
that function. log exception, release resource and return false. Then the
dead code will be work. )
> Potential resource leak in FileTxnLog.truncate
> --
>
> Key: ZOOKEEPER-2572
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2572
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.4.9, 3.5.2
>Reporter: Michael Han
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> In FileTxnLog.truncate, we have:
> {code}
> public boolean truncate(long zxid) throws IOException {
> FileTxnIterator itr = null;
> try {
> itr = new FileTxnIterator(this.logDir, zxid);
> PositionInputStream input = itr.inputStream;
> if(input == null) {
> throw new IOException("No log files found to truncate! This
> could " +
> "happen if you still have snapshots from an old setup
> or " +
> "log files were deleted accidentally or dataLogDir
> was changed in zoo.cfg.");
> }
> long pos = input.getPosition();
> // now, truncate at the current position
> RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
> raf.setLength(pos);
> raf.close();
> while(itr.goToNextLog()) {
> if (!itr.logFile.delete()) {
> LOG.warn("Unable to truncate {}", itr.logFile);
> }
> }
> } finally {
> close(itr);
> }
> return true;
> }
> {code}
> {{raf}} here can be potentially in a state of not closed after leaving the
> method, if there is an (IO) exception thrown from setLength.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
ZooKeeper_branch35_openjdk7 - Build # 655 - Failure
See https://builds.apache.org/job/ZooKeeper_branch35_openjdk7/655/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 34.76 MB...] [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit] at java.lang.Thread.run(Thread.java:745) [junit] Thread New I/O worker #460 [junit] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) [junit] at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) [junit] at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) [junit] at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) [junit] at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) [junit] at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68) [junit] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434) [junit] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212) [junit] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) [junit] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [junit] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [junit] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [junit] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit] at java.lang.Thread.run(Thread.java:745) [junit] Thread New I/O worker #131 [junit] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) ERROR: H16 is offline; cannot locate jdk-1.7.0_79 (unlimited security) [junit] at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) [junit] at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) [junit] at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) [junit] at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) [junit] at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68) [junit] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434) [junit] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212) [junit] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) [junit] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [junit] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [junit] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [junit] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit] at java.lang.Thread.run(Thread.java:745) [junit] Thread New I/O worker #278 [junit] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) [junit] at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) [junit] at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) [junit] at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) [junit] at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) [junit] at org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:68) [junit] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(AbstractNioSelector.java:434) [junit] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:212) [junit] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) [junit] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [junit] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [junit] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [junit] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit] at java.lang.Thread.run(Thread.java:745) [junit] Thread New I/O worker #127 [junit] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) [junit] at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) [junit] at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) [junit]
