ZooKeeper_branch34_jdk7 - Build # 1740 - Still Failing
See https://builds.apache.org/job/ZooKeeper_branch34_jdk7/1740/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 31.61 MB...] [junit] 2017-11-29 02:41:14,399 [myid:] - INFO [main:ZooKeeperServer@501] - shutting down [junit] 2017-11-29 02:41:14,400 [myid:] - INFO [main:SessionTrackerImpl@226] - Shutting down [junit] 2017-11-29 02:41:14,400 [myid:] - INFO [main:PrepRequestProcessor@769] - Shutting down [junit] 2017-11-29 02:41:14,400 [myid:] - INFO [main:SyncRequestProcessor@208] - Shutting down [junit] 2017-11-29 02:41:14,400 [myid:] - INFO [ProcessThread(sid:0 cport:11221)::PrepRequestProcessor@144] - PrepRequestProcessor exited loop! [junit] 2017-11-29 02:41:14,401 [myid:] - INFO [SyncThread:0:SyncRequestProcessor@186] - SyncRequestProcessor exited! [junit] 2017-11-29 02:41:14,401 [myid:] - INFO [main:FinalRequestProcessor@403] - shutdown of request processor complete [junit] 2017-11-29 02:41:14,402 [myid:] - INFO [main:FourLetterWordMain@65] - connecting to 127.0.0.1 11221 [junit] 2017-11-29 02:41:14,402 [myid:] - INFO [main:JMXEnv@147] - ensureOnly:[] [junit] 2017-11-29 02:41:14,404 [myid:] - INFO [main:ClientBase@489] - STARTING server [junit] 2017-11-29 02:41:14,404 [myid:] - INFO [main:ClientBase@410] - CREATING server instance 127.0.0.1:11221 [junit] 2017-11-29 02:41:14,404 [myid:] - INFO [main:ServerCnxnFactory@117] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory [junit] 2017-11-29 02:41:14,404 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11221 [junit] 2017-11-29 02:41:14,405 [myid:] - INFO [main:ClientBase@385] - STARTING server instance 127.0.0.1:11221 [junit] 2017-11-29 02:41:14,405 [myid:] - INFO [main:ZooKeeperServer@173] - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 6 datadir /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/build/test/tmp/test6336636014720291422.junit.dir/version-2 snapdir /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/build/test/tmp/test6336636014720291422.junit.dir/version-2 [junit] 2017-11-29 02:41:14,410 [myid:] - INFO [main:FourLetterWordMain@65] - connecting to 127.0.0.1 11221 [junit] 2017-11-29 02:41:14,411 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@215] - Accepted socket connection from /127.0.0.1:37381 [junit] 2017-11-29 02:41:14,411 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@892] - Processing stat command from /127.0.0.1:37381 [junit] 2017-11-29 02:41:14,412 [myid:] - INFO [Thread-4:NIOServerCnxn$StatCommand@683] - Stat command output [junit] 2017-11-29 02:41:14,412 [myid:] - INFO [Thread-4:NIOServerCnxn@1040] - Closed socket connection for client /127.0.0.1:37381 (no session established for client) [junit] 2017-11-29 02:41:14,412 [myid:] - INFO [main:JMXEnv@230] - ensureParent:[InMemoryDataTree, StandaloneServer_port] [junit] 2017-11-29 02:41:14,415 [myid:] - INFO [main:JMXEnv@247] - expect:InMemoryDataTree [junit] 2017-11-29 02:41:14,415 [myid:] - INFO [main:JMXEnv@251] - found:InMemoryDataTree org.apache.ZooKeeperService:name0=StandaloneServer_port11221,name1=InMemoryDataTree [junit] 2017-11-29 02:41:14,415 [myid:] - INFO [main:JMXEnv@247] - expect:StandaloneServer_port [junit] 2017-11-29 02:41:14,415 [myid:] - INFO [main:JMXEnv@251] - found:StandaloneServer_port org.apache.ZooKeeperService:name0=StandaloneServer_port11221 [junit] 2017-11-29 02:41:14,416 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 33046 [junit] 2017-11-29 02:41:14,416 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 20 [junit] 2017-11-29 02:41:14,416 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testQuota [junit] 2017-11-29 02:41:14,416 [myid:] - INFO [main:ClientBase@566] - tearDown starting [junit] 2017-11-29 02:41:14,483 [myid:] - INFO [main:ZooKeeper@687] - Session: 0x101a31a30e8 closed [junit] 2017-11-29 02:41:14,483 [myid:] - INFO [main:ClientBase@536] - STOPPING server [junit] 2017-11-29 02:41:14,484 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@520] - EventThread shut down for session: 0x101a31a30e8 [junit] 2017-11-29 02:41:14,484 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@242] - NIOServerCnxn factory exited run method [junit] 2017-11-29 02:41:14,484 [myid:] - INFO [main:ZooKeeperServer@501] - shutting down [junit] 2017-11-29 02:41:14,485 [myid:] - INFO [main:SessionTrackerImpl@226] - Shutting down [junit] 2017-11-29 02:41:14,485 [myid:] - INFO [main:PrepRequestProcessor@769
[jira] [Commented] (ZOOKEEPER-2901) Session ID that is negative causes mis-calculation of Ephemeral Type
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269842#comment-16269842
]
ASF GitHub Bot commented on ZOOKEEPER-2901:
---
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/377#discussion_r153668817
--- Diff: src/java/main/org/apache/zookeeper/server/EphemeralType.java ---
@@ -40,19 +39,21 @@
TTL;
public static final long CONTAINER_EPHEMERAL_OWNER = Long.MIN_VALUE;
-public static final long MAX_TTL = 0x0fffL;
-public static final long TTL_MASK = 0x8000L;
+public static final long MAX_TTL = 0x00ffL;
+public static final long TTL_MASK = 0xff00L;
+public static final long MAX_TTL_SERVER_ID = 0xfe; // 254
public static EphemeralType get(long ephemeralOwner) {
if (ephemeralOwner == CONTAINER_EPHEMERAL_OWNER) {
return CONTAINER;
}
-if (ephemeralOwner < 0) {
+if ((ephemeralOwner & TTL_MASK) == TTL_MASK) {
--- End diff --
In particular you need to add a bunch more testing to ensure this case is
working properly.
> Session ID that is negative causes mis-calculation of Ephemeral Type
>
>
> Key: ZOOKEEPER-2901
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2901
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.5.3
> Environment: Running 3.5.3-beta in Docker container
>Reporter: Mark Johnson
>Assignee: Jordan Zimmerman
>Priority: Blocker
>
> In the code that determines the EphemeralType it is looking at the owner
> (which is the client ID or connection ID):
> EphemeralType.java:
>public static EphemeralType get(long ephemeralOwner) {
>if (ephemeralOwner == CONTAINER_EPHEMERAL_OWNER) {
>return CONTAINER;
>}
>if (ephemeralOwner < 0) {
>return TTL;
>}
>return (ephemeralOwner == 0) ? VOID : NORMAL;
>}
> However my connection ID is:
> header.getClientId(): -720548323429908480
> This causes the code to think this is a TTL Ephemeral node instead of a
> NORMAL Ephemeral node.
> This also explains why this is random - if my client ID is non-negative
> then the node gets added correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] zookeeper pull request #377: [ZOOKEEPER-2901] TTL Nodes don't work with Serv...
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/377#discussion_r153668817
--- Diff: src/java/main/org/apache/zookeeper/server/EphemeralType.java ---
@@ -40,19 +39,21 @@
TTL;
public static final long CONTAINER_EPHEMERAL_OWNER = Long.MIN_VALUE;
-public static final long MAX_TTL = 0x0fffL;
-public static final long TTL_MASK = 0x8000L;
+public static final long MAX_TTL = 0x00ffL;
+public static final long TTL_MASK = 0xff00L;
+public static final long MAX_TTL_SERVER_ID = 0xfe; // 254
public static EphemeralType get(long ephemeralOwner) {
if (ephemeralOwner == CONTAINER_EPHEMERAL_OWNER) {
return CONTAINER;
}
-if (ephemeralOwner < 0) {
+if ((ephemeralOwner & TTL_MASK) == TTL_MASK) {
--- End diff --
In particular you need to add a bunch more testing to ensure this case is
working properly.
---
[jira] [Commented] (ZOOKEEPER-2901) Session ID that is negative causes mis-calculation of Ephemeral Type
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269839#comment-16269839
]
ASF GitHub Bot commented on ZOOKEEPER-2901:
---
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/377#discussion_r153668593
--- Diff: src/java/main/org/apache/zookeeper/server/EphemeralType.java ---
@@ -40,19 +39,21 @@
TTL;
public static final long CONTAINER_EPHEMERAL_OWNER = Long.MIN_VALUE;
-public static final long MAX_TTL = 0x0fffL;
-public static final long TTL_MASK = 0x8000L;
+public static final long MAX_TTL = 0x00ffL;
+public static final long TTL_MASK = 0xff00L;
+public static final long MAX_TTL_SERVER_ID = 0xfe; // 254
public static EphemeralType get(long ephemeralOwner) {
if (ephemeralOwner == CONTAINER_EPHEMERAL_OWNER) {
return CONTAINER;
}
-if (ephemeralOwner < 0) {
+if ((ephemeralOwner & TTL_MASK) == TTL_MASK) {
--- End diff --
I think this is broken. If, for example,
org.apache.zookeeper.server.DataTree#deleteNode calls this method with TTL
turned off, but using a server id of 255 it will end up calling this code
} else if (ephemeralType == EphemeralType.TTL) {
ttls.remove(path);
and the node will not be removed properly in deleteNode.
> Session ID that is negative causes mis-calculation of Ephemeral Type
>
>
> Key: ZOOKEEPER-2901
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2901
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.5.3
> Environment: Running 3.5.3-beta in Docker container
>Reporter: Mark Johnson
>Assignee: Jordan Zimmerman
>Priority: Blocker
>
> In the code that determines the EphemeralType it is looking at the owner
> (which is the client ID or connection ID):
> EphemeralType.java:
>public static EphemeralType get(long ephemeralOwner) {
>if (ephemeralOwner == CONTAINER_EPHEMERAL_OWNER) {
>return CONTAINER;
>}
>if (ephemeralOwner < 0) {
>return TTL;
>}
>return (ephemeralOwner == 0) ? VOID : NORMAL;
>}
> However my connection ID is:
> header.getClientId(): -720548323429908480
> This causes the code to think this is a TTL Ephemeral node instead of a
> NORMAL Ephemeral node.
> This also explains why this is random - if my client ID is non-negative
> then the node gets added correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] zookeeper pull request #377: [ZOOKEEPER-2901] TTL Nodes don't work with Serv...
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/377#discussion_r153668593
--- Diff: src/java/main/org/apache/zookeeper/server/EphemeralType.java ---
@@ -40,19 +39,21 @@
TTL;
public static final long CONTAINER_EPHEMERAL_OWNER = Long.MIN_VALUE;
-public static final long MAX_TTL = 0x0fffL;
-public static final long TTL_MASK = 0x8000L;
+public static final long MAX_TTL = 0x00ffL;
+public static final long TTL_MASK = 0xff00L;
+public static final long MAX_TTL_SERVER_ID = 0xfe; // 254
public static EphemeralType get(long ephemeralOwner) {
if (ephemeralOwner == CONTAINER_EPHEMERAL_OWNER) {
return CONTAINER;
}
-if (ephemeralOwner < 0) {
+if ((ephemeralOwner & TTL_MASK) == TTL_MASK) {
--- End diff --
I think this is broken. If, for example,
org.apache.zookeeper.server.DataTree#deleteNode calls this method with TTL
turned off, but using a server id of 255 it will end up calling this code
} else if (ephemeralType == EphemeralType.TTL) {
ttls.remove(path);
and the node will not be removed properly in deleteNode.
---
[jira] [Commented] (ZOOKEEPER-2949) SSL ServerName not set when using hostname, some proxies may failed to proxy the request.
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269750#comment-16269750
]
ASF GitHub Bot commented on ZOOKEEPER-2949:
---
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/423#discussion_r153658262
--- Diff: src/java/main/org/apache/zookeeper/ClientCnxnSocketNetty.java ---
@@ -340,13 +340,24 @@ public static Packet getInstance() {
return instance;
}
}
+
/**
* ZKClientPipelineFactory is the netty pipeline factory for this netty
* connection implementation.
*/
private class ZKClientPipelineFactory implements
ChannelPipelineFactory {
private SSLContext sslContext = null;
private SSLEngine sslEngine = null;
+private String host;
+private int port;
+
+public ZKClientPipelineFactory() {
--- End diff --
do we still need this constructor?
> SSL ServerName not set when using hostname, some proxies may failed to proxy
> the request.
> -
>
> Key: ZOOKEEPER-2949
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2949
> Project: ZooKeeper
> Issue Type: Bug
> Components: java client
>Affects Versions: 3.5.3
> Environment: In our environment, the zk clusters are all behind a
> proxy, the proxy decide to transfer the request from client based on the
> "ServerName" field in SSL Hello packet(the proxy served on SSL only). but the
> Hello packets that zk client sended do proxy do not contain the "ServerName"
> field in it. after inspect the codes, we have found that it is because that
> zk client did not specify the peerHost when initializing the SSLContext.
>Reporter: Feng Shaobao
> Fix For: 3.6.0
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> In our environment, the zk clusters are all behind a proxy, the proxy decide
> to transfer the request from client based on the "ServerName" field in SSL
> Hello packet(the proxy served on SSL only). but the Hello packets that zk
> client sended do proxy do not contain the "ServerName" field in it. after
> inspect the codes, we have found that it is because that zk client did not
> specify the peerHost when initializing the SSLContext.
> In the method initSSL of class ZKClientPipelineFactory, it initialize the
> SSLEngine like below:
> sslEngine = sslContext.createSSLEngine();
> Actually the sslContext provide another factory method that receives the
> hostName and port parameter.
> public final SSLEngine createSSLEngine(String hostName, int port)
> If we call this method to create the SSLEngine, then the proxy will know
> which zk cluster it really want to access.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] zookeeper pull request #423: ZOOKEEPER-2949: using hostname and port to crea...
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/423#discussion_r153658262
--- Diff: src/java/main/org/apache/zookeeper/ClientCnxnSocketNetty.java ---
@@ -340,13 +340,24 @@ public static Packet getInstance() {
return instance;
}
}
+
/**
* ZKClientPipelineFactory is the netty pipeline factory for this netty
* connection implementation.
*/
private class ZKClientPipelineFactory implements
ChannelPipelineFactory {
private SSLContext sslContext = null;
private SSLEngine sslEngine = null;
+private String host;
+private int port;
+
+public ZKClientPipelineFactory() {
--- End diff --
do we still need this constructor?
---
ZooKeeper-trunk - Build # 3635 - Failure
See https://builds.apache.org/job/ZooKeeper-trunk/3635/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 61.62 MB...]
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2017-11-28 23:26:48,566 [myid:] - INFO [New I/O boss
#7497:ClientCnxnSocketNetty@208] - channel is told closing
[junit] 2017-11-28 23:26:48,566 [myid:127.0.0.1:22043] - INFO
[main-SendThread(127.0.0.1:22043):ClientCnxn$SendThread@1231] - channel for
sessionid 0x1082e9f48c70001 is lost, closing socket connection and attempting
reconnect
[junit] 2017-11-28 23:26:48,625 [myid:127.0.0.1:22058] - INFO
[main-SendThread(127.0.0.1:22058):ClientCnxn$SendThread@1113] - Opening socket
connection to server 127.0.0.1/127.0.0.1:22058. Will not attempt to
authenticate using SASL (unknown error)
[junit] 2017-11-28 23:26:48,626 [myid:] - INFO [New I/O boss
#10927:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
[junit] java.net.ConnectException: Connection refused:
127.0.0.1/127.0.0.1:22058
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2017-11-28 23:26:48,627 [myid:] - WARN [New I/O boss
#10927:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id:
0x70c971a0] EXCEPTION: java.net.ConnectException: Connection refused:
127.0.0.1/127.0.0.1:22058
[junit] java.net.ConnectException: Connection refused:
127.0.0.1/127.0.0.1:22058
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2017-11-28 23:26:48,627 [myid:] - INFO [New I/O boss
#10927:ClientCnxnSocketNetty@208] - channel is told closing
[junit] 2017-11-28 23:26:48,627 [myid:127.0.0.1:22058] - INFO
[main-SendThread(127.0.0.1:22058):ClientCnxn$SendThread@1231] - channel for
sessionid 0x1082e9f8d82 is lost, closing socket connection and attempting
reconnect
fail.build.on.test.failure:
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1395: The
following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1276: The
following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1280: Tests
failed!
Total time: 10 minutes 2 seconds
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
[WARNINGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording fingerprints
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Publishing Javadoc
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269663#comment-16269663
]
ASF GitHub Bot commented on ZOOKEEPER-2924:
---
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153652218
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -18,77 +18,58 @@
package org.apache.zookeeper.test;
-import java.io.ByteArrayOutputStream;
import java.io.File;
-import java.io.FileInputStream;
import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.zookeeper.common.Time;
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.BinaryOutputArchive;
-import org.apache.jute.Record;
+
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException.NoNodeException;
-import org.apache.zookeeper.PortAssignment;
-import org.apache.zookeeper.ZKTestCase;
import org.apache.zookeeper.ZooDefs.Ids;
-import org.apache.zookeeper.ZooDefs.OpCode;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
-import org.apache.zookeeper.server.DataNode;
-import org.apache.zookeeper.server.DataTree;
import org.apache.zookeeper.server.ServerCnxnFactory;
import org.apache.zookeeper.server.SyncRequestProcessor;
import org.apache.zookeeper.server.ZooKeeperServer;
-import org.apache.zookeeper.server.persistence.FileHeader;
import org.apache.zookeeper.server.persistence.FileTxnLog;
import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
import org.apache.zookeeper.server.persistence.Util;
import org.apache.zookeeper.server.persistence.FileTxnLog.FileTxnIterator;
import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.MultiTxn;
-import org.apache.zookeeper.txn.Txn;
import org.apache.zookeeper.txn.TxnHeader;
+import org.eclipse.jetty.util.SocketAddressResolver;
+import org.junit.After;
import org.junit.Assert;
+import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-public class LoadFromLogTest extends ZKTestCase {
-private static final String HOST = "127.0.0.1:";
-private static final int CONNECTION_TIMEOUT = 3000;
+public class LoadFromLogTest extends ClientBase {
private static final int NUM_MESSAGES = 300;
protected static final Logger LOG =
LoggerFactory.getLogger(LoadFromLogTest.class);
// setting up the quorum has a transaction overhead for creating and
closing the session
private static final int TRANSACTION_OVERHEAD = 2;
private static final int TOTAL_TRANSACTIONS = NUM_MESSAGES +
TRANSACTION_OVERHEAD;
+@Before
+public void setUp() throws Exception {
+SyncRequestProcessor.setSnapCount(100);
--- End diff --
I think @phunt is mostly correct. There are a couple issues with
`snapCount`.
First, if want to change it while the syncProcessor thread has started,
shouldn't it be volatile?
Second, I disagree that it has "no effect" on a server that has already
been started but the effect is not the desired behavior. Taking the issue of
the `snapCount` field not being volatile out of the picture, the code in
`SyncProcessor` looks like:
```java
public void run() {
try {
int randRoll = r.nextInt(snapCount/2);
while (true) {
Request si = null;
//get a request
if (si != null) {
// track the number of records written to the log
if (zks.getZKDatabase().append(si)) {
logCount++;
if (logCount > (snapCount / 2 + randRoll)) {
randRoll = r.nextInt(snapCount/2);
```
So my reading is that if `snapCount` is changed from `previousSnapCount`
while the `SyncProcessor` is already running to `desiredSnapCount` it will take
at least `desiredSnapCount` and at worst `desiredSnapCount +
previousSnapCount/2` transactions for the "effective" `snapCount` to become
`desiredSnapCount` (again ignoring the volatility issue).
I agree with @phunt's solution.
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> U
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269664#comment-16269664
]
ASF GitHub Bot commented on ZOOKEEPER-2924:
---
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153636108
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -18,77 +18,58 @@
package org.apache.zookeeper.test;
-import java.io.ByteArrayOutputStream;
import java.io.File;
-import java.io.FileInputStream;
import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.zookeeper.common.Time;
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.BinaryOutputArchive;
-import org.apache.jute.Record;
+
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException.NoNodeException;
-import org.apache.zookeeper.PortAssignment;
-import org.apache.zookeeper.ZKTestCase;
import org.apache.zookeeper.ZooDefs.Ids;
-import org.apache.zookeeper.ZooDefs.OpCode;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
-import org.apache.zookeeper.server.DataNode;
-import org.apache.zookeeper.server.DataTree;
import org.apache.zookeeper.server.ServerCnxnFactory;
import org.apache.zookeeper.server.SyncRequestProcessor;
import org.apache.zookeeper.server.ZooKeeperServer;
-import org.apache.zookeeper.server.persistence.FileHeader;
import org.apache.zookeeper.server.persistence.FileTxnLog;
import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
import org.apache.zookeeper.server.persistence.Util;
import org.apache.zookeeper.server.persistence.FileTxnLog.FileTxnIterator;
import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.MultiTxn;
-import org.apache.zookeeper.txn.Txn;
import org.apache.zookeeper.txn.TxnHeader;
+import org.eclipse.jetty.util.SocketAddressResolver;
--- End diff --
nit: not sure this import is needed
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2924
> Project: ZooKeeper
> Issue Type: Bug
> Components: server, tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
> Labels: flaky, flaky-test
> Fix For: 3.4.12
>
>
> From https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1682/
> Same issue happens in jdk8 and jdk9 builds as well.
> Issue has already been fixed by
> https://issues.apache.org/jira/browse/ZOOKEEPER-2484 , but I believe that the
> root cause here is that test startup / cleanup code is included in the tests
> instead of using try-finally block or Before-After methods.
> As a consequence, when exception happens during test execution, ZK test
> server doesn't get shutdown properly and still listening on the port bound to
> the test class.
> As mentioned above there could be 2 approaches to address this:
> #1 Wrap cleanup code block with finally
> #2 Use JUnit's Before-After methods for initialization and cleanup
> Test where original issue happens:
> {noformat}
> ...
> [junit] 2017-10-12 15:05:20,135 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8c zxid:0x8d
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,137 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8d zxid:0x8e
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,139 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8e zxid:0x8f
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,142 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8f zxid:0x90
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,144 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x90 zxid:0x91
> txntype:-1 req$
>
[GitHub] zookeeper pull request #420: ZOOKEEPER-2924. Refactor tests of LoadFromLogTe...
Github user afine commented on a diff in the pull request: https://github.com/apache/zookeeper/pull/420#discussion_r153636108 --- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java --- @@ -18,77 +18,58 @@ package org.apache.zookeeper.test; -import java.io.ByteArrayOutputStream; import java.io.File; -import java.io.FileInputStream; import java.io.IOException; -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -import org.apache.zookeeper.common.Time; -import org.apache.jute.BinaryInputArchive; -import org.apache.jute.BinaryOutputArchive; -import org.apache.jute.Record; + import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException.NoNodeException; -import org.apache.zookeeper.PortAssignment; -import org.apache.zookeeper.ZKTestCase; import org.apache.zookeeper.ZooDefs.Ids; -import org.apache.zookeeper.ZooDefs.OpCode; import org.apache.zookeeper.ZooKeeper; import org.apache.zookeeper.data.Stat; -import org.apache.zookeeper.server.DataNode; -import org.apache.zookeeper.server.DataTree; import org.apache.zookeeper.server.ServerCnxnFactory; import org.apache.zookeeper.server.SyncRequestProcessor; import org.apache.zookeeper.server.ZooKeeperServer; -import org.apache.zookeeper.server.persistence.FileHeader; import org.apache.zookeeper.server.persistence.FileTxnLog; import org.apache.zookeeper.server.persistence.FileTxnSnapLog; import org.apache.zookeeper.server.persistence.Util; import org.apache.zookeeper.server.persistence.FileTxnLog.FileTxnIterator; import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator; -import org.apache.zookeeper.txn.CreateTxn; -import org.apache.zookeeper.txn.DeleteTxn; -import org.apache.zookeeper.txn.MultiTxn; -import org.apache.zookeeper.txn.Txn; import org.apache.zookeeper.txn.TxnHeader; +import org.eclipse.jetty.util.SocketAddressResolver; --- End diff -- nit: not sure this import is needed ---
[GitHub] zookeeper pull request #420: ZOOKEEPER-2924. Refactor tests of LoadFromLogTe...
Github user afine commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153652218
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -18,77 +18,58 @@
package org.apache.zookeeper.test;
-import java.io.ByteArrayOutputStream;
import java.io.File;
-import java.io.FileInputStream;
import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.zookeeper.common.Time;
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.BinaryOutputArchive;
-import org.apache.jute.Record;
+
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException.NoNodeException;
-import org.apache.zookeeper.PortAssignment;
-import org.apache.zookeeper.ZKTestCase;
import org.apache.zookeeper.ZooDefs.Ids;
-import org.apache.zookeeper.ZooDefs.OpCode;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
-import org.apache.zookeeper.server.DataNode;
-import org.apache.zookeeper.server.DataTree;
import org.apache.zookeeper.server.ServerCnxnFactory;
import org.apache.zookeeper.server.SyncRequestProcessor;
import org.apache.zookeeper.server.ZooKeeperServer;
-import org.apache.zookeeper.server.persistence.FileHeader;
import org.apache.zookeeper.server.persistence.FileTxnLog;
import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
import org.apache.zookeeper.server.persistence.Util;
import org.apache.zookeeper.server.persistence.FileTxnLog.FileTxnIterator;
import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.MultiTxn;
-import org.apache.zookeeper.txn.Txn;
import org.apache.zookeeper.txn.TxnHeader;
+import org.eclipse.jetty.util.SocketAddressResolver;
+import org.junit.After;
import org.junit.Assert;
+import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-public class LoadFromLogTest extends ZKTestCase {
-private static final String HOST = "127.0.0.1:";
-private static final int CONNECTION_TIMEOUT = 3000;
+public class LoadFromLogTest extends ClientBase {
private static final int NUM_MESSAGES = 300;
protected static final Logger LOG =
LoggerFactory.getLogger(LoadFromLogTest.class);
// setting up the quorum has a transaction overhead for creating and
closing the session
private static final int TRANSACTION_OVERHEAD = 2;
private static final int TOTAL_TRANSACTIONS = NUM_MESSAGES +
TRANSACTION_OVERHEAD;
+@Before
+public void setUp() throws Exception {
+SyncRequestProcessor.setSnapCount(100);
--- End diff --
I think @phunt is mostly correct. There are a couple issues with
`snapCount`.
First, if want to change it while the syncProcessor thread has started,
shouldn't it be volatile?
Second, I disagree that it has "no effect" on a server that has already
been started but the effect is not the desired behavior. Taking the issue of
the `snapCount` field not being volatile out of the picture, the code in
`SyncProcessor` looks like:
```java
public void run() {
try {
int randRoll = r.nextInt(snapCount/2);
while (true) {
Request si = null;
//get a request
if (si != null) {
// track the number of records written to the log
if (zks.getZKDatabase().append(si)) {
logCount++;
if (logCount > (snapCount / 2 + randRoll)) {
randRoll = r.nextInt(snapCount/2);
```
So my reading is that if `snapCount` is changed from `previousSnapCount`
while the `SyncProcessor` is already running to `desiredSnapCount` it will take
at least `desiredSnapCount` and at worst `desiredSnapCount +
previousSnapCount/2` transactions for the "effective" `snapCount` to become
`desiredSnapCount` (again ignoring the volatility issue).
I agree with @phunt's solution.
---
Success: ZOOKEEPER- PreCommit Build #1319
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1319/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 73.30 MB...] [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 40 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1319//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1319//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1319//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 1ccfa1257fc8832238c1304486c5a3d3c2275a58 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD SUCCESSFUL Total time: 22 minutes 51 seconds Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-236 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Success Sending email for trigger: Success Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (ZOOKEEPER-236) SSL Support for Atomic Broadcast protocol
[ https://issues.apache.org/jira/browse/ZOOKEEPER-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269397#comment-16269397 ] Hadoop QA commented on ZOOKEEPER-236: - +1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 40 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1319//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1319//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1319//console This message is automatically generated. > SSL Support for Atomic Broadcast protocol > - > > Key: ZOOKEEPER-236 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-236 > Project: ZooKeeper > Issue Type: New Feature > Components: quorum, security, server >Reporter: Benjamin Reed >Assignee: Abraham Fine > Labels: ssl > > We should have the ability to use SSL to authenticate and encrypt the traffic > between ZooKeeper servers. For the most part this is a very easy change. We > would probably only want to support this for TCP based leader elections. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] zookeeper pull request #420: ZOOKEEPER-2924. Refactor tests of LoadFromLogTe...
Github user anmolnar commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153610044
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -18,77 +18,58 @@
package org.apache.zookeeper.test;
-import java.io.ByteArrayOutputStream;
import java.io.File;
-import java.io.FileInputStream;
import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.zookeeper.common.Time;
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.BinaryOutputArchive;
-import org.apache.jute.Record;
+
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException.NoNodeException;
-import org.apache.zookeeper.PortAssignment;
-import org.apache.zookeeper.ZKTestCase;
import org.apache.zookeeper.ZooDefs.Ids;
-import org.apache.zookeeper.ZooDefs.OpCode;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
-import org.apache.zookeeper.server.DataNode;
-import org.apache.zookeeper.server.DataTree;
import org.apache.zookeeper.server.ServerCnxnFactory;
import org.apache.zookeeper.server.SyncRequestProcessor;
import org.apache.zookeeper.server.ZooKeeperServer;
-import org.apache.zookeeper.server.persistence.FileHeader;
import org.apache.zookeeper.server.persistence.FileTxnLog;
import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
import org.apache.zookeeper.server.persistence.Util;
import org.apache.zookeeper.server.persistence.FileTxnLog.FileTxnIterator;
import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.MultiTxn;
-import org.apache.zookeeper.txn.Txn;
import org.apache.zookeeper.txn.TxnHeader;
+import org.eclipse.jetty.util.SocketAddressResolver;
+import org.junit.After;
import org.junit.Assert;
+import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-public class LoadFromLogTest extends ZKTestCase {
-private static final String HOST = "127.0.0.1:";
-private static final int CONNECTION_TIMEOUT = 3000;
+public class LoadFromLogTest extends ClientBase {
private static final int NUM_MESSAGES = 300;
protected static final Logger LOG =
LoggerFactory.getLogger(LoadFromLogTest.class);
// setting up the quorum has a transaction overhead for creating and
closing the session
private static final int TRANSACTION_OVERHEAD = 2;
private static final int TOTAL_TRANSACTIONS = NUM_MESSAGES +
TRANSACTION_OVERHEAD;
+@Before
+public void setUp() throws Exception {
+SyncRequestProcessor.setSnapCount(100);
+super.setUp();
+}
+
+@After
+public void tearDown() throws Exception {
+stopServer();
--- End diff --
Good point. super.tearDown() includes stopServer() so I'll remove this one
completely.
---
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269389#comment-16269389
]
ASF GitHub Bot commented on ZOOKEEPER-2924:
---
Github user anmolnar commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153610044
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -18,77 +18,58 @@
package org.apache.zookeeper.test;
-import java.io.ByteArrayOutputStream;
import java.io.File;
-import java.io.FileInputStream;
import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.zookeeper.common.Time;
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.BinaryOutputArchive;
-import org.apache.jute.Record;
+
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException.NoNodeException;
-import org.apache.zookeeper.PortAssignment;
-import org.apache.zookeeper.ZKTestCase;
import org.apache.zookeeper.ZooDefs.Ids;
-import org.apache.zookeeper.ZooDefs.OpCode;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
-import org.apache.zookeeper.server.DataNode;
-import org.apache.zookeeper.server.DataTree;
import org.apache.zookeeper.server.ServerCnxnFactory;
import org.apache.zookeeper.server.SyncRequestProcessor;
import org.apache.zookeeper.server.ZooKeeperServer;
-import org.apache.zookeeper.server.persistence.FileHeader;
import org.apache.zookeeper.server.persistence.FileTxnLog;
import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
import org.apache.zookeeper.server.persistence.Util;
import org.apache.zookeeper.server.persistence.FileTxnLog.FileTxnIterator;
import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.MultiTxn;
-import org.apache.zookeeper.txn.Txn;
import org.apache.zookeeper.txn.TxnHeader;
+import org.eclipse.jetty.util.SocketAddressResolver;
+import org.junit.After;
import org.junit.Assert;
+import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-public class LoadFromLogTest extends ZKTestCase {
-private static final String HOST = "127.0.0.1:";
-private static final int CONNECTION_TIMEOUT = 3000;
+public class LoadFromLogTest extends ClientBase {
private static final int NUM_MESSAGES = 300;
protected static final Logger LOG =
LoggerFactory.getLogger(LoadFromLogTest.class);
// setting up the quorum has a transaction overhead for creating and
closing the session
private static final int TRANSACTION_OVERHEAD = 2;
private static final int TOTAL_TRANSACTIONS = NUM_MESSAGES +
TRANSACTION_OVERHEAD;
+@Before
+public void setUp() throws Exception {
+SyncRequestProcessor.setSnapCount(100);
+super.setUp();
+}
+
+@After
+public void tearDown() throws Exception {
+stopServer();
--- End diff --
Good point. super.tearDown() includes stopServer() so I'll remove this one
completely.
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2924
> Project: ZooKeeper
> Issue Type: Bug
> Components: server, tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
> Labels: flaky, flaky-test
> Fix For: 3.4.12
>
>
> From https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1682/
> Same issue happens in jdk8 and jdk9 builds as well.
> Issue has already been fixed by
> https://issues.apache.org/jira/browse/ZOOKEEPER-2484 , but I believe that the
> root cause here is that test startup / cleanup code is included in the tests
> instead of using try-finally block or Before-After methods.
> As a consequence, when exception happens during test execution, ZK test
> server doesn't get shutdown properly and still listening on the port bound to
> the test class.
> As mentioned above there could be 2 approaches to address this:
> #1 Wrap cleanup code block with finally
> #2 Use JUnit's Before-After methods for initialization and cleanup
> Test where original issue happens:
> {noformat}
> ...
> [junit] 2017-10-12 15:05:20,135 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcess
ZooKeeper-trunk-openjdk7 - Build # 1714 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1714/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 67.92 MB...] [junit] at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214) [junit] 2017-11-28 20:07:34,051 [myid:127.0.0.1:16611] - INFO [main-SendThread(127.0.0.1:16611):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:16611. Will not attempt to authenticate using SASL (unknown error) [junit] 2017-11-28 20:07:34,052 [myid:127.0.0.1:16611] - WARN [main-SendThread(127.0.0.1:16611):ClientCnxn$SendThread@1235] - Session 0x105c08d0ad1 for server 127.0.0.1/127.0.0.1:16611, unexpected error, closing socket connection and attempting reconnect [junit] java.net.ConnectException: Connection refused [junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [junit] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) [junit] at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214) [junit] 2017-11-28 20:07:34,953 [myid:127.0.0.1:16608] - INFO [main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:16608. Will not attempt to authenticate using SASL (unknown error) [junit] 2017-11-28 20:07:34,954 [myid:127.0.0.1:16608] - WARN [main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1235] - Session 0x5c08d08370001 for server 127.0.0.1/127.0.0.1:16608, unexpected error, closing socket connection and attempting reconnect [junit] java.net.ConnectException: Connection refused [junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [junit] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) [junit] at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214) [junit] 2017-11-28 20:07:35,284 [myid:127.0.0.1:16611] - INFO [main-SendThread(127.0.0.1:16611):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:16611. Will not attempt to authenticate using SASL (unknown error) [junit] 2017-11-28 20:07:35,285 [myid:127.0.0.1:16611] - WARN [main-SendThread(127.0.0.1:16611):ClientCnxn$SendThread@1235] - Session 0x105c08d0ad1 for server 127.0.0.1/127.0.0.1:16611, unexpected error, closing socket connection and attempting reconnect [junit] java.net.ConnectException: Connection refused [junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [junit] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) [junit] at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214) [junit] 2017-11-28 20:07:35,393 [myid:127.0.0.1:16611] - INFO [main-SendThread(127.0.0.1:16611):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:16611. Will not attempt to authenticate using SASL (unknown error) [junit] 2017-11-28 20:07:35,394 [myid:127.0.0.1:16611] - WARN [main-SendThread(127.0.0.1:16611):ClientCnxn$SendThread@1235] - Session 0x105c08d0ad10001 for server 127.0.0.1/127.0.0.1:16611, unexpected error, closing socket connection and attempting reconnect [junit] java.net.ConnectException: Connection refused [junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [junit] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) [junit] at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214) [junit] 2017-11-28 20:07:35,573 [myid:127.0.0.1:16608] - INFO [main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:16608. Will not attempt to authenticate using SASL (unknown error) [junit] 2017-11-28 20:07:35,573 [myid:127.0.0.1:16608] - WARN [main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1235] - Session 0x5c08d0837 for server 127.0.0.1/127.0.0.1:16608, unexpected error, closing socket connection and attempting reconnect [junit] java.net.ConnectException: Connection refused [junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [junit] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) [junit] at org.apache.zooke
[jira] [Commented] (ZOOKEEPER-2901) Session ID that is negative causes mis-calculation of Ephemeral Type
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269363#comment-16269363
]
ASF GitHub Bot commented on ZOOKEEPER-2901:
---
Github user phunt commented on the issue:
https://github.com/apache/zookeeper/pull/377
@Randgalt I had to resolve some conflicts in order to compare this to
current master. Posted the updated branch here:
https://github.com/phunt/zookeeper/commits/ZOOKEEPER-2901 Can you review this
and update this patch when you have a chance (I'm still reviewing/testing the
updates at that link)
> Session ID that is negative causes mis-calculation of Ephemeral Type
>
>
> Key: ZOOKEEPER-2901
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2901
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.5.3
> Environment: Running 3.5.3-beta in Docker container
>Reporter: Mark Johnson
>Assignee: Jordan Zimmerman
>Priority: Blocker
>
> In the code that determines the EphemeralType it is looking at the owner
> (which is the client ID or connection ID):
> EphemeralType.java:
>public static EphemeralType get(long ephemeralOwner) {
>if (ephemeralOwner == CONTAINER_EPHEMERAL_OWNER) {
>return CONTAINER;
>}
>if (ephemeralOwner < 0) {
>return TTL;
>}
>return (ephemeralOwner == 0) ? VOID : NORMAL;
>}
> However my connection ID is:
> header.getClientId(): -720548323429908480
> This causes the code to think this is a TTL Ephemeral node instead of a
> NORMAL Ephemeral node.
> This also explains why this is random - if my client ID is non-negative
> then the node gets added correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] zookeeper issue #377: [ZOOKEEPER-2901] TTL Nodes don't work with Server IDs ...
Github user phunt commented on the issue: https://github.com/apache/zookeeper/pull/377 @Randgalt I had to resolve some conflicts in order to compare this to current master. Posted the updated branch here: https://github.com/phunt/zookeeper/commits/ZOOKEEPER-2901 Can you review this and update this patch when you have a chance (I'm still reviewing/testing the updates at that link) ---
[jira] [Commented] (ZOOKEEPER-236) SSL Support for Atomic Broadcast protocol
[ https://issues.apache.org/jira/browse/ZOOKEEPER-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269361#comment-16269361 ] ASF GitHub Bot commented on ZOOKEEPER-236: -- GitHub user afine opened a pull request: https://github.com/apache/zookeeper/pull/424 ZOOKEEPER-236: SSL Support for Atomic Broadcast protocol This is the same thing as https://github.com/apache/zookeeper/pull/184 just targeting master You can merge this pull request into a Git repository by running: $ git pull https://github.com/afine/zookeeper ZOOKEEPER-236_master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/424.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #424 commit f4032913dd90628ec852cb531c7d50ee80e4bcb0 Author: Abraham Fine Date: 2017-03-06T23:12:59Z ZOOKEEPER-236: SSL Support for Atomic Broadcast protocol > SSL Support for Atomic Broadcast protocol > - > > Key: ZOOKEEPER-236 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-236 > Project: ZooKeeper > Issue Type: New Feature > Components: quorum, security, server >Reporter: Benjamin Reed >Assignee: Abraham Fine > Labels: ssl > > We should have the ability to use SSL to authenticate and encrypt the traffic > between ZooKeeper servers. For the most part this is a very easy change. We > would probably only want to support this for TCP based leader elections. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] zookeeper pull request #424: ZOOKEEPER-236: SSL Support for Atomic Broadcast...
GitHub user afine opened a pull request: https://github.com/apache/zookeeper/pull/424 ZOOKEEPER-236: SSL Support for Atomic Broadcast protocol This is the same thing as https://github.com/apache/zookeeper/pull/184 just targeting master You can merge this pull request into a Git repository by running: $ git pull https://github.com/afine/zookeeper ZOOKEEPER-236_master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/424.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #424 commit f4032913dd90628ec852cb531c7d50ee80e4bcb0 Author: Abraham Fine Date: 2017-03-06T23:12:59Z ZOOKEEPER-236: SSL Support for Atomic Broadcast protocol ---
Success: ZOOKEEPER- PreCommit Build #1318
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1318/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 74.73 MB...] [exec] [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +0 tests included. The patch appears to be a documentation patch that doesn't require tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1318//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1318//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1318//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Error: No value specified for option "issue" [exec] 85f5064810aa62e0d984a423b6edd476578e210e logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD SUCCESSFUL Total time: 20 minutes 33 seconds Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2935 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Success Sending email for trigger: Success Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## All tests passed
[GitHub] zookeeper pull request #420: ZOOKEEPER-2924. Refactor tests of LoadFromLogTe...
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153592393
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -413,20 +240,10 @@ public void testRestore() throws Exception {
*/
@Test
public void testRestoreWithTransactionErrors() throws Exception {
-final String hostPort = HOST + PortAssignment.unique();
-// setup a single server cluster
-File tmpDir = ClientBase.createTmpDir();
-ClientBase.setupTestEnv();
-ZooKeeperServer zks = new ZooKeeperServer(tmpDir, tmpDir, 3000);
SyncRequestProcessor.setSnapCount(1);
--- End diff --
same...
---
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269297#comment-16269297
]
ASF GitHub Bot commented on ZOOKEEPER-2924:
---
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153592257
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -370,20 +202,17 @@ public void testRestore() throws Exception {
// Force snapshot and restore
zks.takeSnapshot();
zks.shutdown();
-f.shutdown();
+stopServer();
-zks = new ZooKeeperServer(tmpDir, tmpDir, 3000);
SyncRequestProcessor.setSnapCount(1);
--- End diff --
I believe this is ok as the server is restarted (but you might want to
verify).
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2924
> Project: ZooKeeper
> Issue Type: Bug
> Components: server, tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
> Labels: flaky, flaky-test
> Fix For: 3.4.12
>
>
> From https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1682/
> Same issue happens in jdk8 and jdk9 builds as well.
> Issue has already been fixed by
> https://issues.apache.org/jira/browse/ZOOKEEPER-2484 , but I believe that the
> root cause here is that test startup / cleanup code is included in the tests
> instead of using try-finally block or Before-After methods.
> As a consequence, when exception happens during test execution, ZK test
> server doesn't get shutdown properly and still listening on the port bound to
> the test class.
> As mentioned above there could be 2 approaches to address this:
> #1 Wrap cleanup code block with finally
> #2 Use JUnit's Before-After methods for initialization and cleanup
> Test where original issue happens:
> {noformat}
> ...
> [junit] 2017-10-12 15:05:20,135 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8c zxid:0x8d
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,137 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8d zxid:0x8e
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,139 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8e zxid:0x8f
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,142 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8f zxid:0x90
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,144 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x90 zxid:0x91
> txntype:-1 req$
> [junit] 2017-10-12 15:05:30,479 [myid:] - INFO
> [SessionTracker:ZooKeeperServer@354] - Expiring session 0x104cd7b190c,
> timeout of 6000ms exceeded
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:ping cxid:0xfffe
> zxid:0xf$
> [junit] 2017-10-12 15:05:24,147 [myid:] - WARN
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@1159] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c, closing socket connectio$
> [junit] 2017-10-12 15:05:21,479 [myid:] - INFO
> [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl exited loop!
> [junit] 2017-10-12 15:05:32,998 [myid:] - WARN
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@376] - Unable to
> read additional data from client sessionid 0x104cd7b190c, likely client
> has closed socket
> [junit] 2017-10-12 15:05:33,067 [myid:] - INFO
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@1040] - Closed
> socket connection for client /127.0.0.1:45735 which had sessionid
> 0x104cd7b190c00
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269298#comment-16269298
]
ASF GitHub Bot commented on ZOOKEEPER-2924:
---
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153592781
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -449,50 +267,25 @@ public void testRestoreWithTransactionErrors() throws
Exception {
// Force snapshot and restore
zks.takeSnapshot();
zks.shutdown();
-f.shutdown();
+stopServer();
zks = new ZooKeeperServer(tmpDir, tmpDir, 3000);
SyncRequestProcessor.setSnapCount(1);
-f = ServerCnxnFactory.createFactory(PORT, -1);
-f.startup(zks);
-Assert.assertTrue("waiting for server being up ", ClientBase
-.waitForServerUp(hostPort, CONNECTION_TIMEOUT));
-
-f.shutdown();
-zks.shutdown();
+startServer();
}
/**
* Verify snap/log dir create with/without autocreate enabled.
*/
@Test
public void testDatadirAutocreate() throws Exception {
-ClientBase.setupTestEnv();
--- End diff --
not sure if it matters here and in the following test method but the
snapcount is different after the current patch.
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2924
> Project: ZooKeeper
> Issue Type: Bug
> Components: server, tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
> Labels: flaky, flaky-test
> Fix For: 3.4.12
>
>
> From https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1682/
> Same issue happens in jdk8 and jdk9 builds as well.
> Issue has already been fixed by
> https://issues.apache.org/jira/browse/ZOOKEEPER-2484 , but I believe that the
> root cause here is that test startup / cleanup code is included in the tests
> instead of using try-finally block or Before-After methods.
> As a consequence, when exception happens during test execution, ZK test
> server doesn't get shutdown properly and still listening on the port bound to
> the test class.
> As mentioned above there could be 2 approaches to address this:
> #1 Wrap cleanup code block with finally
> #2 Use JUnit's Before-After methods for initialization and cleanup
> Test where original issue happens:
> {noformat}
> ...
> [junit] 2017-10-12 15:05:20,135 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8c zxid:0x8d
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,137 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8d zxid:0x8e
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,139 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8e zxid:0x8f
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,142 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8f zxid:0x90
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,144 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x90 zxid:0x91
> txntype:-1 req$
> [junit] 2017-10-12 15:05:30,479 [myid:] - INFO
> [SessionTracker:ZooKeeperServer@354] - Expiring session 0x104cd7b190c,
> timeout of 6000ms exceeded
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:ping cxid:0xfffe
> zxid:0xf$
> [junit] 2017-10-12 15:05:24,147 [myid:] - WARN
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@1159] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269296#comment-16269296
]
ASF GitHub Bot commented on ZOOKEEPER-2924:
---
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153591967
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -136,21 +113,10 @@ public void testLoad() throws Exception {
*/
@Test
public void testLoadFailure() throws Exception {
-final String hostPort = HOST + PortAssignment.unique();
-// setup a single server cluster
-File tmpDir = ClientBase.createTmpDir();
-ClientBase.setupTestEnv();
-ZooKeeperServer zks = new ZooKeeperServer(tmpDir, tmpDir, 3000);
-// So we have at least 4 logs
SyncRequestProcessor.setSnapCount(50);
--- End diff --
iiuc the server has already been started in setUp method and as a result
this will have no effect. Am I reading that right. Instead you might need to
set this then call setUp.
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2924
> Project: ZooKeeper
> Issue Type: Bug
> Components: server, tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
> Labels: flaky, flaky-test
> Fix For: 3.4.12
>
>
> From https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1682/
> Same issue happens in jdk8 and jdk9 builds as well.
> Issue has already been fixed by
> https://issues.apache.org/jira/browse/ZOOKEEPER-2484 , but I believe that the
> root cause here is that test startup / cleanup code is included in the tests
> instead of using try-finally block or Before-After methods.
> As a consequence, when exception happens during test execution, ZK test
> server doesn't get shutdown properly and still listening on the port bound to
> the test class.
> As mentioned above there could be 2 approaches to address this:
> #1 Wrap cleanup code block with finally
> #2 Use JUnit's Before-After methods for initialization and cleanup
> Test where original issue happens:
> {noformat}
> ...
> [junit] 2017-10-12 15:05:20,135 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8c zxid:0x8d
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,137 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8d zxid:0x8e
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,139 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8e zxid:0x8f
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,142 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8f zxid:0x90
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,144 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x90 zxid:0x91
> txntype:-1 req$
> [junit] 2017-10-12 15:05:30,479 [myid:] - INFO
> [SessionTracker:ZooKeeperServer@354] - Expiring session 0x104cd7b190c,
> timeout of 6000ms exceeded
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:ping cxid:0xfffe
> zxid:0xf$
> [junit] 2017-10-12 15:05:24,147 [myid:] - WARN
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@1159] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c, closing socket connectio$
> [junit] 2017-10-12 15:05:21,479 [myid:] - INFO
> [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl exited loop!
> [junit] 2017-10-12 15:05:32,998 [myid:] - WARN
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@376] - Unable to
> read additional data from client se
[GitHub] zookeeper pull request #420: ZOOKEEPER-2924. Refactor tests of LoadFromLogTe...
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153592079
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -332,20 +173,10 @@ public void testPad() throws Exception {
*/
@Test
public void testRestore() throws Exception {
-final String hostPort = HOST + PortAssignment.unique();
-// setup a single server cluster
-File tmpDir = ClientBase.createTmpDir();
-ClientBase.setupTestEnv();
-ZooKeeperServer zks = new ZooKeeperServer(tmpDir, tmpDir, 3000);
SyncRequestProcessor.setSnapCount(1);
--- End diff --
Same issue here...
---
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269301#comment-16269301
]
ASF GitHub Bot commented on ZOOKEEPER-2924:
---
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153592393
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -413,20 +240,10 @@ public void testRestore() throws Exception {
*/
@Test
public void testRestoreWithTransactionErrors() throws Exception {
-final String hostPort = HOST + PortAssignment.unique();
-// setup a single server cluster
-File tmpDir = ClientBase.createTmpDir();
-ClientBase.setupTestEnv();
-ZooKeeperServer zks = new ZooKeeperServer(tmpDir, tmpDir, 3000);
SyncRequestProcessor.setSnapCount(1);
--- End diff --
same...
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2924
> Project: ZooKeeper
> Issue Type: Bug
> Components: server, tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
> Labels: flaky, flaky-test
> Fix For: 3.4.12
>
>
> From https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1682/
> Same issue happens in jdk8 and jdk9 builds as well.
> Issue has already been fixed by
> https://issues.apache.org/jira/browse/ZOOKEEPER-2484 , but I believe that the
> root cause here is that test startup / cleanup code is included in the tests
> instead of using try-finally block or Before-After methods.
> As a consequence, when exception happens during test execution, ZK test
> server doesn't get shutdown properly and still listening on the port bound to
> the test class.
> As mentioned above there could be 2 approaches to address this:
> #1 Wrap cleanup code block with finally
> #2 Use JUnit's Before-After methods for initialization and cleanup
> Test where original issue happens:
> {noformat}
> ...
> [junit] 2017-10-12 15:05:20,135 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8c zxid:0x8d
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,137 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8d zxid:0x8e
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,139 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8e zxid:0x8f
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,142 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8f zxid:0x90
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,144 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x90 zxid:0x91
> txntype:-1 req$
> [junit] 2017-10-12 15:05:30,479 [myid:] - INFO
> [SessionTracker:ZooKeeperServer@354] - Expiring session 0x104cd7b190c,
> timeout of 6000ms exceeded
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:ping cxid:0xfffe
> zxid:0xf$
> [junit] 2017-10-12 15:05:24,147 [myid:] - WARN
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@1159] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c, closing socket connectio$
> [junit] 2017-10-12 15:05:21,479 [myid:] - INFO
> [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl exited loop!
> [junit] 2017-10-12 15:05:32,998 [myid:] - WARN
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@376] - Unable to
> read additional data from client sessionid 0x104cd7b190c, likely client
> has closed socket
> [junit] 2017-10-12 15:05:33,067 [myid:] - INFO
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@1040] - Clos
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269300#comment-16269300
]
ASF GitHub Bot commented on ZOOKEEPER-2924:
---
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153592079
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -332,20 +173,10 @@ public void testPad() throws Exception {
*/
@Test
public void testRestore() throws Exception {
-final String hostPort = HOST + PortAssignment.unique();
-// setup a single server cluster
-File tmpDir = ClientBase.createTmpDir();
-ClientBase.setupTestEnv();
-ZooKeeperServer zks = new ZooKeeperServer(tmpDir, tmpDir, 3000);
SyncRequestProcessor.setSnapCount(1);
--- End diff --
Same issue here...
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2924
> Project: ZooKeeper
> Issue Type: Bug
> Components: server, tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
> Labels: flaky, flaky-test
> Fix For: 3.4.12
>
>
> From https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1682/
> Same issue happens in jdk8 and jdk9 builds as well.
> Issue has already been fixed by
> https://issues.apache.org/jira/browse/ZOOKEEPER-2484 , but I believe that the
> root cause here is that test startup / cleanup code is included in the tests
> instead of using try-finally block or Before-After methods.
> As a consequence, when exception happens during test execution, ZK test
> server doesn't get shutdown properly and still listening on the port bound to
> the test class.
> As mentioned above there could be 2 approaches to address this:
> #1 Wrap cleanup code block with finally
> #2 Use JUnit's Before-After methods for initialization and cleanup
> Test where original issue happens:
> {noformat}
> ...
> [junit] 2017-10-12 15:05:20,135 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8c zxid:0x8d
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,137 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8d zxid:0x8e
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,139 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8e zxid:0x8f
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,142 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8f zxid:0x90
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,144 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x90 zxid:0x91
> txntype:-1 req$
> [junit] 2017-10-12 15:05:30,479 [myid:] - INFO
> [SessionTracker:ZooKeeperServer@354] - Expiring session 0x104cd7b190c,
> timeout of 6000ms exceeded
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:ping cxid:0xfffe
> zxid:0xf$
> [junit] 2017-10-12 15:05:24,147 [myid:] - WARN
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@1159] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c, closing socket connectio$
> [junit] 2017-10-12 15:05:21,479 [myid:] - INFO
> [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl exited loop!
> [junit] 2017-10-12 15:05:32,998 [myid:] - WARN
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@376] - Unable to
> read additional data from client sessionid 0x104cd7b190c, likely client
> has closed socket
> [junit] 2017-10-12 15:05:33,067 [myid:] - INFO
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@1040] - Closed
> socket c
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269299#comment-16269299
]
ASF GitHub Bot commented on ZOOKEEPER-2924:
---
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153591342
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -18,77 +18,58 @@
package org.apache.zookeeper.test;
-import java.io.ByteArrayOutputStream;
import java.io.File;
-import java.io.FileInputStream;
import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.zookeeper.common.Time;
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.BinaryOutputArchive;
-import org.apache.jute.Record;
+
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException.NoNodeException;
-import org.apache.zookeeper.PortAssignment;
-import org.apache.zookeeper.ZKTestCase;
import org.apache.zookeeper.ZooDefs.Ids;
-import org.apache.zookeeper.ZooDefs.OpCode;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
-import org.apache.zookeeper.server.DataNode;
-import org.apache.zookeeper.server.DataTree;
import org.apache.zookeeper.server.ServerCnxnFactory;
import org.apache.zookeeper.server.SyncRequestProcessor;
import org.apache.zookeeper.server.ZooKeeperServer;
-import org.apache.zookeeper.server.persistence.FileHeader;
import org.apache.zookeeper.server.persistence.FileTxnLog;
import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
import org.apache.zookeeper.server.persistence.Util;
import org.apache.zookeeper.server.persistence.FileTxnLog.FileTxnIterator;
import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.MultiTxn;
-import org.apache.zookeeper.txn.Txn;
import org.apache.zookeeper.txn.TxnHeader;
+import org.eclipse.jetty.util.SocketAddressResolver;
+import org.junit.After;
import org.junit.Assert;
+import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-public class LoadFromLogTest extends ZKTestCase {
-private static final String HOST = "127.0.0.1:";
-private static final int CONNECTION_TIMEOUT = 3000;
+public class LoadFromLogTest extends ClientBase {
private static final int NUM_MESSAGES = 300;
protected static final Logger LOG =
LoggerFactory.getLogger(LoadFromLogTest.class);
// setting up the quorum has a transaction overhead for creating and
closing the session
private static final int TRANSACTION_OVERHEAD = 2;
private static final int TOTAL_TRANSACTIONS = NUM_MESSAGES +
TRANSACTION_OVERHEAD;
+@Before
+public void setUp() throws Exception {
+SyncRequestProcessor.setSnapCount(100);
--- End diff --
Unfortunately this is going to have the side-effect that all servers of all
the tests in this class will be started with a snapCount of 100. Subsequently
setting this value in the test itself will have no effect as the server has
already been started (syncprocessor thread running). Unless the server is
restarted of course.
See comments below and please verify this finding yourself.
I'm thinking that you might need to make this setup empty, and call setUp
from the very beginning of the test after setting the count explicitly.
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2924
> Project: ZooKeeper
> Issue Type: Bug
> Components: server, tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
> Labels: flaky, flaky-test
> Fix For: 3.4.12
>
>
> From https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1682/
> Same issue happens in jdk8 and jdk9 builds as well.
> Issue has already been fixed by
> https://issues.apache.org/jira/browse/ZOOKEEPER-2484 , but I believe that the
> root cause here is that test startup / cleanup code is included in the tests
> instead of using try-finally block or Before-After methods.
> As a consequence, when exception happens during test execution, ZK test
> server doesn't get shutdown properly and still listening on the port bound to
> the test class.
> As mentioned above th
[GitHub] zookeeper pull request #420: ZOOKEEPER-2924. Refactor tests of LoadFromLogTe...
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153592781
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -449,50 +267,25 @@ public void testRestoreWithTransactionErrors() throws
Exception {
// Force snapshot and restore
zks.takeSnapshot();
zks.shutdown();
-f.shutdown();
+stopServer();
zks = new ZooKeeperServer(tmpDir, tmpDir, 3000);
SyncRequestProcessor.setSnapCount(1);
-f = ServerCnxnFactory.createFactory(PORT, -1);
-f.startup(zks);
-Assert.assertTrue("waiting for server being up ", ClientBase
-.waitForServerUp(hostPort, CONNECTION_TIMEOUT));
-
-f.shutdown();
-zks.shutdown();
+startServer();
}
/**
* Verify snap/log dir create with/without autocreate enabled.
*/
@Test
public void testDatadirAutocreate() throws Exception {
-ClientBase.setupTestEnv();
--- End diff --
not sure if it matters here and in the following test method but the
snapcount is different after the current patch.
---
[GitHub] zookeeper pull request #420: ZOOKEEPER-2924. Refactor tests of LoadFromLogTe...
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153592257
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -370,20 +202,17 @@ public void testRestore() throws Exception {
// Force snapshot and restore
zks.takeSnapshot();
zks.shutdown();
-f.shutdown();
+stopServer();
-zks = new ZooKeeperServer(tmpDir, tmpDir, 3000);
SyncRequestProcessor.setSnapCount(1);
--- End diff --
I believe this is ok as the server is restarted (but you might want to
verify).
---
[GitHub] zookeeper pull request #420: ZOOKEEPER-2924. Refactor tests of LoadFromLogTe...
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153591342
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -18,77 +18,58 @@
package org.apache.zookeeper.test;
-import java.io.ByteArrayOutputStream;
import java.io.File;
-import java.io.FileInputStream;
import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.zookeeper.common.Time;
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.BinaryOutputArchive;
-import org.apache.jute.Record;
+
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException.NoNodeException;
-import org.apache.zookeeper.PortAssignment;
-import org.apache.zookeeper.ZKTestCase;
import org.apache.zookeeper.ZooDefs.Ids;
-import org.apache.zookeeper.ZooDefs.OpCode;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
-import org.apache.zookeeper.server.DataNode;
-import org.apache.zookeeper.server.DataTree;
import org.apache.zookeeper.server.ServerCnxnFactory;
import org.apache.zookeeper.server.SyncRequestProcessor;
import org.apache.zookeeper.server.ZooKeeperServer;
-import org.apache.zookeeper.server.persistence.FileHeader;
import org.apache.zookeeper.server.persistence.FileTxnLog;
import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
import org.apache.zookeeper.server.persistence.Util;
import org.apache.zookeeper.server.persistence.FileTxnLog.FileTxnIterator;
import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.MultiTxn;
-import org.apache.zookeeper.txn.Txn;
import org.apache.zookeeper.txn.TxnHeader;
+import org.eclipse.jetty.util.SocketAddressResolver;
+import org.junit.After;
import org.junit.Assert;
+import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-public class LoadFromLogTest extends ZKTestCase {
-private static final String HOST = "127.0.0.1:";
-private static final int CONNECTION_TIMEOUT = 3000;
+public class LoadFromLogTest extends ClientBase {
private static final int NUM_MESSAGES = 300;
protected static final Logger LOG =
LoggerFactory.getLogger(LoadFromLogTest.class);
// setting up the quorum has a transaction overhead for creating and
closing the session
private static final int TRANSACTION_OVERHEAD = 2;
private static final int TOTAL_TRANSACTIONS = NUM_MESSAGES +
TRANSACTION_OVERHEAD;
+@Before
+public void setUp() throws Exception {
+SyncRequestProcessor.setSnapCount(100);
--- End diff --
Unfortunately this is going to have the side-effect that all servers of all
the tests in this class will be started with a snapCount of 100. Subsequently
setting this value in the test itself will have no effect as the server has
already been started (syncprocessor thread running). Unless the server is
restarted of course.
See comments below and please verify this finding yourself.
I'm thinking that you might need to make this setup empty, and call setUp
from the very beginning of the test after setting the count explicitly.
---
[GitHub] zookeeper pull request #420: ZOOKEEPER-2924. Refactor tests of LoadFromLogTe...
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153591967
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -136,21 +113,10 @@ public void testLoad() throws Exception {
*/
@Test
public void testLoadFailure() throws Exception {
-final String hostPort = HOST + PortAssignment.unique();
-// setup a single server cluster
-File tmpDir = ClientBase.createTmpDir();
-ClientBase.setupTestEnv();
-ZooKeeperServer zks = new ZooKeeperServer(tmpDir, tmpDir, 3000);
-// So we have at least 4 logs
SyncRequestProcessor.setSnapCount(50);
--- End diff --
iiuc the server has already been started in setUp method and as a result
this will have no effect. Am I reading that right. Instead you might need to
set this then call setUp.
---
[jira] [Commented] (ZOOKEEPER-236) SSL Support for Atomic Broadcast protocol
[ https://issues.apache.org/jira/browse/ZOOKEEPER-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269292#comment-16269292 ] Hadoop QA commented on ZOOKEEPER-236: - -1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 40 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1316//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1316//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1316//console This message is automatically generated. > SSL Support for Atomic Broadcast protocol > - > > Key: ZOOKEEPER-236 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-236 > Project: ZooKeeper > Issue Type: New Feature > Components: quorum, security, server >Reporter: Benjamin Reed >Assignee: Abraham Fine > Labels: ssl > > We should have the ability to use SSL to authenticate and encrypt the traffic > between ZooKeeper servers. For the most part this is a very easy change. We > would probably only want to support this for TCP based leader elections. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Failed: ZOOKEEPER- PreCommit Build #1316
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1316/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 73.52 MB...] [exec] +1 tests included. The patch appears to include 40 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] -1 core tests. The patch failed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1316//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1316//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1316//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 7b8fd93e1909288bbcd1b1c16221d317b7adcffb logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD FAILED /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1722: exec returned: 1 Total time: 22 minutes 7 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-236 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## 1 tests failed. FAILED: org.apache.zookeeper.test.ClientSSLTest.testClientServerSSL Error Message: waiting for server 0 being up Stack Trace: junit.framework.AssertionFailedError: waiting for server 0 being up at org.apache.zookeeper.test.ClientSSLTest.testClientServerSSL(ClientSSLTest.java:98) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
[jira] [Commented] (ZOOKEEPER-236) SSL Support for Atomic Broadcast protocol
[ https://issues.apache.org/jira/browse/ZOOKEEPER-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269284#comment-16269284 ] Hadoop QA commented on ZOOKEEPER-236: - -1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 40 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1317//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1317//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1317//console This message is automatically generated. > SSL Support for Atomic Broadcast protocol > - > > Key: ZOOKEEPER-236 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-236 > Project: ZooKeeper > Issue Type: New Feature > Components: quorum, security, server >Reporter: Benjamin Reed >Assignee: Abraham Fine > Labels: ssl > > We should have the ability to use SSL to authenticate and encrypt the traffic > between ZooKeeper servers. For the most part this is a very easy change. We > would probably only want to support this for TCP based leader elections. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Failed: ZOOKEEPER- PreCommit Build #1317
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1317/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 78.10 MB...] [exec] +1 tests included. The patch appears to include 40 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] -1 core tests. The patch failed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1317//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1317//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1317//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] df30920140b6a6e99ff248a137b3b5d6691a0175 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD FAILED /home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1722: exec returned: 1 Total time: 15 minutes 19 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-236 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## 1 tests failed. FAILED: org.apache.zookeeper.test.ReconfigTest.testPortChange Error Message: Existing client disconnected when client port changed! Stack Trace: junit.framework.AssertionFailedError: Existing client disconnected when client port changed! at org.apache.zookeeper.test.ReconfigTest.testPortChange(ReconfigTest.java:621) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269232#comment-16269232
]
ASF GitHub Bot commented on ZOOKEEPER-2924:
---
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153584084
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -18,77 +18,58 @@
package org.apache.zookeeper.test;
-import java.io.ByteArrayOutputStream;
import java.io.File;
-import java.io.FileInputStream;
import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.zookeeper.common.Time;
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.BinaryOutputArchive;
-import org.apache.jute.Record;
+
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException.NoNodeException;
-import org.apache.zookeeper.PortAssignment;
-import org.apache.zookeeper.ZKTestCase;
import org.apache.zookeeper.ZooDefs.Ids;
-import org.apache.zookeeper.ZooDefs.OpCode;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
-import org.apache.zookeeper.server.DataNode;
-import org.apache.zookeeper.server.DataTree;
import org.apache.zookeeper.server.ServerCnxnFactory;
import org.apache.zookeeper.server.SyncRequestProcessor;
import org.apache.zookeeper.server.ZooKeeperServer;
-import org.apache.zookeeper.server.persistence.FileHeader;
import org.apache.zookeeper.server.persistence.FileTxnLog;
import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
import org.apache.zookeeper.server.persistence.Util;
import org.apache.zookeeper.server.persistence.FileTxnLog.FileTxnIterator;
import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.MultiTxn;
-import org.apache.zookeeper.txn.Txn;
import org.apache.zookeeper.txn.TxnHeader;
+import org.eclipse.jetty.util.SocketAddressResolver;
+import org.junit.After;
import org.junit.Assert;
+import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-public class LoadFromLogTest extends ZKTestCase {
-private static final String HOST = "127.0.0.1:";
-private static final int CONNECTION_TIMEOUT = 3000;
+public class LoadFromLogTest extends ClientBase {
private static final int NUM_MESSAGES = 300;
protected static final Logger LOG =
LoggerFactory.getLogger(LoadFromLogTest.class);
// setting up the quorum has a transaction overhead for creating and
closing the session
private static final int TRANSACTION_OVERHEAD = 2;
private static final int TOTAL_TRANSACTIONS = NUM_MESSAGES +
TRANSACTION_OVERHEAD;
+@Before
+public void setUp() throws Exception {
+SyncRequestProcessor.setSnapCount(100);
+super.setUp();
+}
+
+@After
+public void tearDown() throws Exception {
+stopServer();
--- End diff --
shouldn't this should be calling super.tearDown() ?
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2924
> Project: ZooKeeper
> Issue Type: Bug
> Components: server, tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
> Labels: flaky, flaky-test
> Fix For: 3.4.12
>
>
> From https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1682/
> Same issue happens in jdk8 and jdk9 builds as well.
> Issue has already been fixed by
> https://issues.apache.org/jira/browse/ZOOKEEPER-2484 , but I believe that the
> root cause here is that test startup / cleanup code is included in the tests
> instead of using try-finally block or Before-After methods.
> As a consequence, when exception happens during test execution, ZK test
> server doesn't get shutdown properly and still listening on the port bound to
> the test class.
> As mentioned above there could be 2 approaches to address this:
> #1 Wrap cleanup code block with finally
> #2 Use JUnit's Before-After methods for initialization and cleanup
> Test where original issue happens:
> {noformat}
> ...
> [junit] 2017-10-12 15:05:20,135 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperExceptio
[GitHub] zookeeper pull request #420: ZOOKEEPER-2924. Refactor tests of LoadFromLogTe...
Github user phunt commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/420#discussion_r153584084
--- Diff: src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java ---
@@ -18,77 +18,58 @@
package org.apache.zookeeper.test;
-import java.io.ByteArrayOutputStream;
import java.io.File;
-import java.io.FileInputStream;
import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.zookeeper.common.Time;
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.BinaryOutputArchive;
-import org.apache.jute.Record;
+
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException.NoNodeException;
-import org.apache.zookeeper.PortAssignment;
-import org.apache.zookeeper.ZKTestCase;
import org.apache.zookeeper.ZooDefs.Ids;
-import org.apache.zookeeper.ZooDefs.OpCode;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
-import org.apache.zookeeper.server.DataNode;
-import org.apache.zookeeper.server.DataTree;
import org.apache.zookeeper.server.ServerCnxnFactory;
import org.apache.zookeeper.server.SyncRequestProcessor;
import org.apache.zookeeper.server.ZooKeeperServer;
-import org.apache.zookeeper.server.persistence.FileHeader;
import org.apache.zookeeper.server.persistence.FileTxnLog;
import org.apache.zookeeper.server.persistence.FileTxnSnapLog;
import org.apache.zookeeper.server.persistence.Util;
import org.apache.zookeeper.server.persistence.FileTxnLog.FileTxnIterator;
import org.apache.zookeeper.server.persistence.TxnLog.TxnIterator;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.MultiTxn;
-import org.apache.zookeeper.txn.Txn;
import org.apache.zookeeper.txn.TxnHeader;
+import org.eclipse.jetty.util.SocketAddressResolver;
+import org.junit.After;
import org.junit.Assert;
+import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-public class LoadFromLogTest extends ZKTestCase {
-private static final String HOST = "127.0.0.1:";
-private static final int CONNECTION_TIMEOUT = 3000;
+public class LoadFromLogTest extends ClientBase {
private static final int NUM_MESSAGES = 300;
protected static final Logger LOG =
LoggerFactory.getLogger(LoadFromLogTest.class);
// setting up the quorum has a transaction overhead for creating and
closing the session
private static final int TRANSACTION_OVERHEAD = 2;
private static final int TOTAL_TRANSACTIONS = NUM_MESSAGES +
TRANSACTION_OVERHEAD;
+@Before
+public void setUp() throws Exception {
+SyncRequestProcessor.setSnapCount(100);
+super.setUp();
+}
+
+@After
+public void tearDown() throws Exception {
+stopServer();
--- End diff --
shouldn't this should be calling super.tearDown() ?
---
Success: ZOOKEEPER- PreCommit Build #1315
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1315/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 74.98 MB...] [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 34 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1315//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1315//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1315//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 8e57a5035c01ed05a30dbfa6428300e9d46aabf3 logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD SUCCESSFUL Total time: 19 minutes 8 seconds Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Description set: ZOOKEEPER-2924 Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Success Sending email for trigger: Success Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (ZOOKEEPER-2924) Flaky Test: org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269108#comment-16269108
]
Hadoop QA commented on ZOOKEEPER-2924:
--
+1 overall. GitHub Pull Request Build
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 34 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1315//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1315//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1315//console
This message is automatically generated.
> Flaky Test:
> org.apache.zookeeper.test.LoadFromLogTest.testRestoreWithTransactionErrors
> --
>
> Key: ZOOKEEPER-2924
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2924
> Project: ZooKeeper
> Issue Type: Bug
> Components: server, tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
> Labels: flaky, flaky-test
> Fix For: 3.4.12
>
>
> From https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1682/
> Same issue happens in jdk8 and jdk9 builds as well.
> Issue has already been fixed by
> https://issues.apache.org/jira/browse/ZOOKEEPER-2484 , but I believe that the
> root cause here is that test startup / cleanup code is included in the tests
> instead of using try-finally block or Before-After methods.
> As a consequence, when exception happens during test execution, ZK test
> server doesn't get shutdown properly and still listening on the port bound to
> the test class.
> As mentioned above there could be 2 approaches to address this:
> #1 Wrap cleanup code block with finally
> #2 Use JUnit's Before-After methods for initialization and cleanup
> Test where original issue happens:
> {noformat}
> ...
> [junit] 2017-10-12 15:05:20,135 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8c zxid:0x8d
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,137 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8d zxid:0x8e
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,139 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8e zxid:0x8f
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,142 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x8f zxid:0x90
> txntype:-1 req$
> [junit] 2017-10-12 15:05:20,144 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:create cxid:0x90 zxid:0x91
> txntype:-1 req$
> [junit] 2017-10-12 15:05:30,479 [myid:] - INFO
> [SessionTracker:ZooKeeperServer@354] - Expiring session 0x104cd7b190c,
> timeout of 6000ms exceeded
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO [ProcessThread(sid:0
> cport:11221)::PrepRequestProcessor@653] - Got user-level KeeperException when
> processing sessionid:0x104cd7b190c type:ping cxid:0xfffe
> zxid:0xf$
> [junit] 2017-10-12 15:05:24,147 [myid:] - WARN
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c
> [junit] 2017-10-12 15:05:32,996 [myid:] - INFO
> [main-SendThread(127.0.0.1:11221):ClientCnxn$SendThread@1159] - Client
> session timed out, have not heard from server in 4002ms for sessionid
> 0x104cd7b190c, closing socket connectio$
> [junit] 2017-10-12 15:05:21,479 [myid:] - INFO
> [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl exited loop!
> [junit] 2017-10-12 15:05:32,998 [myid:] - WARN
> [NIOServe
ZooKeeper_branch34_openjdk7 - Build # 1733 - Failure
See https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1733/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 24.48 MB...] [junit] 2017-11-28 15:19:54,248 [myid:] - INFO [ProcessThread(sid:0 cport:11221)::PrepRequestProcessor@144] - PrepRequestProcessor exited loop! [junit] 2017-11-28 15:19:54,248 [myid:] - INFO [SyncThread:0:SyncRequestProcessor@186] - SyncRequestProcessor exited! [junit] 2017-11-28 15:19:54,249 [myid:] - INFO [main:FinalRequestProcessor@403] - shutdown of request processor complete [junit] 2017-11-28 15:19:54,249 [myid:] - INFO [main:FourLetterWordMain@65] - connecting to 127.0.0.1 11221 [junit] 2017-11-28 15:19:54,249 [myid:] - INFO [main:JMXEnv@147] - ensureOnly:[] [junit] 2017-11-28 15:19:54,250 [myid:] - INFO [main:ClientBase@489] - STARTING server [junit] 2017-11-28 15:19:54,250 [myid:] - INFO [main:ClientBase@410] - CREATING server instance 127.0.0.1:11221 [junit] 2017-11-28 15:19:54,251 [myid:] - INFO [main:ServerCnxnFactory@117] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory [junit] 2017-11-28 15:19:54,251 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11221 [junit] 2017-11-28 15:19:54,251 [myid:] - INFO [main:ClientBase@385] - STARTING server instance 127.0.0.1:11221 [junit] 2017-11-28 15:19:54,251 [myid:] - INFO [main:ZooKeeperServer@173] - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 6 datadir /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk7/build/test/tmp/test2763550114049615179.junit.dir/version-2 snapdir /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk7/build/test/tmp/test2763550114049615179.junit.dir/version-2 [junit] 2017-11-28 15:19:54,255 [myid:] - INFO [main:FourLetterWordMain@65] - connecting to 127.0.0.1 11221 [junit] 2017-11-28 15:19:54,255 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@215] - Accepted socket connection from /127.0.0.1:44653 [junit] 2017-11-28 15:19:54,255 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@892] - Processing stat command from /127.0.0.1:44653 [junit] 2017-11-28 15:19:54,255 [myid:] - INFO [Thread-4:NIOServerCnxn$StatCommand@683] - Stat command output [junit] 2017-11-28 15:19:54,256 [myid:] - INFO [Thread-4:NIOServerCnxn@1040] - Closed socket connection for client /127.0.0.1:44653 (no session established for client) [junit] 2017-11-28 15:19:54,256 [myid:] - INFO [main:JMXEnv@230] - ensureParent:[InMemoryDataTree, StandaloneServer_port] [junit] 2017-11-28 15:19:54,257 [myid:] - INFO [main:JMXEnv@247] - expect:InMemoryDataTree [junit] 2017-11-28 15:19:54,257 [myid:] - INFO [main:JMXEnv@251] - found:InMemoryDataTree org.apache.ZooKeeperService:name0=StandaloneServer_port11221,name1=InMemoryDataTree [junit] 2017-11-28 15:19:54,258 [myid:] - INFO [main:JMXEnv@247] - expect:StandaloneServer_port [junit] 2017-11-28 15:19:54,258 [myid:] - INFO [main:JMXEnv@251] - found:StandaloneServer_port org.apache.ZooKeeperService:name0=StandaloneServer_port11221 [junit] 2017-11-28 15:19:54,258 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 20133 [junit] 2017-11-28 15:19:54,258 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 20 [junit] 2017-11-28 15:19:54,258 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testQuota [junit] 2017-11-28 15:19:54,258 [myid:] - INFO [main:ClientBase@566] - tearDown starting [junit] 2017-11-28 15:19:54,335 [myid:] - INFO [main:ZooKeeper@687] - Session: 0x105bf935638 closed [junit] 2017-11-28 15:19:54,335 [myid:] - INFO [main:ClientBase@536] - STOPPING server [junit] 2017-11-28 15:19:54,335 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@520] - EventThread shut down for session: 0x105bf935638 [junit] 2017-11-28 15:19:54,335 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@242] - NIOServerCnxn factory exited run method [junit] 2017-11-28 15:19:54,336 [myid:] - INFO [main:ZooKeeperServer@501] - shutting down [junit] 2017-11-28 15:19:54,336 [myid:] - INFO [main:SessionTrackerImpl@226] - Shutting down [junit] 2017-11-28 15:19:54,336 [myid:] - INFO [main:PrepRequestProcessor@769] - Shutting down [junit] 2017-11-28 15:19:54,336 [myid:] - INFO [main:SyncRequestProcessor@208] - Shutting down [junit] 2017-11-28 15:19:54,336 [myid:] - INFO [ProcessThread(sid:0 cport:11221)::PrepRequestProcessor@144] - PrepRequestProcessor exited loop! [junit] 2017-11-28 15:19:54,336 [myid:] - INFO [SyncThread:0:SyncRequestProcessor@186] - SyncRequestProcessor
ZooKeeper_branch35_jdk7 - Build # 1200 - Failure
See https://builds.apache.org/job/ZooKeeper_branch35_jdk7/1200/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 69.45 MB...] [junit] 2017-11-28 08:52:48,741 [myid:] - INFO [ProcessThread(sid:0 cport:11468)::PrepRequestProcessor@611] - Processed session termination for sessionid: 0x10059318c46 [junit] 2017-11-28 08:52:48,756 [myid:] - INFO [SyncThread:0:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port11468,name1=Connections,name2=127.0.0.1,name3=0x10059318c46] [junit] 2017-11-28 08:52:48,757 [myid:] - INFO [main:ZooKeeper@1334] - Session: 0x10059318c46 closed [junit] 2017-11-28 08:52:48,757 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for session: 0x10059318c46 [junit] 2017-11-28 08:52:48,757 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 214621 [junit] 2017-11-28 08:52:48,757 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 2431 [junit] 2017-11-28 08:52:48,757 [myid:] - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD testWatcherAutoResetWithLocal [junit] 2017-11-28 08:52:48,757 [myid:] - INFO [main:ClientBase@586] - tearDown starting [junit] 2017-11-28 08:52:48,758 [myid:] - INFO [main:ClientBase@556] - STOPPING server [junit] 2017-11-28 08:52:48,758 [myid:] - INFO [main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:11468 [junit] 2017-11-28 08:52:48,762 [myid:127.0.0.1:11348] - INFO [main-SendThread(127.0.0.1:11348):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:11348. Will not attempt to authenticate using SASL (unknown error) [junit] 2017-11-28 08:52:48,762 [myid:127.0.0.1:11348] - WARN [main-SendThread(127.0.0.1:11348):ClientCnxn$SendThread@1235] - Session 0x200592e77db for server 127.0.0.1/127.0.0.1:11348, unexpected error, closing socket connection and attempting reconnect [junit] java.net.ConnectException: Connection refused [junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [junit] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) [junit] at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214) [junit] 2017-11-28 08:52:48,762 [myid:] - INFO [main:ZooKeeperServer@541] - shutting down [junit] 2017-11-28 08:52:48,763 [myid:] - INFO [main:SessionTrackerImpl@232] - Shutting down [junit] 2017-11-28 08:52:48,763 [myid:] - INFO [main:PrepRequestProcessor@1005] - Shutting down [junit] 2017-11-28 08:52:48,763 [myid:] - INFO [main:SyncRequestProcessor@191] - Shutting down [junit] 2017-11-28 08:52:48,763 [myid:] - INFO [ProcessThread(sid:0 cport:11468)::PrepRequestProcessor@155] - PrepRequestProcessor exited loop! [junit] 2017-11-28 08:52:48,763 [myid:] - INFO [SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited! [junit] 2017-11-28 08:52:48,763 [myid:] - INFO [main:FinalRequestProcessor@481] - shutdown of request processor complete [junit] 2017-11-28 08:52:48,764 [myid:] - INFO [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port11468,name1=InMemoryDataTree] [junit] 2017-11-28 08:52:48,764 [myid:] - INFO [main:MBeanRegistry@128] - Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port11468] [junit] 2017-11-28 08:52:48,764 [myid:] - INFO [main:FourLetterWordMain@87] - connecting to 127.0.0.1 11468 [junit] 2017-11-28 08:52:48,765 [myid:] - INFO [main:JMXEnv@146] - ensureOnly:[] [junit] 2017-11-28 08:52:48,775 [myid:] - INFO [main:ClientBase@611] - fdcount after test is: 7165 at start it was 7165 [junit] 2017-11-28 08:52:48,776 [myid:] - INFO [main:ZKTestCase$1@68] - SUCCEEDED testWatcherAutoResetWithLocal [junit] 2017-11-28 08:52:48,776 [myid:] - INFO [main:ZKTestCase$1@63] - FINISHED testWatcherAutoResetWithLocal [junit] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 550.248 sec, Thread: 1, Class: org.apache.zookeeper.test.NioNettySuiteTest [junit] 2017-11-28 08:52:48,877 [myid:] - INFO [SessionTracker:SessionTrackerImpl@158] - SessionTrackerImpl exited loop! [junit] 2017-11-28 08:52:49,109 [myid:127.0.0.1:11286] - INFO [main-SendThread(127.0.0.1:11286):ClientCnxn$SendThread@1113] - Opening socket connection to server 127.0.0.1/127.0.0.1:11286. Will not attempt to authenticate using SASL (unknown error) [junit] 2017-11-28 08:52:49,110 [myid:127.0.0.1:11286] - WARN [main-SendThread(127.0.0.1:11286):ClientCnxn$SendThread@1235] - Session 0x100592b729f for
