Re: Glide path to getting 3.5.x out of beta
Great work Michael,
I am totally +1 on using docker for network isolation
I think that Apache CI may allow out-of-the-box execution in Docker
containers, in fact we have the "CloudBees Docker Custom Build
Environment Plugin"
We can use a public image or provide on the repo a Dockerfile.
In my company we are taking another approach, we launch a script which
sets up the container(s) and that run the tests.
The former approach (built in jenkins) is easy to try, the latter is
more complex but maybe you already have some script,
but our automatic QA script is quite complex and needs a lot of third party tool
when we will be on Maven we would not need external findbugs,forrest,ant.
${ANT_HOME}/bin/ant \
-Dpatch.file=foobar \
-Dscratch.dir=$PATCH_DIR \
-Dps.cmd=/bin/ps \
-Dwget.cmd=/usr/bin/wget \
-Djiracli.cmd=/home/jenkins/tools/jiracli/latest/jira.sh \
-Dgit.cmd=/usr/bin/git \
-Dgrep.cmd=/bin/grep \
-Dpatch.cmd=/usr/bin/patch \
-Dfindbugs.home=/home/jenkins/tools/findbugs/latest/ \
-Dforrest.home=/home/jenkins/tools/forrest/latest/ \
-Djira.passwd=no-shown-here \
-Djava5.home=/home/jenkins/tools/java5/latest/ \
-Dcurl.cmd=/usr/bin/curl \
-Dtest.junit.maxmem=2g \
qa-test-pullrequest
I am not a committer, but I have write access to Apache CI, so if
ZooKeeper PMCs agree on trying the docker config on the CI jobs I will
be happy to try
Enrico
Il giorno gio 29 nov 2018 alle ore 06:27 Michael K. Edwards
ha scritto:
>
> With the use of a Docker container (to prevent port collisions) and a
> stack of cleanups to test code, I've made some progress towards
> reliable test runs in our environment.
> (https://github.com/mkedwards/zookeeper/commits/rollup-3.5, if you're
> curious.) The list below consists of the "top 40" slowest individual
> tests. Note that several appear multiple times, because of the
> inclusion of classes containing slow tests in NioNettySuiteTest and
> NettyNettySuiteTest.
>
> I'm somewhat hesitant to undertake further overhauls of the test
> suite, because I've already found myself having to make the kinds of
> changes that tend to be uphill battles, code-review-wise -- especially
> coming from an outsider.
> https://github.com/mkedwards/zookeeper/commit/e02eb705c6550f51ebb860a474ce711ec68c7a24
> is an example. If a Zookeeper committer is interested in working with
> me on this, maybe email me? Otherwise, I'll try to keep this branch
> rebased regularly, and hammer on the remaining flaky tests to see what
> I can learn.
>
> $ grep ' Ran ' build.log | sort -n -t '[' -r -k 4 | head -40
> [junit] [39343@1f20a9d731ad] Ran [68.657]
> org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortLeader
> ... OK
> [junit] [44264@1f20a9d731ad] Ran [68.607]
> org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortFollower
> ... OK
> [junit] [59151@1f20a9d731ad] Ran [67.535]
> org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortLeader
> ... OK
> [junit] [59151@1f20a9d731ad] Ran [67.397]
> org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortFollower
> ... OK
> [junit] [24817@1f20a9d731ad] Ran [66.555]
> org.apache.zookeeper.server.quorum.ReconfigRecoveryTest:testNextConfigUnreachable
> ... OK
> [junit] [39343@1f20a9d731ad] Ran [66.345]
> org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortFollower
> ... OK
> [junit] [44264@1f20a9d731ad] Ran [65.382]
> org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortLeader
> ... OK
> [junit] [33311@1f20a9d731ad] Ran [64.39]
> org.apache.zookeeper.test.DisconnectedWatcherTest:testManyChildWatchersAutoReset
> ... OK
> [junit] [40332@1f20a9d731ad] Ran [60.907]
> org.apache.zookeeper.test.AsyncHammerTest:testHammer ... OK
> [junit] [26094@1f20a9d731ad] Ran [58.559]
> org.apache.zookeeper.server.quorum.StandaloneDisabledTest:startSingleServerTest
> ... OK
> [junit] [34470@1f20a9d731ad] Ran [52.229]
> org.apache.zookeeper.test.FollowerResyncConcurrencyTest:testResyncByTxnlogThenDiffAfterFollowerCrashes
> ... OK
> [junit] [19354@1f20a9d731ad] Ran [49.956]
> org.apache.zookeeper.server.quorum.QuorumSSLTest:testHostnameVerificationWithInvalidIPAddress
> ... OK
> [junit] [18406@1f20a9d731ad] Ran [48.65]
> org.apache.zookeeper.server.quorum.QuorumPeerMainTest:testFailedTxnAsPartOfQuorumLoss
> ... OK
> [junit] [34470@1f20a9d731ad] Ran [48.582]
> org.apache.zookeeper.test.FollowerResyncConcurrencyTest:testResyncBySnapThenDiffAfterFollowerCrashes
> ... OK
> [junit] [18406@1f20a9d731ad] Ran [47.115]
> org.apache.zookeeper.server.quorum.QuorumPeerMainTest:testEarlyLeaderAbandonment
> ... OK
> [junit] [19354@1f20a9d731ad] Ran [46.184]
> org.apache.zookeeper.server.quorum.QuorumSSLTest:testCipherSuites ...
> OK
> [junit] [28926@1f20a9d731ad] Ran [45.764]
>
[GitHub] zookeeper pull request #703: [ZOOKEEPER-1818] Correctly handle potential inc...
Github user hanm commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/703#discussion_r237367857
--- Diff:
zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeer.java
---
@@ -1989,6 +1989,38 @@ private boolean updateVote(long designatedLeader,
long zxid){
/**
* Updates leader election info to avoid inconsistencies when
* a new server tries to join the ensemble.
+ *
+ * Here is the inconsistency scenario we try to solve by updating the
peer
+ * epoch after following leader:
+ *
+ * Let's say we have an ensemble with 3 servers z1, z2 and z3.
+ *
+ * 1. z1, z2 were following z3 with peerEpoch to be 0xb8, the new
epoch is
+ *0xb9, aka current accepted epoch on disk.
+ * 2. z2 get restarted, which will use 0xb9 as it's peer epoch when
loading
+ *the current accept epoch from disk.
+ * 3. z2 received notification from z1 and z3, which is following z3
with
+ *epoch 0xb8, so it started following z3 again with peer epoch
0xb8.
+ * 4. before z2 successfully connected to z3, z3 get restarted with
new
+ *epoch 0xb9.
+ * 5. z2 will retry around a few round (default 5s) before giving up,
+ *meanwhile it will report z3 as leader.
+ * 6. z1 restarted, and looking with peer epoch 0xb9.
+ * 7. z1 voted z3, and z3 was elected as leader again with peer epoch
0xb9.
+ * 8. z2 successfully connected to z3 before giving up, but with peer
+ *epoch 0xb8.
+ * 9. z1 get restarted, looking for leader with peer epoch 0xba, but
cannot
+ *join, because z2 is reporting peer epoch 0xb8, while z3 is
reporting
+ *0xb9.
+ *
+ * By updating the election vote after actually following leader, we
can
--- End diff --
should be 'election epoch' or 'electionEpoch' rather than 'election vote'.
---
[GitHub] zookeeper pull request #703: [ZOOKEEPER-1818] Correctly handle potential inc...
Github user hanm commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/703#discussion_r237368501
--- Diff:
zookeeper-server/src/test/java/org/apache/zookeeper/server/quorum/FLEOutOfElectionTest.java
---
@@ -0,0 +1,136 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.server.quorum;
+
+import java.io.File;
+import java.net.InetSocketAddress;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.apache.zookeeper.PortAssignment;
+import org.apache.zookeeper.server.quorum.QuorumPeer;
+import org.apache.zookeeper.server.quorum.FastLeaderElection.Notification;
+import org.apache.zookeeper.server.quorum.Vote;
+import org.apache.zookeeper.server.quorum.QuorumPeer.QuorumServer;
+import org.apache.zookeeper.server.quorum.QuorumPeer.ServerState;
+import org.apache.zookeeper.server.util.ZxidUtils;
+import org.apache.zookeeper.test.ClientBase;
+import org.apache.zookeeper.test.FLETest;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+/**
+ * Test FastLeaderElection with out of election servers.
+ */
+public class FLEOutOfElectionTest {
+
+private FastLeaderElection fle;
+
+@Before
+public void setUp() throws Exception {
+File tmpdir = ClientBase.createTmpDir();
+Map peers = new HashMap();
+for(int i = 0; i < 5; i++) {
+peers.put(Long.valueOf(i), new QuorumServer(Long.valueOf(i),
+new InetSocketAddress("127.0.0.1",
PortAssignment.unique(;
+}
+QuorumPeer peer = new QuorumPeer(peers, tmpdir, tmpdir,
+PortAssignment.unique(), 3, 3, 1000, 2, 2);
+fle = new FastLeaderElection(peer, peer.createCnxnManager());
+}
+
+@Test
+public void testIgnoringZxidElectionEpoch() {
+Map votes = new HashMap();
+votes.put(0L, new Vote(0x1, 4L, ZxidUtils.makeZxid(1, 1), 1, 2,
ServerState.FOLLOWING));
+votes.put(1L, new Vote(0x1, 4L, ZxidUtils.makeZxid(1, 2), 1, 2,
ServerState.FOLLOWING));
+votes.put(3L, new Vote(0x1, 4L, ZxidUtils.makeZxid(2, 1), 2, 2,
ServerState.FOLLOWING));
+votes.put(4L, new Vote(0x1, 4L, ZxidUtils.makeZxid(2, 1), 2, 2,
ServerState.LEADING));
+
+Assert.assertTrue(fle.getVoteTracker(votes,
+new Vote(4L, ZxidUtils.makeZxid(2, 1), 2, 2,
ServerState.FOLLOWING)).hasAllQuorums());
+}
+
+@Test
+public void testElectionWIthDifferentVersion() {
+Map votes = new HashMap();
+votes.put(0L, new Vote(0x1, 4L, ZxidUtils.makeZxid(1, 1), 1, 1,
ServerState.FOLLOWING));
+votes.put(1L, new Vote(0x1, 4L, ZxidUtils.makeZxid(1, 1), 1, 1,
ServerState.FOLLOWING));
+votes.put(3L, new Vote(4L, ZxidUtils.makeZxid(2, 1), 2, 2,
ServerState.FOLLOWING));
+votes.put(4L, new Vote(4L, ZxidUtils.makeZxid(2, 1), 2, 2,
ServerState.LEADING));
+
+Assert.assertTrue(fle.getVoteTracker(votes,
+new Vote(4L, ZxidUtils.makeZxid(2, 1), 2, 2,
ServerState.FOLLOWING)).hasAllQuorums());
+}
+
+@Test
+public void testLookingNormal() {
+Map votes = new HashMap();
+votes.put(0L, new Vote(4L, ZxidUtils.makeZxid(2, 1), 1, 1,
ServerState.LOOKING));
+votes.put(1L, new Vote(4L, ZxidUtils.makeZxid(2, 1), 1, 1,
ServerState.LOOKING));
+votes.put(3L, new Vote(4L, ZxidUtils.makeZxid(2, 1), 1, 1,
ServerState.LOOKING));
+votes.put(4L, new Vote(4L, ZxidUtils.makeZxid(2, 1), 1, 1,
ServerState.LEADING));
+
+Assert.assertTrue(fle.getVoteTracker(votes,
+new Vote(4L, ZxidUtils.makeZxid(2, 1), 1, 1,
ServerState.LOOKING)).hasAllQuorums());
+}
+
+@Test
+public void testLookingDiffRounds() {
+HashMap votes = new HashMap();
+
[GitHub] zookeeper pull request #703: [ZOOKEEPER-1818] Correctly handle potential inc...
Github user hanm commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/703#discussion_r237367990
--- Diff:
zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeer.java
---
@@ -1989,6 +1989,38 @@ private boolean updateVote(long designatedLeader,
long zxid){
/**
* Updates leader election info to avoid inconsistencies when
* a new server tries to join the ensemble.
+ *
+ * Here is the inconsistency scenario we try to solve by updating the
peer
--- End diff --
awesome comments.
---
[GitHub] zookeeper pull request #703: [ZOOKEEPER-1818] Correctly handle potential inc...
Github user hanm commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/703#discussion_r237367761
--- Diff:
zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/FastLeaderElection.java
---
@@ -1058,29 +1050,20 @@ else if (validVoter(n.sid) && validVoter(n.leader))
{
/*
* Before joining an established ensemble, verify
that
* a majority are following the same leader.
- * Only peer epoch is used to check that the votes
come
- * from the same ensemble. This is because there
is at
- * least one corner case in which the ensemble can
be
- * created with inconsistent zxid and election
epoch
- * info. However, given that only one ensemble can
be
- * running at a single point in time and that each
- * epoch is used only once, using only the epoch to
- * compare the votes is sufficient.
- *
- * @see
https://issues.apache.org/jira/browse/ZOOKEEPER-1732
*/
-outofelection.put(n.sid, new Vote(n.leader,
-IGNOREVALUE, IGNOREVALUE, n.peerEpoch,
n.state));
-voteSet = getVoteTracker(
-outofelection, new Vote(n.leader,
-IGNOREVALUE, IGNOREVALUE, n.peerEpoch,
n.state));
-if (voteSet.hasAllQuorums()
-&& checkLeader(outofelection, n.leader,
IGNOREVALUE)) {
+outofelection.put(n.sid, new Vote(n.version,
n.leader,
+n.zxid, n.electionEpoch, n.peerEpoch,
n.state));
+voteSet = getVoteTracker(outofelection, new
Vote(n.version,
+n.leader, n.zxid, n.electionEpoch,
n.peerEpoch, n.state));
+
+if (voteSet.hasAllQuorums() &&
+checkLeader(outofelection, n.leader,
n.electionEpoch)) {
synchronized(this){
logicalclock.set(n.electionEpoch);
setPeerState(n.leader, voteSet);
}
-Vote endVote = new Vote(n.leader, n.zxid,
n.peerEpoch);
+Vote endVote = new Vote(n.leader, n.zxid,
+n.electionEpoch, n.peerEpoch);
--- End diff --
nit: indentation
---
[GitHub] zookeeper pull request #703: [ZOOKEEPER-1818] Correctly handle potential inc...
Github user hanm commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/703#discussion_r237367684
--- Diff:
zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/FastLeaderElection.java
---
@@ -1042,14 +1035,13 @@ else if (validVoter(n.sid) && validVoter(n.leader))
{
*/
if(n.electionEpoch == logicalclock.get()){
recvset.put(n.sid, new Vote(n.leader, n.zxid,
n.electionEpoch, n.peerEpoch));
-voteSet = getVoteTracker(
-recvset, new Vote(n.leader, n.zxid,
-n.electionEpoch, n.peerEpoch,
n.state));
-if(voteSet.hasAllQuorums()
-&& checkLeader(outofelection,
n.leader, n.electionEpoch)) {
+voteSet = getVoteTracker(recvset, new
Vote(n.version,
+ n.leader, n.zxid, n.electionEpoch,
n.peerEpoch, n.state));
+if (voteSet.hasAllQuorums() &&
+checkLeader(outofelection, n.leader,
n.electionEpoch)) {
setPeerState(n.leader, voteSet);
-
-Vote endVote = new Vote(n.leader, n.zxid,
n.peerEpoch);
+Vote endVote = new Vote(n.leader,
+n.zxid, n.electionEpoch,
n.peerEpoch);
--- End diff --
nit: indentation
---
[GitHub] zookeeper pull request #703: [ZOOKEEPER-1818] Correctly handle potential inc...
Github user hanm commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/703#discussion_r237368267
--- Diff:
zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/Vote.java ---
@@ -125,11 +125,47 @@ public boolean equals(Object o) {
return false;
}
Vote other = (Vote) o;
-return (id == other.id
+
+if ((state == ServerState.LOOKING) ||
+(other.state == ServerState.LOOKING)) {
+return (id == other.id
&& zxid == other.zxid
&& electionEpoch == other.electionEpoch
&& peerEpoch == other.peerEpoch);
-
+} else {
+/*
+ * There are two things going on in the logic below:
+ *
+ * 1. skip comparing the zxid and electionEpoch for votes for
servers
+ *out of election.
+ *
+ *Need to skip those because they can be inconsistent due
to
+ *scenarios described in QuorumPeer.updateElectionVote.
+ *
+ *And given that only one ensemble can be running at a
single point
+ *in time and that each epoch is used only once, using
only id and
+ *epoch to compare the votes is sufficient.
+ *
--- End diff --
maybe move the reference of {@see
https://issues.apache.org/jira/browse/ZOOKEEPER-1805} from bottom to here, save
some brain cycles for pattern matching between comments and issues.
---
[GitHub] zookeeper pull request #703: [ZOOKEEPER-1818] Correctly handle potential inc...
Github user hanm commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/703#discussion_r237367977
--- Diff:
zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumPeer.java
---
@@ -1989,6 +1989,38 @@ private boolean updateVote(long designatedLeader,
long zxid){
/**
* Updates leader election info to avoid inconsistencies when
* a new server tries to join the ensemble.
+ *
+ * Here is the inconsistency scenario we try to solve by updating the
peer
+ * epoch after following leader:
+ *
+ * Let's say we have an ensemble with 3 servers z1, z2 and z3.
+ *
+ * 1. z1, z2 were following z3 with peerEpoch to be 0xb8, the new
epoch is
+ *0xb9, aka current accepted epoch on disk.
+ * 2. z2 get restarted, which will use 0xb9 as it's peer epoch when
loading
+ *the current accept epoch from disk.
+ * 3. z2 received notification from z1 and z3, which is following z3
with
+ *epoch 0xb8, so it started following z3 again with peer epoch
0xb8.
+ * 4. before z2 successfully connected to z3, z3 get restarted with
new
+ *epoch 0xb9.
+ * 5. z2 will retry around a few round (default 5s) before giving up,
+ *meanwhile it will report z3 as leader.
+ * 6. z1 restarted, and looking with peer epoch 0xb9.
+ * 7. z1 voted z3, and z3 was elected as leader again with peer epoch
0xb9.
+ * 8. z2 successfully connected to z3 before giving up, but with peer
+ *epoch 0xb8.
+ * 9. z1 get restarted, looking for leader with peer epoch 0xba, but
cannot
+ *join, because z2 is reporting peer epoch 0xb8, while z3 is
reporting
+ *0xb9.
+ *
+ * By updating the election vote after actually following leader, we
can
+ * avoid this kind of stuck happened.
+ *
+ * Btw, the zxid and electionEpoch could be inconsistent because of
the same
+ * reason, it's better to update these as well after syncing with
leader, but
+ * that required protocol change which is non trivial. This problem is
worked
+ * around by skipping comparing the zxid and electionEpoch when
counting for
+ * votes for out of election servers during looking for leader.
*
* @see https://issues.apache.org/jira/browse/ZOOKEEPER-1732
--- End diff --
would this be better syntax wise: {@see
https://issues.apache.org/jira/browse/ZOOKEEPER-1732}
---
[GitHub] zookeeper pull request #703: [ZOOKEEPER-1818] Correctly handle potential inc...
Github user hanm commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/703#discussion_r237367661
--- Diff:
zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/FastLeaderElection.java
---
@@ -1023,9 +1016,9 @@ else if (validVoter(n.sid) && validVoter(n.leader)) {
*/
if (n == null) {
setPeerState(proposedLeader, voteSet);
-
Vote endVote = new Vote(proposedLeader,
-proposedZxid, proposedEpoch);
+proposedZxid, logicalclock.get(),
--- End diff --
nit: indentations looks a little bit off here?
---
[GitHub] zookeeper issue #719: [ZOOKEEPER-2778] QuorumPeer: address potential reconfi...
Github user eolivelli commented on the issue: https://github.com/apache/zookeeper/pull/719 @anmolnar yes, it is okay ---
Re: Glide path to getting 3.5.x out of beta
With the use of a Docker container (to prevent port collisions) and a stack of cleanups to test code, I've made some progress towards reliable test runs in our environment. (https://github.com/mkedwards/zookeeper/commits/rollup-3.5, if you're curious.) The list below consists of the "top 40" slowest individual tests. Note that several appear multiple times, because of the inclusion of classes containing slow tests in NioNettySuiteTest and NettyNettySuiteTest. I'm somewhat hesitant to undertake further overhauls of the test suite, because I've already found myself having to make the kinds of changes that tend to be uphill battles, code-review-wise -- especially coming from an outsider. https://github.com/mkedwards/zookeeper/commit/e02eb705c6550f51ebb860a474ce711ec68c7a24 is an example. If a Zookeeper committer is interested in working with me on this, maybe email me? Otherwise, I'll try to keep this branch rebased regularly, and hammer on the remaining flaky tests to see what I can learn. $ grep ' Ran ' build.log | sort -n -t '[' -r -k 4 | head -40 [junit] [39343@1f20a9d731ad] Ran [68.657] org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortLeader ... OK [junit] [44264@1f20a9d731ad] Ran [68.607] org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortFollower ... OK [junit] [59151@1f20a9d731ad] Ran [67.535] org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortLeader ... OK [junit] [59151@1f20a9d731ad] Ran [67.397] org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortFollower ... OK [junit] [24817@1f20a9d731ad] Ran [66.555] org.apache.zookeeper.server.quorum.ReconfigRecoveryTest:testNextConfigUnreachable ... OK [junit] [39343@1f20a9d731ad] Ran [66.345] org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortFollower ... OK [junit] [44264@1f20a9d731ad] Ran [65.382] org.apache.zookeeper.test.ReconfigTest:testPortChangeToBlockedPortLeader ... OK [junit] [33311@1f20a9d731ad] Ran [64.39] org.apache.zookeeper.test.DisconnectedWatcherTest:testManyChildWatchersAutoReset ... OK [junit] [40332@1f20a9d731ad] Ran [60.907] org.apache.zookeeper.test.AsyncHammerTest:testHammer ... OK [junit] [26094@1f20a9d731ad] Ran [58.559] org.apache.zookeeper.server.quorum.StandaloneDisabledTest:startSingleServerTest ... OK [junit] [34470@1f20a9d731ad] Ran [52.229] org.apache.zookeeper.test.FollowerResyncConcurrencyTest:testResyncByTxnlogThenDiffAfterFollowerCrashes ... OK [junit] [19354@1f20a9d731ad] Ran [49.956] org.apache.zookeeper.server.quorum.QuorumSSLTest:testHostnameVerificationWithInvalidIPAddress ... OK [junit] [18406@1f20a9d731ad] Ran [48.65] org.apache.zookeeper.server.quorum.QuorumPeerMainTest:testFailedTxnAsPartOfQuorumLoss ... OK [junit] [34470@1f20a9d731ad] Ran [48.582] org.apache.zookeeper.test.FollowerResyncConcurrencyTest:testResyncBySnapThenDiffAfterFollowerCrashes ... OK [junit] [18406@1f20a9d731ad] Ran [47.115] org.apache.zookeeper.server.quorum.QuorumPeerMainTest:testEarlyLeaderAbandonment ... OK [junit] [19354@1f20a9d731ad] Ran [46.184] org.apache.zookeeper.server.quorum.QuorumSSLTest:testCipherSuites ... OK [junit] [28926@1f20a9d731ad] Ran [45.764] org.apache.zookeeper.test.AsyncHammerTest:testHammer ... OK [junit] [39059@1f20a9d731ad] Ran [44.87] org.apache.zookeeper.test.AsyncHammerTest:testHammer ... OK [junit] [39343@1f20a9d731ad] Ran [44.588] org.apache.zookeeper.test.ReconfigTest:testPortChange ... OK [junit] [50794@1f20a9d731ad] Ran [43.504] org.apache.zookeeper.test.QuorumZxidSyncTest:testBehindLeader ... OK [junit] [44264@1f20a9d731ad] Ran [43.183] org.apache.zookeeper.test.ReconfigTest:testPortChange ... OK [junit] [19354@1f20a9d731ad] Ran [42.234] org.apache.zookeeper.server.quorum.QuorumSSLTest:testHostnameVerificationWithInvalidHostname ... OK [junit] [59151@1f20a9d731ad] Ran [41.083] org.apache.zookeeper.test.ReconfigTest:testPortChange ... OK [junit] [48135@1f20a9d731ad] Ran [40.15] org.apache.zookeeper.test.QuorumHammerTest:testHammerBasic ... OK [junit] [19354@1f20a9d731ad] Ran [39.466] org.apache.zookeeper.server.quorum.QuorumSSLTest:testCertificateRevocationList ... OK [junit] [19354@1f20a9d731ad] Ran [39.114] org.apache.zookeeper.server.quorum.QuorumSSLTest:testQuorumSSL ... OK [junit] [19354@1f20a9d731ad] Ran [38.894] org.apache.zookeeper.server.quorum.QuorumSSLTest:testOCSP ... OK [junit] [19354@1f20a9d731ad] Ran [37.875] org.apache.zookeeper.server.quorum.QuorumSSLTest:testHostnameVerificationWithInvalidIpAddressAndInvalidHostname ... OK [junit] [19354@1f20a9d731ad] Ran [37.469] org.apache.zookeeper.server.quorum.QuorumSSLTest:testProtocolVersion ... OK [junit] [24817@1f20a9d731ad] Ran [36.251] org.apache.zookeeper.server.quorum.ReconfigRecoveryTest:testCurrentObserverIsParticipantInNewConfig ... OK [junit] [28926@1f20a9d731ad] Ran [34.052]
[GitHub] zookeeper issue #719: [ZOOKEEPER-2778] QuorumPeer: address potential reconfi...
Github user mkedwards commented on the issue: https://github.com/apache/zookeeper/pull/719 I believe #707 has @eolivelli's LGTM now. ---
[jira] [Commented] (ZOOKEEPER-3183) Interrupting or notifying the WatcherCleaner thread during shutdown if it is waiting for dead watchers get certain number(watcherCleanThreshold) and also stop add
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702645#comment-16702645 ] Hudson commented on ZOOKEEPER-3183: --- SUCCESS: Integrated in Jenkins build ZooKeeper-trunk #288 (See [https://builds.apache.org/job/ZooKeeper-trunk/288/]) ZOOKEEPER-3183: Interrupting the WatcherCleaner thread during shutdown (fangmin: rev 061e76123e91db4b9c28ab77e58df1b723df00d9) * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/watch/WatcherCleaner.java > Interrupting or notifying the WatcherCleaner thread during shutdown if it > is waiting for dead watchers get certain number(watcherCleanThreshold) and > also stop adding incoming deadWatcher to deadWatchersList when shutdown is > initiated. > - > > Key: ZOOKEEPER-3183 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3183 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Venkateswarlu Tumati >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 7h 20m > Remaining Estimate: 0h > > Interrupting or notifying the WatcherCleaner thread during shutdown if it > is waiting for dead watchers get certain number(watcherCleanThreshold) and > also stop adding incoming deadWatcher to deadWatchersList when shutdown is > initiated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zookeeper issue #726: ZOOKEEPER-3205: o.a.jute test cases
Github user lvfangmin commented on the issue: https://github.com/apache/zookeeper/pull/726 Only a few nits, other LGTM. ---
[jira] [Commented] (ZOOKEEPER-3183) Interrupting or notifying the WatcherCleaner thread during shutdown if it is waiting for dead watchers get certain number(watcherCleanThreshold) and also stop add
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702602#comment-16702602 ] Hudson commented on ZOOKEEPER-3183: --- SUCCESS: Integrated in Jenkins build Zookeeper-trunk-single-thread #127 (See [https://builds.apache.org/job/Zookeeper-trunk-single-thread/127/]) ZOOKEEPER-3183: Interrupting the WatcherCleaner thread during shutdown (fangmin: rev 061e76123e91db4b9c28ab77e58df1b723df00d9) * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/watch/WatcherCleaner.java > Interrupting or notifying the WatcherCleaner thread during shutdown if it > is waiting for dead watchers get certain number(watcherCleanThreshold) and > also stop adding incoming deadWatcher to deadWatchersList when shutdown is > initiated. > - > > Key: ZOOKEEPER-3183 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3183 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Venkateswarlu Tumati >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 7h 20m > Remaining Estimate: 0h > > Interrupting or notifying the WatcherCleaner thread during shutdown if it > is waiting for dead watchers get certain number(watcherCleanThreshold) and > also stop adding incoming deadWatcher to deadWatchersList when shutdown is > initiated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zookeeper pull request #712: Maintain the configuration will be used by serv...
Github user tumativ commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/712#discussion_r237324365
--- Diff:
zookeeper-server/src/main/java/org/apache/zookeeper/common/ZookeeperServerStabilizerConfig.java
---
@@ -0,0 +1,30 @@
+/**
+ *
+ */
+package org.apache.zookeeper.common;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ZookeeperServerStabilizerConfig {
+ protected static final Logger LOG =
LoggerFactory.getLogger(ZookeeperServerStabilizerConfig.class);
--- End diff --
Done
---
[GitHub] zookeeper issue #689: ZOOKEEPER-3183:Notifying the WatcherCleaner thread and...
Github user tumativ commented on the issue: https://github.com/apache/zookeeper/pull/689 @lvfangmin Thanks for merging. My JIRA user id is Tumati. ---
[jira] [Assigned] (ZOOKEEPER-3183) Interrupting or notifying the WatcherCleaner thread during shutdown if it is waiting for dead watchers get certain number(watcherCleanThreshold) and also stop addi
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangmin Lv reassigned ZOOKEEPER-3183: - Assignee: (was: Fangmin Lv) > Interrupting or notifying the WatcherCleaner thread during shutdown if it > is waiting for dead watchers get certain number(watcherCleanThreshold) and > also stop adding incoming deadWatcher to deadWatchersList when shutdown is > initiated. > - > > Key: ZOOKEEPER-3183 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3183 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Venkateswarlu Tumati >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Interrupting or notifying the WatcherCleaner thread during shutdown if it > is waiting for dead watchers get certain number(watcherCleanThreshold) and > also stop adding incoming deadWatcher to deadWatchersList when shutdown is > initiated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (ZOOKEEPER-3183) Interrupting or notifying the WatcherCleaner thread during shutdown if it is waiting for dead watchers get certain number(watcherCleanThreshold) and also stop addi
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangmin Lv resolved ZOOKEEPER-3183. --- Resolution: Fixed Issue resolved by pull request 689 [https://github.com/apache/zookeeper/pull/689] > Interrupting or notifying the WatcherCleaner thread during shutdown if it > is waiting for dead watchers get certain number(watcherCleanThreshold) and > also stop adding incoming deadWatcher to deadWatchersList when shutdown is > initiated. > - > > Key: ZOOKEEPER-3183 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3183 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Venkateswarlu Tumati >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Interrupting or notifying the WatcherCleaner thread during shutdown if it > is waiting for dead watchers get certain number(watcherCleanThreshold) and > also stop adding incoming deadWatcher to deadWatchersList when shutdown is > initiated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zookeeper issue #689: ZOOKEEPER-3183:Notifying the WatcherCleaner thread and...
Github user lvfangmin commented on the issue: https://github.com/apache/zookeeper/pull/689 Merged, thanks @tumativ. Btw, what's your Jira user id, I tried to assign this JIRA to you but I cannot find you. ---
[jira] [Assigned] (ZOOKEEPER-3183) Interrupting or notifying the WatcherCleaner thread during shutdown if it is waiting for dead watchers get certain number(watcherCleanThreshold) and also stop addi
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangmin Lv reassigned ZOOKEEPER-3183: - Assignee: Fangmin Lv > Interrupting or notifying the WatcherCleaner thread during shutdown if it > is waiting for dead watchers get certain number(watcherCleanThreshold) and > also stop adding incoming deadWatcher to deadWatchersList when shutdown is > initiated. > - > > Key: ZOOKEEPER-3183 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3183 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Venkateswarlu Tumati >Assignee: Fangmin Lv >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Interrupting or notifying the WatcherCleaner thread during shutdown if it > is waiting for dead watchers get certain number(watcherCleanThreshold) and > also stop adding incoming deadWatcher to deadWatchersList when shutdown is > initiated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zookeeper pull request #689: ZOOKEEPER-3183:Notifying the WatcherCleaner thr...
Github user asfgit closed the pull request at: https://github.com/apache/zookeeper/pull/689 ---
[GitHub] zookeeper pull request #722: [ZOOKEEPER-3203] Tracking the number of non vot...
Github user lvfangmin commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/722#discussion_r23726
--- Diff:
zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/LeaderBean.java
---
@@ -51,6 +51,15 @@ public String followerInfo() {
return sb.toString();
}
+@Override
+public String nonVotingFollowerInfo() {
+StringBuilder sb = new StringBuilder();
+for (LearnerHandler handler : leader.getNonVotingFollowers()) {
+sb.append(handler.toString()).append("\n");
--- End diff --
That's a fair point, but I'd like to keep the same behavior in this patch
as what we're doing now for the followInfo in this LeaderBean.
---
[GitHub] zookeeper issue #654: ZOOKEEPER-3160: Custom User SSLContext
Github user arankin-irl commented on the issue: https://github.com/apache/zookeeper/pull/654 Ah, I hadn't realised that 3.5 was closed for stabilisation. So I should recreate this PR for the master branch then? ---
[GitHub] zookeeper issue #703: [ZOOKEEPER-1818] Correctly handle potential inconsiste...
Github user hanm commented on the issue: https://github.com/apache/zookeeper/pull/703 @lvfangmin will review and provide feedback tonight. thanks ---
[GitHub] zookeeper issue #689: ZOOKEEPER-3183:Notifying the WatcherCleaner thread and...
Github user lvfangmin commented on the issue: https://github.com/apache/zookeeper/pull/689 LGTM, +1 Thanks @tumativ for all your effort on this, I'll commit this today. ---
[GitHub] zookeeper issue #703: [ZOOKEEPER-1818] Correctly handle potential inconsiste...
Github user lvfangmin commented on the issue: https://github.com/apache/zookeeper/pull/703 May need an extra +1 from another committer, @hanm do you have time to review this? ---
[GitHub] zookeeper issue #654: ZOOKEEPER-3160: Custom User SSLContext
Github user nkalmar commented on the issue: https://github.com/apache/zookeeper/pull/654 I just realized this is for 3.5. Did you create a PR for the master branch? 3.5 is pretty much closed for new features, in order to stabilize it for the stable release. ---
[GitHub] zookeeper issue #654: ZOOKEEPER-3160: Custom User SSLContext
Github user arankin-irl commented on the issue: https://github.com/apache/zookeeper/pull/654 Addressed merge conflicts. @nkalmar - Just wondering if there's anything else I can do to help this get merged? ---
ZooKeeper_branch35_jdk8 - Build # 1211 - Failure
See https://builds.apache.org/job/ZooKeeper_branch35_jdk8/1211/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 66.28 KB...] [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.532 sec, Thread: 4, Class: org.apache.zookeeper.test.SaslClientTest [junit] Running org.apache.zookeeper.test.SaslSuperUserTest in thread 1 [junit] Running org.apache.zookeeper.test.ServerCnxnTest in thread 2 [junit] Running org.apache.zookeeper.test.SessionInvalidationTest in thread 4 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.388 sec, Thread: 1, Class: org.apache.zookeeper.test.SaslSuperUserTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.393 sec, Thread: 4, Class: org.apache.zookeeper.test.SessionInvalidationTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.196 sec, Thread: 2, Class: org.apache.zookeeper.test.ServerCnxnTest [junit] Running org.apache.zookeeper.test.SessionTest in thread 1 [junit] Running org.apache.zookeeper.test.SessionTimeoutTest in thread 4 [junit] Running org.apache.zookeeper.test.SessionTrackerCheckTest in thread 2 [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.774 sec, Thread: 2, Class: org.apache.zookeeper.test.SessionTrackerCheckTest [junit] Running org.apache.zookeeper.test.SessionUpgradeTest in thread 2 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.593 sec, Thread: 4, Class: org.apache.zookeeper.test.SessionTimeoutTest [junit] Running org.apache.zookeeper.test.StandaloneTest in thread 4 [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.466 sec, Thread: 1, Class: org.apache.zookeeper.test.SessionTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.154 sec, Thread: 4, Class: org.apache.zookeeper.test.StandaloneTest [junit] Running org.apache.zookeeper.test.StatTest in thread 1 [junit] Running org.apache.zookeeper.test.StaticHostProviderTest in thread 4 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.807 sec, Thread: 1, Class: org.apache.zookeeper.test.StatTest [junit] Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.319 sec, Thread: 4, Class: org.apache.zookeeper.test.StaticHostProviderTest [junit] Running org.apache.zookeeper.test.StringUtilTest in thread 1 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.43 sec, Thread: 1, Class: org.apache.zookeeper.test.StringUtilTest [junit] Running org.apache.zookeeper.test.SyncCallTest in thread 4 [junit] Running org.apache.zookeeper.test.TruncateTest in thread 1 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.642 sec, Thread: 2, Class: org.apache.zookeeper.test.SessionUpgradeTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.204 sec, Thread: 4, Class: org.apache.zookeeper.test.SyncCallTest [junit] Running org.apache.zookeeper.test.WatchEventWhenAutoResetTest in thread 2 [junit] Running org.apache.zookeeper.test.WatchedEventTest in thread 4 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.664 sec, Thread: 4, Class: org.apache.zookeeper.test.WatchedEventTest [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.6 sec, Thread: 1, Class: org.apache.zookeeper.test.TruncateTest [junit] Running org.apache.zookeeper.test.WatcherFuncTest in thread 4 [junit] Running org.apache.zookeeper.test.WatcherTest in thread 1 [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.25 sec, Thread: 4, Class: org.apache.zookeeper.test.WatcherFuncTest [junit] Running org.apache.zookeeper.test.X509AuthTest in thread 4 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.769 sec, Thread: 4, Class: org.apache.zookeeper.test.X509AuthTest [junit] Running org.apache.zookeeper.test.ZkDatabaseCorruptionTest in thread 4 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.349 sec, Thread: 2, Class: org.apache.zookeeper.test.WatchEventWhenAutoResetTest [junit] Running org.apache.zookeeper.test.ZooKeeperQuotaTest in thread 2 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.332 sec, Thread: 2, Class: org.apache.zookeeper.test.ZooKeeperQuotaTest [junit] Running org.apache.zookeeper.util.PemReaderTest in thread 2 [junit] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.438 sec, Thread: 1, Class: org.apache.zookeeper.test.WatcherTest [junit] Running org.apache.jute.BinaryInputArchiveTest in thread 1 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.622 sec, Thread: 1,
[GitHub] zookeeper issue #726: ZOOKEEPER-3205: o.a.jute test cases
Github user akkumar commented on the issue: https://github.com/apache/zookeeper/pull/726 @eolivelli - import static added. Let me know if this looks ok ! ---
[GitHub] zookeeper pull request #726: ZOOKEEPER-3205: o.a.jute test cases
Github user eolivelli commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/726#discussion_r236977045
--- Diff:
zookeeper-jute/src/test/java/org/apache/jute/XmlInputArchiveTest.java ---
@@ -0,0 +1,120 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jute;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.xml.sax.SAXException;
+
+import javax.xml.parsers.ParserConfigurationException;
+import java.io.ByteArrayOutputStream;
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+import java.io.IOException;
+
+public class XmlInputArchiveTest {
+
+void checkWriterAndReader(TestWriter writer, TestReader reader) {
+ByteArrayOutputStream baos = new ByteArrayOutputStream();
+
+try {
+XmlOutputArchive oa = XmlOutputArchive.getArchive(baos);
+writer.write(oa);
+} catch (IOException e) {
+Assert.fail("Should not throw IOException");
+}
+InputStream is = new ByteArrayInputStream(baos.toByteArray());
+try {
+XmlInputArchive ia = XmlInputArchive.getArchive(is);
+reader.read(ia);
+} catch (ParserConfigurationException e) {
+Assert.fail("Should not throw ParserConfigurationException
while reading back");
+} catch (SAXException e) {
+Assert.fail("Should not throw SAXException while reading
back");
+} catch (IOException e) {
+Assert.fail("Should not throw IOException while reading back");
+}
+}
+
+@Test
+public void testWriteInt() {
+final int expected = 4;
+final String tag = "tag1";
+checkWriterAndReader(
+(oa) -> oa.writeInt(expected, tag),
+(ia) -> {
+int actual = ia.readInt(tag);
+Assert.assertEquals(expected, actual);
+}
+);
+}
+
+@Test
+public void testWriteBool() {
+final boolean expected = false;
+final String tag = "tag1";
+checkWriterAndReader(
+(oa) -> oa.writeBool(expected, tag),
+(ia) -> {
+boolean actual = ia.readBool(tag);
+Assert.assertEquals(expected, actual);
--- End diff --
Sorry, last nit: any reason for not using static imports?
---
