[jira] [Commented] (ZOOKEEPER-2789) Reassign `ZXID` for solving 32bit overflow problem

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022350#comment-16022350
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2789:
---

Github user nerdyyatrice commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/262#discussion_r118167311
  
--- Diff: 
src/contrib/loggraph/src/java/org/apache/zookeeper/graph/JsonGenerator.java ---
@@ -121,8 +121,8 @@ public JsonGenerator(LogIterator iter) {
} else if ((m = newElectionP.matcher(e.getEntry())).find()) {
Iterator iterator = servers.iterator();
long zxid = Long.valueOf(m.group(2));
-   int count = (int)zxid;// & 0xL;
-   int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
+   long count = zxid & 0xffL;
+   int epoch = (int)Long.rotateRight(zxid, 40);// >> 40;
--- End diff --

same,  40 shouldn't fly around in the code base like this


> Reassign `ZXID` for solving 32bit overflow problem
> --
>
> Key: ZOOKEEPER-2789
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2789
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: quorum
>Affects Versions: 3.5.3
>Reporter: Benedict Jin
> Fix For: 3.6.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> If it is `1k/s` ops, then as long as $2^32 / (86400 * 1000) \approx 49.7$ 
> days ZXID will exhausted. But, if we reassign the `ZXID` into 16bit for 
> `epoch` and 48bit for `counter`, then the problem will not occur until after  
> $Math.min(2^16 / 365, 2^48 / (86400 * 1000 * 365)) \approx Math.min(179.6, 
> 8925.5) = 179.6$ years.
> However, i thought the ZXID is `long` type, reading and writing the long type 
> (and `double` type the same) in JVM, is divided into high 32bit and low 32bit 
> part of the operation, and because the `ZXID` variable is not  modified with 
> `volatile` and is not boxed for the corresponding reference type (`Long` / 
> `Double`), so it belongs to [non-atomic operation] 
> (https://docs.oracle.com/javase/specs/jls/se8 /html/jls-17.html#jls-17.7). 
> Thus, if the lower 32 bits of the upper 32 bits are divided into the entire 
> 32 bits of the `long`, there may be a concurrent problem.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] zookeeper pull request #262: ZOOKEEPER-2789: Reassign `ZXID` for solving 32b...

[nerdyyatrice]

Github user nerdyyatrice commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/262#discussion_r118167311
  
--- Diff: 
src/contrib/loggraph/src/java/org/apache/zookeeper/graph/JsonGenerator.java ---
@@ -121,8 +121,8 @@ public JsonGenerator(LogIterator iter) {
} else if ((m = newElectionP.matcher(e.getEntry())).find()) {
Iterator iterator = servers.iterator();
long zxid = Long.valueOf(m.group(2));
-   int count = (int)zxid;// & 0xL;
-   int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
+   long count = zxid & 0xffL;
+   int epoch = (int)Long.rotateRight(zxid, 40);// >> 40;
--- End diff --

same,  40 shouldn't fly around in the code base like this


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ZOOKEEPER-2789) Reassign `ZXID` for solving 32bit overflow problem

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022348#comment-16022348
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2789:
---

Github user nerdyyatrice commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/262#discussion_r118167130
  
--- Diff: 
src/contrib/loggraph/src/java/org/apache/zookeeper/graph/JsonGenerator.java ---
@@ -121,8 +121,8 @@ public JsonGenerator(LogIterator iter) {
} else if ((m = newElectionP.matcher(e.getEntry())).find()) {
Iterator iterator = servers.iterator();
long zxid = Long.valueOf(m.group(2));
-   int count = (int)zxid;// & 0xL;
-   int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
+   long count = zxid & 0xffL;
--- End diff --

How can this be all over the code base instead of a function somewhere in a 
util file


> Reassign `ZXID` for solving 32bit overflow problem
> --
>
> Key: ZOOKEEPER-2789
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2789
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: quorum
>Affects Versions: 3.5.3
>Reporter: Benedict Jin
> Fix For: 3.6.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> If it is `1k/s` ops, then as long as $2^32 / (86400 * 1000) \approx 49.7$ 
> days ZXID will exhausted. But, if we reassign the `ZXID` into 16bit for 
> `epoch` and 48bit for `counter`, then the problem will not occur until after  
> $Math.min(2^16 / 365, 2^48 / (86400 * 1000 * 365)) \approx Math.min(179.6, 
> 8925.5) = 179.6$ years.
> However, i thought the ZXID is `long` type, reading and writing the long type 
> (and `double` type the same) in JVM, is divided into high 32bit and low 32bit 
> part of the operation, and because the `ZXID` variable is not  modified with 
> `volatile` and is not boxed for the corresponding reference type (`Long` / 
> `Double`), so it belongs to [non-atomic operation] 
> (https://docs.oracle.com/javase/specs/jls/se8 /html/jls-17.html#jls-17.7). 
> Thus, if the lower 32 bits of the upper 32 bits are divided into the entire 
> 32 bits of the `long`, there may be a concurrent problem.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] zookeeper pull request #262: ZOOKEEPER-2789: Reassign `ZXID` for solving 32b...

[nerdyyatrice]

Github user nerdyyatrice commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/262#discussion_r118167130
  
--- Diff: 
src/contrib/loggraph/src/java/org/apache/zookeeper/graph/JsonGenerator.java ---
@@ -121,8 +121,8 @@ public JsonGenerator(LogIterator iter) {
} else if ((m = newElectionP.matcher(e.getEntry())).find()) {
Iterator iterator = servers.iterator();
long zxid = Long.valueOf(m.group(2));
-   int count = (int)zxid;// & 0xL;
-   int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
+   long count = zxid & 0xffL;
--- End diff --

How can this be all over the code base instead of a function somewhere in a 
util file


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ZOOKEEPER-2767) Correct the exception messages in X509Util if truststore location or password is not configured

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022343#comment-16022343
 ] 

Hadoop QA commented on ZOOKEEPER-2767:
--

+1 overall.  GitHub Pull Request  Build
  

+1 @author.  The patch does not contain any @author tags.

+0 tests included.  The patch appears to be a documentation patch that 
doesn't require tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 3.0.1) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/740//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/740//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/740//console

This message is automatically generated.

> Correct the exception messages in X509Util if truststore location or password 
> is not configured
> ---
>
> Key: ZOOKEEPER-2767
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2767
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: java client, server
>Affects Versions: 3.5.4, 3.6.0
>Reporter: Abhishek Kumar
>Priority: Trivial
>
> In 
> org.apache.zookeeper.common.X509Util.org.apache.zookeeper.common.X509Util.createSSLContext
>  exception messages contains keystore related messages instead of truststore 
> messages for truststore location/password checks:
> {noformat}
> if (trustStoreLocationProp == null && trustStorePasswordProp == null) {
> LOG.warn("keystore not specified for client connection");
> } else {
> if (trustStoreLocationProp == null) {
> throw new SSLContextException("keystore location not 
> specified for client connection");
> }
> if (trustStorePasswordProp == null) {
> throw new SSLContextException("keystore password not 
> specified for client connection");
> }
> try {
> trustManagers = new TrustManager[]{
> createTrustManager(trustStoreLocationProp, 
> trustStorePasswordProp)};
> } catch (TrustManagerException e) {
> throw new SSLContextException("Failed to create KeyManager", 
> e);
> }
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Success: ZOOKEEPER- PreCommit Build #740

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/740/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 69.07 MB...]
 [exec] 
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec] 
 [exec] +0 tests included.  The patch appears to be a documentation 
patch that doesn't require tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
(version 3.0.1) warnings.
 [exec] 
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.
 [exec] 
 [exec] +1 core tests.  The patch passed core unit tests.
 [exec] 
 [exec] +1 contrib tests.  The patch passed contrib unit tests.
 [exec] 
 [exec] Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/740//testReport/
 [exec] Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/740//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
 [exec] Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/740//console
 [exec] 
 [exec] This message is automatically generated.
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Adding comment to Jira.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] Comment added.
 [exec] c2ff075dfaea334c433914ad2a27706d2541ec05 logged out
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Finished build.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] mv: 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 and 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 are the same file

BUILD SUCCESSFUL
Total time: 21 minutes 12 seconds
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[description-setter] Description set: ZOOKEEPER-2767
Putting comment on the pull request
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Success
Sending email for trigger: Success
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###
## FAILED TESTS (if any) 
##
All tests passed

[GitHub] zookeeper pull request #263: ZOOKEEPER-2767:Corrected keystore related messa...

[ramu11]

GitHub user ramu11 opened a pull request:

https://github.com/apache/zookeeper/pull/263

ZOOKEEPER-2767:Corrected keystore related messages to truststore mess…

possible fix for "https://issues.apache.org/jira/browse/ZOOKEEPER-2767;

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ramu11/zookeeper ZOOKEEPER-2767

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/263.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #263


commit bffd8e9157ea83d7fc13990c05ef45144c71c6d3
Author: Ramu 
Date:   2017-05-24T05:15:29Z

ZOOKEEPER-2767:Corrected keystore related messages to truststore messages 
for truststore location/password checks




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ZOOKEEPER-2767) Correct the exception messages in X509Util if truststore location or password is not configured

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022330#comment-16022330
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2767:
---

GitHub user ramu11 opened a pull request:

https://github.com/apache/zookeeper/pull/263

ZOOKEEPER-2767:Corrected keystore related messages to truststore mess…

possible fix for "https://issues.apache.org/jira/browse/ZOOKEEPER-2767;

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ramu11/zookeeper ZOOKEEPER-2767

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/263.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #263


commit bffd8e9157ea83d7fc13990c05ef45144c71c6d3
Author: Ramu 
Date:   2017-05-24T05:15:29Z

ZOOKEEPER-2767:Corrected keystore related messages to truststore messages 
for truststore location/password checks




> Correct the exception messages in X509Util if truststore location or password 
> is not configured
> ---
>
> Key: ZOOKEEPER-2767
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2767
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: java client, server
>Affects Versions: 3.5.4, 3.6.0
>Reporter: Abhishek Kumar
>Priority: Trivial
>
> In 
> org.apache.zookeeper.common.X509Util.org.apache.zookeeper.common.X509Util.createSSLContext
>  exception messages contains keystore related messages instead of truststore 
> messages for truststore location/password checks:
> {noformat}
> if (trustStoreLocationProp == null && trustStorePasswordProp == null) {
> LOG.warn("keystore not specified for client connection");
> } else {
> if (trustStoreLocationProp == null) {
> throw new SSLContextException("keystore location not 
> specified for client connection");
> }
> if (trustStorePasswordProp == null) {
> throw new SSLContextException("keystore password not 
> specified for client connection");
> }
> try {
> trustManagers = new TrustManager[]{
> createTrustManager(trustStoreLocationProp, 
> trustStorePasswordProp)};
> } catch (TrustManagerException e) {
> throw new SSLContextException("Failed to create KeyManager", 
> e);
> }
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] zookeeper issue #261: Branch 3.4

[hanm]

Github user hanm commented on the issue:

https://github.com/apache/zookeeper/pull/261
  
hmm, is this a test PR? It does nothing. Please close it @Jiangjiafu



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: FYI - zetcd: running ZooKeeper apps without ZooKeeper

[Michael Han]

>> That said, both etcd and consul are more k/v stores than they are
distributed coordinators.

Not sure about consul, but for etcd v3, it seems has evolved towards a
direction where more and more features that's critical for a coordinator is
being added, instead of just a generic key value store. Some changes in v3
such as introduction of the concept of lease (ZK session) and the
multiplexed gRPC streams allows more efficient implementation of watcher as
well as features such as ephemeral nodes which is very useful for
coordination use cases. Previously the v2 etcd seems stateless and it would
be hard to implement session and ephemeral effectively on top of stateless
server and API semantics. So it looks to me that etcd folks are
very consciously improving it for coordination use cases, which made zetcd
easier to implement as well.

On Tue, May 23, 2017 at 6:57 AM, Patrick Hunt  wrote:

> Lots of different requirements out there. VMWare recently pushed this to
> github:
>
> https://github.com/vmware/haret
> https://github.com/vmware/haret/blob/master/docs/why.md
>
> Patrick
>
>
> On Tue, May 23, 2017 at 3:33 AM, Flavio Junqueira  wrote:
>
> > Yeah, that's a good reference, thanks for starting the thread.
> >
> > @phunt
> > I see a lot of systems trying to implement the API of others in the same
> > space hoping that they can make the migration easier. I feel that it
> isn't
> > so much about us being a de facto standard, but more about the widespread
> > use. Another point is that some applications have decided that they like
> > etcd/Consul better, and they still need to have that pesky zookeeper
> > because of some other dependency, e.g., Kafka.
> >
> > @jordan
> > I'm really interested in your observation about the Consul herding.
> Could
> > you elaborate on the herding argument? Perhaps we should write a post
> about
> > it...
> >
> > -Flavio
> >
> > > On 20 May 2017, at 18:44, Jordan Zimmerman  >
> > wrote:
> > >
> > > I think they're trying to displace ZK for places that use things like
> > Kafka too. They can make the argument that you can install Hashicorp
> > everything and get rid of having to manage anything else. That said, both
> > etcd and consul are more k/v stores than they are distributed
> coordinators.
> > I was playing around with Consul this past week and it's locking recipes
> > are far inferior to ZooKeeper, for example (it suffers serious herding
> when
> > contending for a new lock holder).
> > >
> > > -Jordan
> > >
> > >> On May 20, 2017, at 6:40 PM, Patrick Hunt  wrote:
> > >>
> > >> I saw that a few days ago, seems like it could be a real boon for
> folks
> > >> running in K8s (for example). The long term stability of our APIs
> really
> > >> reduce the pain of implementing something like this. Does Hashicorp
> have
> > >> something like this yet?
> > >>
> > >> If I knew ten years ago that we would become the standard I would have
> > >> pushed harder to fix some of the rough(er) edges. ;-)
> > >>
> > >> Patrick
> > >>
> > >> On Fri, May 19, 2017 at 10:34 PM, Jordan Zimmerman <
> > >> jor...@jordanzimmerman.com> wrote:
> > >>
> > >>> "The zetcd proxy sits in front of an etcd cluster and serves an
> > emulated
> > >>> ZooKeeper client port, letting unmodified ZooKeeper applications run
> > on top
> > >>> of etcd."
> > >>>
> > >>> https://coreos.com/blog/introducing-zetcd
> > >
> >
> >
>



-- 
Cheers
Michael.

[jira] [Commented] (ZOOKEEPER-2757) Incorrect path crashes zkCli

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022319#comment-16022319
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2757:
---

Github user hanm commented on the issue:

https://github.com/apache/zookeeper/pull/240
  
Agree with what you said, Abe, in a generic context : ) - for CLI use cases 
though I am wondering why we should crash even in cases as you mentioned "So if 
it does pop up for reasons other than a bad path" - in such case should we 
always print the error message and not crashing the CLI? Would that be more 
helpful for user experience comparing to crash and leave no ideas to users what 
went wrong? Basically my point is that I don't any use case where the CLI's 
state is so bad that it can't recover from an unchecked exception so it has to 
be killed. 

Other than this patch looks good to me. One nit - a new compiler waring is 
introduced.


> Incorrect path crashes zkCli
> 
>
> Key: ZOOKEEPER-2757
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2757
> Project: ZooKeeper
>  Issue Type: Bug
>Affects Versions: 3.5.3
>Reporter: Flavio Junqueira
>Assignee: Abraham Fine
>Priority: Minor
> Fix For: 3.5.4
>
>
> If I try {{delete test}} without the leading /, then the CLI crashes with 
> this exception:
> {noformat}
> Exception in thread "main" java.lang.IllegalArgumentException: Path must 
> start with / character
>   at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:51)
>   at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:1659)
>   at org.apache.zookeeper.cli.DeleteCommand.exec(DeleteCommand.java:83)
>   at 
> org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:655)
>   at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:586)
>   at 
> org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:370)
>   at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:330)
>   at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:292)
> {noformat}
> It should really fail the operation rather than crash the CLI.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] zookeeper issue #240: ZOOKEEPER-2757: Incorrect path crashes zkCli

[hanm]

Github user hanm commented on the issue:

https://github.com/apache/zookeeper/pull/240
  
Agree with what you said, Abe, in a generic context : ) - for CLI use cases 
though I am wondering why we should crash even in cases as you mentioned "So if 
it does pop up for reasons other than a bad path" - in such case should we 
always print the error message and not crashing the CLI? Would that be more 
helpful for user experience comparing to crash and leave no ideas to users what 
went wrong? Basically my point is that I don't any use case where the CLI's 
state is so bad that it can't recover from an unchecked exception so it has to 
be killed. 

Other than this patch looks good to me. One nit - a new compiler waring is 
introduced.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ZOOKEEPER-2767) Correct the exception messages in X509Util if truststore location or password is not configured

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022311#comment-16022311
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2767:
---

Github user ramu11 closed the pull request at:

https://github.com/apache/zookeeper/pull/246


> Correct the exception messages in X509Util if truststore location or password 
> is not configured
> ---
>
> Key: ZOOKEEPER-2767
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2767
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: java client, server
>Affects Versions: 3.5.4, 3.6.0
>Reporter: Abhishek Kumar
>Priority: Trivial
>
> In 
> org.apache.zookeeper.common.X509Util.org.apache.zookeeper.common.X509Util.createSSLContext
>  exception messages contains keystore related messages instead of truststore 
> messages for truststore location/password checks:
> {noformat}
> if (trustStoreLocationProp == null && trustStorePasswordProp == null) {
> LOG.warn("keystore not specified for client connection");
> } else {
> if (trustStoreLocationProp == null) {
> throw new SSLContextException("keystore location not 
> specified for client connection");
> }
> if (trustStorePasswordProp == null) {
> throw new SSLContextException("keystore password not 
> specified for client connection");
> }
> try {
> trustManagers = new TrustManager[]{
> createTrustManager(trustStoreLocationProp, 
> trustStorePasswordProp)};
> } catch (TrustManagerException e) {
> throw new SSLContextException("Failed to create KeyManager", 
> e);
> }
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] zookeeper pull request #246: Corrected keystore related messages to truststo...

[ramu11]

Github user ramu11 closed the pull request at:

https://github.com/apache/zookeeper/pull/246


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zookeeper issue #246: Corrected keystore related messages to truststore mess...

[ramu11]

Github user ramu11 commented on the issue:

https://github.com/apache/zookeeper/pull/246
  
This closes  #246


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zookeeper pull request #246: Corrected keystore related messages to truststo...

[ramu11]

Github user ramu11 commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/246#discussion_r118161237
  
--- Diff: src/java/main/org/apache/zookeeper/common/X509Util.java ---
@@ -199,4 +199,4 @@ public static X509TrustManager 
createTrustManager(String trustStoreLocation, Str
 }
 }
 }
-}
\ No newline at end of file
+}
--- End diff --

Corrected the unnecessary change +}
Closing this pull req to raise new with corrected changes


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ZOOKEEPER-2691) recreateSocketAddresses may recreate the unreachable IP address

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022297#comment-16022297
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2691:
---

Github user hanm commented on the issue:

https://github.com/apache/zookeeper/pull/173
  
Thanks for updates, @JiangJiafu. Patch looks good to me, will merge soon.


> recreateSocketAddresses may recreate the unreachable IP address
> ---
>
> Key: ZOOKEEPER-2691
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2691
> Project: ZooKeeper
>  Issue Type: Bug
>Affects Versions: 3.4.8, 3.4.9, 3.4.10, 3.5.0, 3.5.1, 3.5.2, 3.4.11
> Environment: Centos6.5
> Java8
> ZooKeeper3.4.8
>Reporter: JiangJiafu
>Priority: Minor
>
> The QuorumPeer$QuorumServer.recreateSocketAddress()  is used to resolved the 
> hostname to a new IP address(InetAddress) when any exception happens to the 
> socket. It will be very useful when a hostname can be resolved to more than 
> one IP address.
> But the problem is Java API InetAddress.getByName(String hostname) will 
> always return the first IP address when the hostname can be resolved to more 
> than one IP address, and the first IP address may be unreachable forever. For 
> example, if a machine has two network interfaces: eth0, eth1, say eth0 has 
> ip1, eth1 has ip2, the relationship between hostname and the IP addresses is 
> set in /etc/hosts. When I "close" the eth0 by command "ifdown eth0", the 
> InetAddress.getByName(String hostname)  will still return ip1, which is 
> unreachable forever.
> So I think it will be better to check the IP address by 
> InetAddress.isReachable(long) and choose the reachable IP address. 
> I have modified the ZooKeeper source code, and test the new code in my own 
> environment, and it can work very well when I turn down some network 
> interfaces using "ifdown" command.
> The original code is:
> {code:title=QuorumPeer.java|borderStyle=solid}
> public void recreateSocketAddresses() {
> InetAddress address = null;
> try {
> address = InetAddress.getByName(this.hostname);
> LOG.info("Resolved hostname: {} to address: {}", 
> this.hostname, address);
> this.addr = new InetSocketAddress(address, this.port);
> if (this.electionPort > 0){
> this.electionAddr = new InetSocketAddress(address, 
> this.electionPort);
> }
> } catch (UnknownHostException ex) {
> LOG.warn("Failed to resolve address: {}", this.hostname, ex);
> // Have we succeeded in the past?
> if (this.addr != null) {
> // Yes, previously the lookup succeeded. Leave things as 
> they are
> return;
> }
> // The hostname has never resolved. Create our 
> InetSocketAddress(es) as unresolved
> this.addr = InetSocketAddress.createUnresolved(this.hostname, 
> this.port);
> if (this.electionPort > 0){
> this.electionAddr = 
> InetSocketAddress.createUnresolved(this.hostname,
>
> this.electionPort);
> }
> }
> }
> {code}
> After my modification:
> {code:title=QuorumPeer.java|borderStyle=solid}
> public void recreateSocketAddresses() {
> InetAddress address = null;
> try {
> address = getReachableAddress(this.hostname);
> LOG.info("Resolved hostname: {} to address: {}", 
> this.hostname, address);
> this.addr = new InetSocketAddress(address, this.port);
> if (this.electionPort > 0){
> this.electionAddr = new InetSocketAddress(address, 
> this.electionPort);
> }
> } catch (UnknownHostException ex) {
> LOG.warn("Failed to resolve address: {}", this.hostname, ex);
> // Have we succeeded in the past?
> if (this.addr != null) {
> // Yes, previously the lookup succeeded. Leave things as 
> they are
> return;
> }
> // The hostname has never resolved. Create our 
> InetSocketAddress(es) as unresolved
> this.addr = InetSocketAddress.createUnresolved(this.hostname, 
> this.port);
> if (this.electionPort > 0){
> this.electionAddr = 
> InetSocketAddress.createUnresolved(this.hostname,
>
> this.electionPort);
> }
> }
> }
> public InetAddress 

[GitHub] zookeeper issue #173: ZOOKEEPER-2691: recreateSocketAddresses may recreate t...

[hanm]

Github user hanm commented on the issue:

https://github.com/apache/zookeeper/pull/173
  
Thanks for updates, @JiangJiafu. Patch looks good to me, will merge soon.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zookeeper issue #167: ZOOKEEPER-2684 commitProcessor does not crash when an ...

[hanm]

Github user hanm commented on the issue:

https://github.com/apache/zookeeper/pull/167
  
@nerdyyatrice I see you've closed / reopened this PR, and the build bot is 
kicked off as expected.  There is the same test failure in pre-commit build: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/739/testReport/junit/org.apache.zookeeper.server.quorum/CommitProcessorConcurrencyTest/noCrashOnCommittedRequestsOfUnSeenRequestTest/.
 We should get this fixed before merging this PR.

I'll try reviewing this patch in detail to understand what goes wrong here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ZOOKEEPER-2684) Fix a crashing bug in the mixed workloads commit processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022290#comment-16022290
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2684:
---

Github user hanm commented on the issue:

https://github.com/apache/zookeeper/pull/167
  
@nerdyyatrice I see you've closed / reopened this PR, and the build bot is 
kicked off as expected.  There is the same test failure in pre-commit build: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/739/testReport/junit/org.apache.zookeeper.server.quorum/CommitProcessorConcurrencyTest/noCrashOnCommittedRequestsOfUnSeenRequestTest/.
 We should get this fixed before merging this PR.

I'll try reviewing this patch in detail to understand what goes wrong here.


> Fix a crashing bug in the mixed workloads commit processor
> --
>
> Key: ZOOKEEPER-2684
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2684
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.6.0
> Environment: with pretty heavy load on a real cluster
>Reporter: Ryan Zhang
>Assignee: Ryan Zhang
>Priority: Blocker
> Attachments: ZOOKEEPER-2684.patch
>
>
> We deployed our build with ZOOKEEPER-2024 and it quickly started to crash 
> with the following error
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:24:42,305 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x119fa expected 0x11fc5 for client session id 1009079ba470055
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:32:04,746 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x698 expected 0x928 for client session id 4002eeb3fd0009d
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:34:46,648 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x8904 expected 0x8f34 for client session id 51b8905c90251
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:43:46,834 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x3a8d expected 0x3ebc for client session id 2051af11af900cc
> clearly something is not right in the new commit processor per session queue 
> implementation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-1277) servers stop serving when lower 32bits of zxid roll over

[Benedict Jin (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1609#comment-1609
 ] 

Benedict Jin commented on ZOOKEEPER-1277:
-

I created a new jira ZOOKEEPER-2789 to discuss reassign `ZXID` for solving 
32bit overflow problem. Could you please offer some advice for it?

> servers stop serving when lower 32bits of zxid roll over
> 
>
> Key: ZOOKEEPER-1277
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1277
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.3.3
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
>Priority: Critical
> Fix For: 3.3.5, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1277_br33.patch, ZOOKEEPER-1277_br33.patch, 
> ZOOKEEPER-1277_br33.patch, ZOOKEEPER-1277_br33.patch, 
> ZOOKEEPER-1277_br34.patch, ZOOKEEPER-1277_br34.patch, 
> ZOOKEEPER-1277_trunk.patch, ZOOKEEPER-1277_trunk.patch
>
>
> When the lower 32bits of a zxid "roll over" (zxid is a 64 bit number, however 
> the upper 32 are considered the epoch number) the epoch number (upper 32 
> bits) are incremented and the lower 32 start at 0 again.
> This should work fine, however in the current 3.3 branch the followers see 
> this as a NEWLEADER message, which it's not, and effectively stop serving 
> clients. Attached clients seem to eventually time out given that heartbeats 
> (or any operation) are no longer processed. The follower doesn't recover from 
> this.
> I've tested this out on 3.3 branch and confirmed this problem, however I 
> haven't tried it on 3.4/3.5. It may not happen on the newer branches due to 
> ZOOKEEPER-335, however there is certainly an issue with updating the 
> "acceptedEpoch" files contained in the datadir. (I'll enter a separate jira 
> for that)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (ZOOKEEPER-2785) Server inappropriately throttles connections under load before SASL completes

[Patrick Hunt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt reassigned ZOOKEEPER-2785:
---

Assignee: Abhishek Singh Chouhan

> Server inappropriately throttles connections under load before SASL completes
> -
>
> Key: ZOOKEEPER-2785
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2785
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.10
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
>Priority: Critical
>  Labels: sasl
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> When a zk server is running close to its outstanding requests limit, the 
> server incorrectly throttles the sasl request. This leads to the client 
> waiting for the final sasl packet (session is already established) and 
> deferring all non priming packets till then which also includes the ping 
> packets. The client then waits for the final packet but never gets it and 
> times out saying haven't heard from server. This is fatal for services such 
> as HBase which retry for finite attempts and exit post these attempts.
> Issue being that in ZooKeeperServer.processPacket(..) incase of sasl we send 
> the response and incorrectly also call cnxn.incrOutstandingRequests(h), which 
> throttles the connection if we're running over outstandingrequests limit, 
> which results in the server not processing the subsequent packet from the 
> client. Also we donot have any pending request to send for the connection and 
> hence never call enableRecv(). We should return after sending response to the 
> sasl request.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-1277) servers stop serving when lower 32bits of zxid roll over

[Benedict Jin (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1607#comment-1607
 ] 

Benedict Jin commented on ZOOKEEPER-1277:
-

I see. Thank you! :D

> servers stop serving when lower 32bits of zxid roll over
> 
>
> Key: ZOOKEEPER-1277
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1277
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.3.3
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
>Priority: Critical
> Fix For: 3.3.5, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1277_br33.patch, ZOOKEEPER-1277_br33.patch, 
> ZOOKEEPER-1277_br33.patch, ZOOKEEPER-1277_br33.patch, 
> ZOOKEEPER-1277_br34.patch, ZOOKEEPER-1277_br34.patch, 
> ZOOKEEPER-1277_trunk.patch, ZOOKEEPER-1277_trunk.patch
>
>
> When the lower 32bits of a zxid "roll over" (zxid is a 64 bit number, however 
> the upper 32 are considered the epoch number) the epoch number (upper 32 
> bits) are incremented and the lower 32 start at 0 again.
> This should work fine, however in the current 3.3 branch the followers see 
> this as a NEWLEADER message, which it's not, and effectively stop serving 
> clients. Attached clients seem to eventually time out given that heartbeats 
> (or any operation) are no longer processed. The follower doesn't recover from 
> this.
> I've tested this out on 3.3 branch and confirmed this problem, however I 
> haven't tried it on 3.4/3.5. It may not happen on the newer branches due to 
> ZOOKEEPER-335, however there is certainly an issue with updating the 
> "acceptedEpoch" files contained in the datadir. (I'll enter a separate jira 
> for that)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-1277) servers stop serving when lower 32bits of zxid roll over

[Patrick Hunt (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1601#comment-1601
 ] 

Patrick Hunt commented on ZOOKEEPER-1277:
-

[~benedict jin] Not sure I follow that question. I believe it should be ok to 
add a new server during a re-election, even if that election were triggered by 
a epoch overflow. I've never tried that however.

> servers stop serving when lower 32bits of zxid roll over
> 
>
> Key: ZOOKEEPER-1277
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1277
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.3.3
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
>Priority: Critical
> Fix For: 3.3.5, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1277_br33.patch, ZOOKEEPER-1277_br33.patch, 
> ZOOKEEPER-1277_br33.patch, ZOOKEEPER-1277_br33.patch, 
> ZOOKEEPER-1277_br34.patch, ZOOKEEPER-1277_br34.patch, 
> ZOOKEEPER-1277_trunk.patch, ZOOKEEPER-1277_trunk.patch
>
>
> When the lower 32bits of a zxid "roll over" (zxid is a 64 bit number, however 
> the upper 32 are considered the epoch number) the epoch number (upper 32 
> bits) are incremented and the lower 32 start at 0 again.
> This should work fine, however in the current 3.3 branch the followers see 
> this as a NEWLEADER message, which it's not, and effectively stop serving 
> clients. Attached clients seem to eventually time out given that heartbeats 
> (or any operation) are no longer processed. The follower doesn't recover from 
> this.
> I've tested this out on 3.3 branch and confirmed this problem, however I 
> haven't tried it on 3.4/3.5. It may not happen on the newer branches due to 
> ZOOKEEPER-335, however there is certainly an issue with updating the 
> "acceptedEpoch" files contained in the datadir. (I'll enter a separate jira 
> for that)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2684) Fix a crashing bug in the mixed workloads commit processor

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022129#comment-16022129
 ] 

Hadoop QA commented on ZOOKEEPER-2684:
--

-1 overall.  GitHub Pull Request  Build
  

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 3.0.1) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/739//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/739//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/739//console

This message is automatically generated.

> Fix a crashing bug in the mixed workloads commit processor
> --
>
> Key: ZOOKEEPER-2684
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2684
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.6.0
> Environment: with pretty heavy load on a real cluster
>Reporter: Ryan Zhang
>Assignee: Ryan Zhang
>Priority: Blocker
> Attachments: ZOOKEEPER-2684.patch
>
>
> We deployed our build with ZOOKEEPER-2024 and it quickly started to crash 
> with the following error
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:24:42,305 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x119fa expected 0x11fc5 for client session id 1009079ba470055
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:32:04,746 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x698 expected 0x928 for client session id 4002eeb3fd0009d
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:34:46,648 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x8904 expected 0x8f34 for client session id 51b8905c90251
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:43:46,834 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x3a8d expected 0x3ebc for client session id 2051af11af900cc
> clearly something is not right in the new commit processor per session queue 
> implementation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

ZooKeeper-trunk - Build # 3400 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk/3400/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 61.98 MB...]
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-24 00:08:05,603 [myid:] - INFO  [ProcessThread(sid:0 
cport:24933)::PrepRequestProcessor@617] - Processed session termination for 
sessionid: 0x1028524ca88
[junit] 2017-05-24 00:08:05,603 [myid:] - INFO  
[SyncThread:0:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port24933,name1=Connections,name2=127.0.0.1,name3=0x1028524ca88]
[junit] 2017-05-24 00:08:05,603 [myid:] - INFO  [main:ZooKeeper@1329] - 
Session: 0x1028524ca88 closed
[junit] 2017-05-24 00:08:05,603 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x1028524ca88
[junit] 2017-05-24 00:08:05,603 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 184462
[junit] 2017-05-24 00:08:05,604 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 1644
[junit] 2017-05-24 00:08:05,604 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testWatcherAutoResetWithLocal
[junit] 2017-05-24 00:08:05,604 [myid:] - INFO  [main:ClientBase@582] - 
tearDown starting
[junit] 2017-05-24 00:08:05,604 [myid:] - INFO  [main:ClientBase@552] - 
STOPPING server
[junit] 2017-05-24 00:08:05,604 [myid:] - INFO  
[main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:24933
[junit] 2017-05-24 00:08:05,611 [myid:] - INFO  [main:ZooKeeperServer@542] 
- shutting down
[junit] 2017-05-24 00:08:05,611 [myid:] - ERROR [main:ZooKeeperServer@506] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2017-05-24 00:08:05,611 [myid:] - INFO  
[main:SessionTrackerImpl@232] - Shutting down
[junit] 2017-05-24 00:08:05,612 [myid:] - INFO  
[main:PrepRequestProcessor@1008] - Shutting down
[junit] 2017-05-24 00:08:05,612 [myid:] - INFO  
[main:SyncRequestProcessor@191] - Shutting down
[junit] 2017-05-24 00:08:05,612 [myid:] - INFO  [ProcessThread(sid:0 
cport:24933)::PrepRequestProcessor@157] - PrepRequestProcessor exited loop!
[junit] 2017-05-24 00:08:05,612 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2017-05-24 00:08:05,613 [myid:] - INFO  
[main:FinalRequestProcessor@481] - shutdown of request processor complete
[junit] 2017-05-24 00:08:05,613 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port24933,name1=InMemoryDataTree]
[junit] 2017-05-24 00:08:05,613 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port24933]
[junit] 2017-05-24 00:08:05,613 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 24933
[junit] 2017-05-24 00:08:05,614 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2017-05-24 00:08:05,620 [myid:] - INFO  [main:ClientBase@607] - 
fdcount after test is: 4835 at start it was 4835
[junit] 2017-05-24 00:08:05,620 [myid:] - INFO  [main:ZKTestCase$1@68] - 
SUCCEEDED testWatcherAutoResetWithLocal
[junit] 2017-05-24 00:08:05,620 [myid:] - INFO  [main:ZKTestCase$1@63] - 
FINISHED testWatcherAutoResetWithLocal
[junit] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
473.427 sec, Thread: 6, Class: org.apache.zookeeper.test.NioNettySuiteTest
[junit] 2017-05-24 00:08:05,990 [myid:127.0.0.1:24810] - INFO  
[main-SendThread(127.0.0.1:24810):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:24810. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-24 00:08:05,991 [myid:127.0.0.1:24810] - WARN  
[main-SendThread(127.0.0.1:24810):ClientCnxn$SendThread@1235] - Session 
0x10285217ff7 for server 127.0.0.1/127.0.0.1:24810, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)

fail.build.on.test.failure:

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1338: The 
following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1219: The 
following error 

[jira] [Commented] (ZOOKEEPER-2684) Fix a crashing bug in the mixed workloads commit processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022069#comment-16022069
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2684:
---

GitHub user nerdyyatrice reopened a pull request:

https://github.com/apache/zookeeper/pull/167

ZOOKEEPER-2684 commitProcessor does not crash when an unseen commit somes

commitProcessor with the zookeeper-2024 improvement patch throws an 
exception when it sees a commit request that is not at the queue head.  It 
turned out that it is actually a valid case when there is session movement. 
After discussion with the community, I submit this pull request to mitigate 
this issue by passing those commits to the next processor instead.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nerdyyatrice/zookeeper zookeeper-2684

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/167.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #167


commit 423e385038d055b034a71e91a503ff31532e84a2
Author: rzhang 
Date:   2017-02-10T01:33:23Z

commitProcessor does not crash when an unseen commit somes

commit cde12800a4ddecce26d41a4870cd19ae8d7e6f15
Author: rzhang 
Date:   2017-02-10T02:24:19Z

commitProcessor does not crash when an unseen commit somes

commit 0f7c2e815c3744d8088f12b6e802365ab164ff9a
Author: rzhang 
Date:   2017-02-16T20:53:56Z

remove the exception in the commit processor as commit requests can arrive 
out of order in terms of CXid

commit d80b715620bbbef173be4acbeb022892ff13934d
Author: rzhang 
Date:   2017-02-16T20:59:34Z

Merge branch 'zookeeper-2684' of https://github.com/nerdyyatrice/zookeeper 
into zookeeper-2684

commit 61f4764b51c92d93cdf5b63ab617efa41e1ee44c
Author: rzhang 
Date:   2017-02-16T23:45:06Z

adjust the test and to re-kick the submit

commit 5191fa83eb88967e91fc7df4c18a106369b0dd0c
Author: rzhang 
Date:   2017-04-21T00:18:51Z

 ZOOKEEPER-2684: Fix a crashing bug in the mixed workloads commit processor

commit fde3b0eba781361e5775445ecc6cfd20efe71f05
Author: rzhang 
Date:   2017-04-21T22:38:07Z

ZOOKEEPER-2684: Fix a crashing bug in the mixed workloads commit processor




> Fix a crashing bug in the mixed workloads commit processor
> --
>
> Key: ZOOKEEPER-2684
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2684
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.6.0
> Environment: with pretty heavy load on a real cluster
>Reporter: Ryan Zhang
>Assignee: Ryan Zhang
>Priority: Blocker
> Attachments: ZOOKEEPER-2684.patch
>
>
> We deployed our build with ZOOKEEPER-2024 and it quickly started to crash 
> with the following error
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:24:42,305 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x119fa expected 0x11fc5 for client session id 1009079ba470055
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:32:04,746 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x698 expected 0x928 for client session id 4002eeb3fd0009d
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:34:46,648 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x8904 expected 0x8f34 for client session id 51b8905c90251
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:43:46,834 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x3a8d expected 0x3ebc for client session id 2051af11af900cc
> clearly something is not right in the new commit processor per session queue 
> implementation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ZOOKEEPER-2684) Fix a crashing bug in the mixed workloads commit processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022068#comment-16022068
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2684:
---

Github user nerdyyatrice closed the pull request at:

https://github.com/apache/zookeeper/pull/167


> Fix a crashing bug in the mixed workloads commit processor
> --
>
> Key: ZOOKEEPER-2684
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2684
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.6.0
> Environment: with pretty heavy load on a real cluster
>Reporter: Ryan Zhang
>Assignee: Ryan Zhang
>Priority: Blocker
> Attachments: ZOOKEEPER-2684.patch
>
>
> We deployed our build with ZOOKEEPER-2024 and it quickly started to crash 
> with the following error
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:24:42,305 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x119fa expected 0x11fc5 for client session id 1009079ba470055
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:32:04,746 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x698 expected 0x928 for client session id 4002eeb3fd0009d
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:34:46,648 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x8904 expected 0x8f34 for client session id 51b8905c90251
> atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:43:46,834 - ERROR 
> [CommitProcessor:2] 
> -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:268)
>  – Got cxid 0x3a8d expected 0x3ebc for client session id 2051af11af900cc
> clearly something is not right in the new commit processor per session queue 
> implementation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] zookeeper pull request #167: ZOOKEEPER-2684 commitProcessor does not crash w...

[nerdyyatrice]

GitHub user nerdyyatrice reopened a pull request:

https://github.com/apache/zookeeper/pull/167

ZOOKEEPER-2684 commitProcessor does not crash when an unseen commit somes

commitProcessor with the zookeeper-2024 improvement patch throws an 
exception when it sees a commit request that is not at the queue head.  It 
turned out that it is actually a valid case when there is session movement. 
After discussion with the community, I submit this pull request to mitigate 
this issue by passing those commits to the next processor instead.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nerdyyatrice/zookeeper zookeeper-2684

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/167.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #167


commit 423e385038d055b034a71e91a503ff31532e84a2
Author: rzhang 
Date:   2017-02-10T01:33:23Z

commitProcessor does not crash when an unseen commit somes

commit cde12800a4ddecce26d41a4870cd19ae8d7e6f15
Author: rzhang 
Date:   2017-02-10T02:24:19Z

commitProcessor does not crash when an unseen commit somes

commit 0f7c2e815c3744d8088f12b6e802365ab164ff9a
Author: rzhang 
Date:   2017-02-16T20:53:56Z

remove the exception in the commit processor as commit requests can arrive 
out of order in terms of CXid

commit d80b715620bbbef173be4acbeb022892ff13934d
Author: rzhang 
Date:   2017-02-16T20:59:34Z

Merge branch 'zookeeper-2684' of https://github.com/nerdyyatrice/zookeeper 
into zookeeper-2684

commit 61f4764b51c92d93cdf5b63ab617efa41e1ee44c
Author: rzhang 
Date:   2017-02-16T23:45:06Z

adjust the test and to re-kick the submit

commit 5191fa83eb88967e91fc7df4c18a106369b0dd0c
Author: rzhang 
Date:   2017-04-21T00:18:51Z

 ZOOKEEPER-2684: Fix a crashing bug in the mixed workloads commit processor

commit fde3b0eba781361e5775445ecc6cfd20efe71f05
Author: rzhang 
Date:   2017-04-21T22:38:07Z

ZOOKEEPER-2684: Fix a crashing bug in the mixed workloads commit processor




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zookeeper pull request #167: ZOOKEEPER-2684 commitProcessor does not crash w...

[nerdyyatrice]

Github user nerdyyatrice closed the pull request at:

https://github.com/apache/zookeeper/pull/167


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

ZooKeeper-trunk-openjdk7 - Build # 1479 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1479/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 64.31 MB...]
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-23 20:08:31,721 [myid:127.0.0.1:16608] - INFO  
[main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:16608. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 20:08:31,722 [myid:127.0.0.1:16608] - WARN  
[main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1235] - Session 
0x461d63e99 for server 127.0.0.1/127.0.0.1:16608, unexpected error, closing 
socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-23 20:08:31,789 [myid:127.0.0.1:16608] - INFO  
[main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:16608. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 20:08:31,789 [myid:127.0.0.1:16608] - WARN  
[main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1235] - Session 
0x461d63e990001 for server 127.0.0.1/127.0.0.1:16608, unexpected error, closing 
socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-23 20:08:33,370 [myid:127.0.0.1:16608] - INFO  
[main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:16608. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 20:08:33,370 [myid:127.0.0.1:16608] - WARN  
[main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1235] - Session 
0x461d63e990001 for server 127.0.0.1/127.0.0.1:16608, unexpected error, closing 
socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-23 20:08:33,442 [myid:127.0.0.1:16611] - INFO  
[main-SendThread(127.0.0.1:16611):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:16611. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 20:08:33,443 [myid:127.0.0.1:16611] - WARN  
[main-SendThread(127.0.0.1:16611):ClientCnxn$SendThread@1235] - Session 
0x10461d6405a for server 127.0.0.1/127.0.0.1:16611, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-23 20:08:33,445 [myid:127.0.0.1:16608] - INFO  
[main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:16608. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 20:08:33,445 [myid:127.0.0.1:16608] - WARN  
[main-SendThread(127.0.0.1:16608):ClientCnxn$SendThread@1235] - Session 
0x461d63e99 for server 127.0.0.1/127.0.0.1:16608, unexpected error, closing 
socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
[junit] at 

[jira] [Commented] (ZOOKEEPER-2763) Utils.toCsvBuffer() omits leading 0 for bytes < 0x10

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021312#comment-16021312
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2763:
---

Github user chelin commented on the issue:

https://github.com/apache/zookeeper/pull/238
  
Regarding 
> for (char ch : s.toCharArray())

The difference is that toCharArray() results in a copy of the intrinsic 
array, leading to increased memory usage


> Utils.toCsvBuffer() omits leading 0 for bytes < 0x10
> 
>
> Key: ZOOKEEPER-2763
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2763
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: jute
>Affects Versions: 3.5.2
>Reporter: Brandon Berg
>Assignee: Alburt Hoffman
>Priority: Minor
>
> org.apache.jute.Utils.toCsvBuffer(), which converts a byte array to a string 
> containing the hex representation of that byte array, omits the leading zero 
> for any byte less than 0x10, due to its use of Integer.toHexString, which has 
> the same behavior.
> https://github.com/apache/zookeeper/blob/master/src/java/main/org/apache/jute/Utils.java#L234
> One consequence of this is that the hex strings printed by 
> ClientCnxn.Packet.toString(), used in the debug logging for 
> ClientCnxn.readResponse(), cannot be parsed to determine the result of a 
> Zookeeper request from client debug logs.
> Utils.toXmlBuffer() appears to have the same issue.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] zookeeper issue #238: ZOOKEEPER-2763: Utils.toCsvBuffer() omits leading 0 fo...

[chelin]

Github user chelin commented on the issue:

https://github.com/apache/zookeeper/pull/238
  
Regarding 
> for (char ch : s.toCharArray())

The difference is that toCharArray() results in a copy of the intrinsic 
array, leading to increased memory usage


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

ZooKeeper_branch34_openjdk7 - Build # 1507 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1507/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 32.09 MB...]
[junit] 2017-05-23 15:14:19,336 [myid:] - INFO  
[main:PrepRequestProcessor@766] - Shutting down
[junit] 2017-05-23 15:14:19,336 [myid:] - INFO  
[main:SyncRequestProcessor@208] - Shutting down
[junit] 2017-05-23 15:14:19,336 [myid:] - INFO  [ProcessThread(sid:0 
cport:11221)::PrepRequestProcessor@144] - PrepRequestProcessor exited loop!
[junit] 2017-05-23 15:14:19,338 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@186] - SyncRequestProcessor exited!
[junit] 2017-05-23 15:14:19,338 [myid:] - INFO  
[main:FinalRequestProcessor@403] - shutdown of request processor complete
[junit] 2017-05-23 15:14:19,339 [myid:] - INFO  
[main:FourLetterWordMain@62] - connecting to 127.0.0.1 11221
[junit] 2017-05-23 15:14:19,339 [myid:] - INFO  [main:JMXEnv@147] - 
ensureOnly:[]
[junit] 2017-05-23 15:14:19,340 [myid:] - INFO  [main:ClientBase@470] - 
STARTING server
[junit] 2017-05-23 15:14:19,340 [myid:] - INFO  [main:ClientBase@391] - 
CREATING server instance 127.0.0.1:11221
[junit] 2017-05-23 15:14:19,341 [myid:] - INFO  
[main:ServerCnxnFactory@111] - Using 
org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
[junit] 2017-05-23 15:14:19,341 [myid:] - INFO  
[main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11221
[junit] 2017-05-23 15:14:19,341 [myid:] - INFO  [main:ClientBase@366] - 
STARTING server instance 127.0.0.1:11221
[junit] 2017-05-23 15:14:19,341 [myid:] - INFO  [main:ZooKeeperServer@173] 
- Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 
6 datadir 
/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk7/build/test/tmp/test2702001098124566362.junit.dir/version-2
 snapdir 
/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk7/build/test/tmp/test2702001098124566362.junit.dir/version-2
[junit] 2017-05-23 15:14:19,344 [myid:] - ERROR [main:ZooKeeperServer@468] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2017-05-23 15:14:19,344 [myid:] - INFO  
[main:FourLetterWordMain@62] - connecting to 127.0.0.1 11221
[junit] 2017-05-23 15:14:19,345 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@192] - 
Accepted socket connection from /127.0.0.1:35173
[junit] 2017-05-23 15:14:19,345 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@883] - Processing 
stat command from /127.0.0.1:35173
[junit] 2017-05-23 15:14:19,346 [myid:] - INFO  
[Thread-4:NIOServerCnxn$StatCommand@674] - Stat command output
[junit] 2017-05-23 15:14:19,346 [myid:] - INFO  
[Thread-4:NIOServerCnxn@1044] - Closed socket connection for client 
/127.0.0.1:35173 (no session established for client)
[junit] 2017-05-23 15:14:19,346 [myid:] - INFO  [main:JMXEnv@230] - 
ensureParent:[InMemoryDataTree, StandaloneServer_port]
[junit] 2017-05-23 15:14:19,348 [myid:] - INFO  [main:JMXEnv@247] - 
expect:InMemoryDataTree
[junit] 2017-05-23 15:14:19,348 [myid:] - INFO  [main:JMXEnv@251] - 
found:InMemoryDataTree 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221,name1=InMemoryDataTree
[junit] 2017-05-23 15:14:19,348 [myid:] - INFO  [main:JMXEnv@247] - 
expect:StandaloneServer_port
[junit] 2017-05-23 15:14:19,348 [myid:] - INFO  [main:JMXEnv@251] - 
found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221
[junit] 2017-05-23 15:14:19,348 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 24640
[junit] 2017-05-23 15:14:19,348 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 20
[junit] 2017-05-23 15:14:19,349 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testQuota
[junit] 2017-05-23 15:14:19,349 [myid:] - INFO  [main:ClientBase@547] - 
tearDown starting
[junit] 2017-05-23 15:14:19,423 [myid:] - INFO  [main:ZooKeeper@684] - 
Session: 0x101f23c4e10 closed
[junit] 2017-05-23 15:14:19,424 [myid:] - INFO  [main:ClientBase@517] - 
STOPPING server
[junit] 2017-05-23 15:14:19,424 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@219] - 
NIOServerCnxn factory exited run method
[junit] 2017-05-23 15:14:19,424 [myid:] - INFO  [main:ZooKeeperServer@501] 
- shutting down
[junit] 2017-05-23 15:14:19,426 [myid:] - ERROR [main:ZooKeeperServer@468] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2017-05-23 15:14:19,426 [myid:] - INFO  
[main:SessionTrackerImpl@226] - Shutting down
[junit] 

Re: FYI - zetcd: running ZooKeeper apps without ZooKeeper

[Patrick Hunt]

Lots of different requirements out there. VMWare recently pushed this to
github:

https://github.com/vmware/haret
https://github.com/vmware/haret/blob/master/docs/why.md

Patrick


On Tue, May 23, 2017 at 3:33 AM, Flavio Junqueira  wrote:

> Yeah, that's a good reference, thanks for starting the thread.
>
> @phunt
> I see a lot of systems trying to implement the API of others in the same
> space hoping that they can make the migration easier. I feel that it isn't
> so much about us being a de facto standard, but more about the widespread
> use. Another point is that some applications have decided that they like
> etcd/Consul better, and they still need to have that pesky zookeeper
> because of some other dependency, e.g., Kafka.
>
> @jordan
> I'm really interested in your observation about the Consul herding.  Could
> you elaborate on the herding argument? Perhaps we should write a post about
> it...
>
> -Flavio
>
> > On 20 May 2017, at 18:44, Jordan Zimmerman 
> wrote:
> >
> > I think they're trying to displace ZK for places that use things like
> Kafka too. They can make the argument that you can install Hashicorp
> everything and get rid of having to manage anything else. That said, both
> etcd and consul are more k/v stores than they are distributed coordinators.
> I was playing around with Consul this past week and it's locking recipes
> are far inferior to ZooKeeper, for example (it suffers serious herding when
> contending for a new lock holder).
> >
> > -Jordan
> >
> >> On May 20, 2017, at 6:40 PM, Patrick Hunt  wrote:
> >>
> >> I saw that a few days ago, seems like it could be a real boon for folks
> >> running in K8s (for example). The long term stability of our APIs really
> >> reduce the pain of implementing something like this. Does Hashicorp have
> >> something like this yet?
> >>
> >> If I knew ten years ago that we would become the standard I would have
> >> pushed harder to fix some of the rough(er) edges. ;-)
> >>
> >> Patrick
> >>
> >> On Fri, May 19, 2017 at 10:34 PM, Jordan Zimmerman <
> >> jor...@jordanzimmerman.com> wrote:
> >>
> >>> "The zetcd proxy sits in front of an etcd cluster and serves an
> emulated
> >>> ZooKeeper client port, letting unmodified ZooKeeper applications run
> on top
> >>> of etcd."
> >>>
> >>> https://coreos.com/blog/introducing-zetcd
> >
>
>

ZooKeeper_branch35_jdk8 - Build # 534 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch35_jdk8/534/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 61.92 MB...]
[junit] 2017-05-23 12:19:27,685 [myid:] - INFO  [main:ZooKeeperServer@541] 
- shutting down
[junit] 2017-05-23 12:19:27,685 [myid:] - ERROR [main:ZooKeeperServer@505] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2017-05-23 12:19:27,685 [myid:] - INFO  
[main:SessionTrackerImpl@232] - Shutting down
[junit] 2017-05-23 12:19:27,685 [myid:] - INFO  
[main:PrepRequestProcessor@1004] - Shutting down
[junit] 2017-05-23 12:19:27,685 [myid:] - INFO  
[main:SyncRequestProcessor@191] - Shutting down
[junit] 2017-05-23 12:19:27,685 [myid:] - INFO  [ProcessThread(sid:0 
cport:30319)::PrepRequestProcessor@156] - PrepRequestProcessor exited loop!
[junit] 2017-05-23 12:19:27,685 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2017-05-23 12:19:27,685 [myid:] - INFO  
[main:FinalRequestProcessor@481] - shutdown of request processor complete
[junit] 2017-05-23 12:19:27,685 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port30319,name1=InMemoryDataTree]
[junit] 2017-05-23 12:19:27,686 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port30319]
[junit] 2017-05-23 12:19:27,686 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 30319
[junit] 2017-05-23 12:19:27,686 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2017-05-23 12:19:27,687 [myid:127.0.0.1:30202] - INFO  
[main-SendThread(127.0.0.1:30202):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:30202. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 12:19:27,687 [myid:] - INFO  [New I/O boss 
#2781:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:30202
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2017-05-23 12:19:27,687 [myid:] - WARN  [New I/O boss 
#2781:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 
0xd8a04574] EXCEPTION: java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:30202
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:30202
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2017-05-23 12:19:27,688 [myid:] - INFO  [New I/O boss 

ZooKeeper-trunk-jdk8 - Build # 1057 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-jdk8/1057/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 63.39 MB...]
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-23 12:04:51,694 [myid:127.0.0.1:19301] - INFO  
[main-SendThread(127.0.0.1:19301):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19301. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 12:04:51,694 [myid:127.0.0.1:19301] - WARN  
[main-SendThread(127.0.0.1:19301):ClientCnxn$SendThread@1235] - Session 
0x103ae960226 for server 127.0.0.1/127.0.0.1:19301, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-23 12:04:51,995 [myid:127.0.0.1:19430] - INFO  
[main-SendThread(127.0.0.1:19430):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19430. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 12:04:51,996 [myid:127.0.0.1:19430] - WARN  
[main-SendThread(127.0.0.1:19430):ClientCnxn$SendThread@1235] - Session 
0x303ae99d721 for server 127.0.0.1/127.0.0.1:19430, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-23 12:04:52,789 [myid:127.0.0.1:19424] - INFO  
[main-SendThread(127.0.0.1:19424):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19424. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 12:04:52,790 [myid:127.0.0.1:19424] - WARN  
[main-SendThread(127.0.0.1:19424):ClientCnxn$SendThread@1235] - Session 
0x103ae99d719 for server 127.0.0.1/127.0.0.1:19424, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-23 12:04:52,933 [myid:127.0.0.1:19427] - INFO  
[main-SendThread(127.0.0.1:19427):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19427. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 12:04:52,934 [myid:127.0.0.1:19427] - WARN  
[main-SendThread(127.0.0.1:19427):ClientCnxn$SendThread@1235] - Session 
0x203ae99d717 for server 127.0.0.1/127.0.0.1:19427, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-05-23 12:04:53,232 [myid:127.0.0.1:19430] - INFO  
[main-SendThread(127.0.0.1:19430):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19430. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-05-23 12:04:53,233 [myid:127.0.0.1:19430] - WARN  
[main-SendThread(127.0.0.1:19430):ClientCnxn$SendThread@1235] - Session 
0x303ae99d721 for server 127.0.0.1/127.0.0.1:19430, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 

Re: FYI - zetcd: running ZooKeeper apps without ZooKeeper

[Jordan Zimmerman]

> @jordan
> I'm really interested in your observation about the Consul herding.  Could 
> you elaborate on the herding argument? Perhaps we should write a post about 
> it...

Here's their recipe for Leader Election (which is the same as a lock in this 
case): https://www.consul.io/docs/guides/leader-election.html 


curl -X PUT -d  http://localhost:8500/v1/kv/?acquire= 

Note that a given session can not hold the same lock more than once. ZooKeeper 
is better here - a single client can have multiple threads participating in the 
same lock
If you don't get the lock via the first call, you start "Watching for changes 
... via a blocking query against ."
In their API, the way you watch for changes is to use what they call "blocking 
queries": https://www.consul.io/api/index.html#blocking-queries 
 - whereby "the HTTP 
request will "hang" until a change in the system occurs, or the maximum timeout 
is reached. ". In other words, when the lock is released all watchers are 
notified - i.e. a herd (also not fair). ZooKeeper is superior here because of 
sequential ephemeral nodes. A client waiting for a lock only watches the node 
the precedes him. This has the benefit of being fair and not causing a herd.

-JZ

Re: FYI - zetcd: running ZooKeeper apps without ZooKeeper

[Flavio Junqueira]

Yeah, that's a good reference, thanks for starting the thread.

@phunt
I see a lot of systems trying to implement the API of others in the same space 
hoping that they can make the migration easier. I feel that it isn't so much 
about us being a de facto standard, but more about the widespread use. Another 
point is that some applications have decided that they like etcd/Consul better, 
and they still need to have that pesky zookeeper because of some other 
dependency, e.g., Kafka.

@jordan
I'm really interested in your observation about the Consul herding.  Could you 
elaborate on the herding argument? Perhaps we should write a post about it...

-Flavio

> On 20 May 2017, at 18:44, Jordan Zimmerman  wrote:
> 
> I think they're trying to displace ZK for places that use things like Kafka 
> too. They can make the argument that you can install Hashicorp everything and 
> get rid of having to manage anything else. That said, both etcd and consul 
> are more k/v stores than they are distributed coordinators. I was playing 
> around with Consul this past week and it's locking recipes are far inferior 
> to ZooKeeper, for example (it suffers serious herding when contending for a 
> new lock holder).
> 
> -Jordan
> 
>> On May 20, 2017, at 6:40 PM, Patrick Hunt  wrote:
>> 
>> I saw that a few days ago, seems like it could be a real boon for folks
>> running in K8s (for example). The long term stability of our APIs really
>> reduce the pain of implementing something like this. Does Hashicorp have
>> something like this yet?
>> 
>> If I knew ten years ago that we would become the standard I would have
>> pushed harder to fix some of the rough(er) edges. ;-)
>> 
>> Patrick
>> 
>> On Fri, May 19, 2017 at 10:34 PM, Jordan Zimmerman <
>> jor...@jordanzimmerman.com> wrote:
>> 
>>> "The zetcd proxy sits in front of an etcd cluster and serves an emulated
>>> ZooKeeper client port, letting unmodified ZooKeeper applications run on top
>>> of etcd."
>>> 
>>> https://coreos.com/blog/introducing-zetcd
> 

[jira] [Commented] (ZOOKEEPER-2789) Reassign `ZXID` for solving 32bit overflow problem

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020942#comment-16020942
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2789:
---

Github user asdf2014 commented on the issue:

https://github.com/apache/zookeeper/pull/262
  
Seems like all test cases 
[passed](https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/738/testReport/),
 but some problems happened in `Zookeeper_operations` :: 
`testOperationsAndDisconnectConcurrently1`:
```bash
 [exec] BUILD FAILED
 [exec] 
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1298:
 The following error occurred while executing this line:
 [exec] 
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1308:
 exec returned: 2
 [exec] 
 [exec] Total time: 15 minutes 45 seconds
 [exec] /bin/kill -9 16911 
 [exec]  [exec] Zookeeper_operations::testAsyncWatcher1 : assertion 
: elapsed 1044
 [exec]  [exec] Zookeeper_operations::testAsyncGetOperation : 
elapsed 4 : OK
 [exec]  [exec] 
Zookeeper_operations::testOperationsAndDisconnectConcurrently1FAIL: zktest-mt
 [exec]  [exec] ==
 [exec]  [exec] 1 of 2 tests failed
 [exec]  [exec] Please report to u...@zookeeper.apache.org
 [exec]  [exec] ==
 [exec]  [exec] make[1]: Leaving directory 
`/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build/test/test-cppunit'
 [exec]  [exec] /bin/bash: line 5: 15116 Segmentation fault  
ZKROOT=/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/src/c/../..
 CLASSPATH=$CLASSPATH:$CLOVER_HOME/lib/clover.jar ${dir}$tst
 [exec]  [exec] make[1]: *** [check-TESTS] Error 1
 [exec]  [exec] make: *** [check-am] Error 2
 [exec] 
 [exec] Running contrib tests.
 [exec] 
==
 [exec] 
 [exec] /home/jenkins/tools/ant/apache-ant-1.9.9/bin/ant 
-DZookeeperPatchProcess= -Dtest.junit.output.format=xml -Dtest.output=yes 
test-contrib
 [exec] Buildfile: 
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml
 [exec] 
 [exec] test-contrib:
 [exec] 
 [exec] BUILD SUCCESSFUL
 [exec] Total time: 0 seconds
```


> Reassign `ZXID` for solving 32bit overflow problem
> --
>
> Key: ZOOKEEPER-2789
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2789
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: quorum
>Affects Versions: 3.5.3
>Reporter: Benedict Jin
> Fix For: 3.6.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> If it is `1k/s` ops, then as long as $2^32 / (86400 * 1000) \approx 49.7$ 
> days ZXID will exhausted. But, if we reassign the `ZXID` into 16bit for 
> `epoch` and 48bit for `counter`, then the problem will not occur until after  
> $Math.min(2^16 / 365, 2^48 / (86400 * 1000 * 365)) \approx Math.min(179.6, 
> 8925.5) = 179.6$ years.
> However, i thought the ZXID is `long` type, reading and writing the long type 
> (and `double` type the same) in JVM, is divided into high 32bit and low 32bit 
> part of the operation, and because the `ZXID` variable is not  modified with 
> `volatile` and is not boxed for the corresponding reference type (`Long` / 
> `Double`), so it belongs to [non-atomic operation] 
> (https://docs.oracle.com/javase/specs/jls/se8 /html/jls-17.html#jls-17.7). 
> Thus, if the lower 32 bits of the upper 32 bits are divided into the entire 
> 32 bits of the `long`, there may be a concurrent problem.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] zookeeper issue #262: ZOOKEEPER-2789: Reassign `ZXID` for solving 32bit over...

[asdf2014]

Github user asdf2014 commented on the issue:

https://github.com/apache/zookeeper/pull/262
  
Seems like all test cases 
[passed](https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/738/testReport/),
 but some problems happened in `Zookeeper_operations` :: 
`testOperationsAndDisconnectConcurrently1`:
```bash
 [exec] BUILD FAILED
 [exec] 
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1298:
 The following error occurred while executing this line:
 [exec] 
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1308:
 exec returned: 2
 [exec] 
 [exec] Total time: 15 minutes 45 seconds
 [exec] /bin/kill -9 16911 
 [exec]  [exec] Zookeeper_operations::testAsyncWatcher1 : assertion 
: elapsed 1044
 [exec]  [exec] Zookeeper_operations::testAsyncGetOperation : 
elapsed 4 : OK
 [exec]  [exec] 
Zookeeper_operations::testOperationsAndDisconnectConcurrently1FAIL: zktest-mt
 [exec]  [exec] ==
 [exec]  [exec] 1 of 2 tests failed
 [exec]  [exec] Please report to u...@zookeeper.apache.org
 [exec]  [exec] ==
 [exec]  [exec] make[1]: Leaving directory 
`/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build/test/test-cppunit'
 [exec]  [exec] /bin/bash: line 5: 15116 Segmentation fault  
ZKROOT=/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/src/c/../..
 CLASSPATH=$CLASSPATH:$CLOVER_HOME/lib/clover.jar ${dir}$tst
 [exec]  [exec] make[1]: *** [check-TESTS] Error 1
 [exec]  [exec] make: *** [check-am] Error 2
 [exec] 
 [exec] Running contrib tests.
 [exec] 
==
 [exec] 
 [exec] /home/jenkins/tools/ant/apache-ant-1.9.9/bin/ant 
-DZookeeperPatchProcess= -Dtest.junit.output.format=xml -Dtest.output=yes 
test-contrib
 [exec] Buildfile: 
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml
 [exec] 
 [exec] test-contrib:
 [exec] 
 [exec] BUILD SUCCESSFUL
 [exec] Total time: 0 seconds
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (ZOOKEEPER-2789) Reassign `ZXID` for solving 32bit overflow problem

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020828#comment-16020828
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2789:
---

Github user asdf2014 commented on the issue:

https://github.com/apache/zookeeper/pull/262
  
Why `jenkins` reported the following message:
```bash
mv: 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 and 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 are the same file
```


> Reassign `ZXID` for solving 32bit overflow problem
> --
>
> Key: ZOOKEEPER-2789
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2789
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: quorum
>Affects Versions: 3.5.3
>Reporter: Benedict Jin
> Fix For: 3.6.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> If it is `1k/s` ops, then as long as $2^32 / (86400 * 1000) \approx 49.7$ 
> days ZXID will exhausted. But, if we reassign the `ZXID` into 16bit for 
> `epoch` and 48bit for `counter`, then the problem will not occur until after  
> $Math.min(2^16 / 365, 2^48 / (86400 * 1000 * 365)) \approx Math.min(179.6, 
> 8925.5) = 179.6$ years.
> However, i thought the ZXID is `long` type, reading and writing the long type 
> (and `double` type the same) in JVM, is divided into high 32bit and low 32bit 
> part of the operation, and because the `ZXID` variable is not  modified with 
> `volatile` and is not boxed for the corresponding reference type (`Long` / 
> `Double`), so it belongs to [non-atomic operation] 
> (https://docs.oracle.com/javase/specs/jls/se8 /html/jls-17.html#jls-17.7). 
> Thus, if the lower 32 bits of the upper 32 bits are divided into the entire 
> 32 bits of the `long`, there may be a concurrent problem.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] zookeeper issue #262: ZOOKEEPER-2789: Reassign `ZXID` for solving 32bit over...

[asdf2014]

Github user asdf2014 commented on the issue:

https://github.com/apache/zookeeper/pull/262
  
Why `jenkins` reported the following message:
```bash
mv: 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 and 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 are the same file
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---