[jira] [Commented] (ZOOKEEPER-3180) Add response cache to improve the throughput of read heavy traffic
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698826#comment-16698826 ] Rakesh R commented on ZOOKEEPER-3180: - [~lvfangmin], Feature looks pretty interesting and thanks for the patch. I will try to review it when I get a chance. I'd like to understand scaling of this cache in the production env. Appreciate if you could help me, adding more details about the memory usage: # How much is the java heap size of ZooKeeper server and the possible number of znodes (am assuming 4MB data per znode). # Based on your experiment, what would be the ideal number of elements to be kept into the cache for a better read performance gain. Since you have mentioned that the intention of read cache feature is to reduce the GC overhead, I would also like to explore the option of enabling the cache backed by {{off heap}} memory. Again, you have {{4MB}} data in a single zNode and it would occupy good amount of {{on heap}} memory area if the number of zNode grows(in future). The off heap implementation uses {{DirectByteByffers}} to manage cache outside of the JVM heap and provides scalability to a large memory sizes without GC overhead. One option could be to provide different cache modes (onheap or offheap) to the users to efficiently utilize the heap area. Does this make sense to you? > Add response cache to improve the throughput of read heavy traffic > --- > > Key: ZOOKEEPER-3180 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3180 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Fangmin Lv >Assignee: Brian Nixon >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 3h > Remaining Estimate: 0h > > On read heavy use case with large response data size, the serialization of > response takes time and added overhead to the GC. > Add response cache helps improving the throughput we can support, which also > reduces the latency in general. > This Jira is going to implement a LRU cache for the response, which shows > some performance gain on some of our production ensembles. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-236) SSL Support for Atomic Broadcast protocol
[ https://issues.apache.org/jira/browse/ZOOKEEPER-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668331#comment-16668331 ] Rakesh R commented on ZOOKEEPER-236: Awesome! Great work everyone! [~andorm], appreciate if you could point me to the user docs. It would be good to capture feature details in [zk cwiki|https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide#ZooKeeperSSLUserGuide-Quorum] or zk docs. > SSL Support for Atomic Broadcast protocol > - > > Key: ZOOKEEPER-236 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-236 > Project: ZooKeeper > Issue Type: New Feature > Components: quorum, security, server >Reporter: Benjamin Reed >Assignee: Andor Molnar >Priority: Major > Labels: pull-request-available, ssl > Fix For: 3.6.0, 3.5.5 > > Time Spent: 27h 20m > Remaining Estimate: 0h > > We should have the ability to use SSL to authenticate and encrypt the traffic > between ZooKeeper servers. For the most part this is a very easy change. We > would probably only want to support this for TCP based leader elections. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-3021) Umbrella: Migrate project structure to Maven build
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445844#comment-16445844 ] Rakesh R commented on ZOOKEEPER-3021: - [~nkalmar] appreciate your efforts. Glad to see, you are identifying sub-tasks and pushing small review-able chunks. Another critical part in the migration is adding dependencies in the POM, we have to make sure adding right version of the jars. > Umbrella: Migrate project structure to Maven build > -- > > Key: ZOOKEEPER-3021 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3021 > Project: ZooKeeper > Issue Type: Improvement > Components: build, build-infrastructure, scripts >Affects Versions: 3.6.0 >Reporter: Norbert Kalmar >Priority: Major > > In multiple steps, Maven should replace current ant build in ZooKeeper. > > First iteration: separate project structure > {noformat} > zookeeper > |-bin > |-conf > |-zk-client-c > |-zk-contrib > | |-zk-contrib-fatjar > | |-zk-contrib-huebrowser > | |-zk-contrib-loggraph > | |-zk-contrib-monitoring > | |-zk-contrib-rest > | |-zk-contrib-zkfuse > | |-zk-contrib-zkperl > | |-zk-contrib-zkpython > | |-zk-contrib-zktreeutil > | \-zk-contrib-zooinspector > |-zk-docs > |-zk-it (integration tests) > |-zk-server > |-zk-recipes > | |-zk-recipes-election > | |-zk-recipes-lock > \ \-zk-recipes-queue > {noformat} > > With this kind of structure, the code change could be kept to a bare > minimum, if any at all. > Just change the ant script to conform to the new structure. > > Second iteration: code changes: > {noformat} > zookeeper > |-bin > |-conf > |-jute > |-zk-client > | |-zk-client-c > | |-zk-client-java > | \-zk-client-go (or any other language) > |-zk-common > |-zk-contrib > | |-zk-contrib-fatjar > | |-zk-contrib-huebrowser > | |-zk-contrib-loggraph > | |-zk-contrib-monitoring > | |-zk-contrib-rest > | |-zk-contrib-zkfuse > | |-zk-contrib-zkperl > | |-zk-contrib-zkpython > | |-zk-contrib-zktreeutil > | \-zk-contrib-zooinspector > |-zk-docs > |-zk-it (integration tests) > |-zk-server > |-zk-recipes > | |-zk-recipes-election > | |-zk-recipes-lock > \ \-zk-recipes-queue > {noformat} > > Here, java client code is separated from the server code (and any other > supported languages client code). > > Third iteration: maven structure > {noformat} > zk-something > |-src > | |-main > | | |-java > | | | \org... > | | \resources > | \test (unit tests only?) > | |-java > | | \org... > | \resources > \pom.xml (build.xml, build.gradle?) > {noformat} > There is already ZOOKEEPER-1078, but it's main approach is to create a maven > proxy on top of ant. > The main idea here is to replace ant with "pure" maven, and update the > project structure accordingly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ZOOKEEPER-2977) Concurrency for addAuth corrupts quorum packets
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2977: --- Assignee: sumit agrawal > Concurrency for addAuth corrupts quorum packets > --- > > Key: ZOOKEEPER-2977 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2977 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.4.9 > Environment: Affects all version in 3.4.x >Reporter: sumit agrawal >Assignee: sumit agrawal >Priority: Critical > Fix For: 3.4.12 > > Attachments: 2977.patch > > > When client performs multiple times addAuth with different credential at > follower concurrently, the communication between follower gets corrupt. This > causes shutdown of Follower due to the failure. > Analysis: > In org.apache.zookeeper.server.quorum.QuorumPacket.serialize method, > * call a_.startVector(authinfo,"authinfo"); which write the length of > authinfo to packet (suppose it writes length 1) > * get length of authinfo to write all details in loop (here gets length as 2) > <-- Here in concurrency scenario, buffer gets corrupt having extra bytes in > channel for additional authinfo. > > So When Leader reads next quorum packet, it reads previous extra bytes > (incorrect) and possibly identify greater size of message (as corrupt byte > pattern) causes exception... > Coordination > Unexpected exception causing shutdown while sock still open > (LearnerHandler.java:633) > java.io.IOException: Unreasonable length = 1885430131 > > > ServerCnxn.getAuthInfo returns Unmodifiable list, but while addAuthInfo, > there is no check. So this causes concurrency issue. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2793) [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic reconfig servers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278196#comment-16278196 ] Rakesh R commented on ZOOKEEPER-2793: - Thanks [~phunt] for the feedback. Sure, I will keep this scenario in mind and update docs. I am planning to prepare an initial patch ready in few days. > [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic > reconfig servers > - > > Key: ZOOKEEPER-2793 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2793 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.5.4, 3.6.0 > > > {{QuorumServer}} will do the authorization checks against configured > authorized hosts. During LE, QuorumLearner will send an authentication packet > to QuorumServer. Now, QuorumServer will check that the connecting > QuorumLearner’s hostname exists in the authorized hosts. If not exists then > connecting peer is not authorized to join this ensemble and the request will > be rejected immediately. > In {{branch-3.4}} building {{authzHosts}} list is pretty straight forward, > can use the ensemble server details in zoo.cfg file. But with dynamic > reconfig, it has to consider the dynamic add/remove/update servers and need > to discuss the ways to handle dynamic cases. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2793) [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic reconfig servers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255066#comment-16255066 ] Rakesh R commented on ZOOKEEPER-2793: - bq. My main remaining concern is that we could end up locking up the cluster and not being able to recover. bq. Perhaps there is a control somewhere else that enables/disables this feature? In the worst-case scenario the admin could disable, recover the cluster, reset the "authzhosts" and then re-enable. That seems like a copout though. Is there a better way? Thanks for emphasising the recovery part. Yes, quorum auth feature already has enable/disable via configuration. Admin can disable it and then allows to join new set of servers and form the quorum. Then, {{/zookeeper/ensemble/authorized_hosts}} znode data can be corrected with required host details. Later, they can enable the flag and re-form the quorum by restarting the servers. Since there is no way to dynamically enable/disable this feature, admin has to restart the server two times. One with disable the flag and secondly after enabling the flag. Does this make sense to you? {code} "quorum.auth.enableSasl=false" // If false, no authentication at all. {code} > [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic > reconfig servers > - > > Key: ZOOKEEPER-2793 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2793 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security > Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.5.4, 3.6.0 > > > {{QuorumServer}} will do the authorization checks against configured > authorized hosts. During LE, QuorumLearner will send an authentication packet > to QuorumServer. Now, QuorumServer will check that the connecting > QuorumLearner’s hostname exists in the authorized hosts. If not exists then > connecting peer is not authorized to join this ensemble and the request will > be rejected immediately. > In {{branch-3.4}} building {{authzHosts}} list is pretty straight forward, > can use the ensemble server details in zoo.cfg file. But with dynamic > reconfig, it has to consider the dynamic add/remove/update servers and need > to discuss the ways to handle dynamic cases. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2793) [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic reconfig servers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240612#comment-16240612 ] Rakesh R commented on ZOOKEEPER-2793: - [~phunt], would be great to see your feedback on my above comments, when you get a chance. Thanks! > [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic > reconfig servers > - > > Key: ZOOKEEPER-2793 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2793 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.5.4, 3.6.0 > > > {{QuorumServer}} will do the authorization checks against configured > authorized hosts. During LE, QuorumLearner will send an authentication packet > to QuorumServer. Now, QuorumServer will check that the connecting > QuorumLearner’s hostname exists in the authorized hosts. If not exists then > connecting peer is not authorized to join this ensemble and the request will > be rejected immediately. > In {{branch-3.4}} building {{authzHosts}} list is pretty straight forward, > can use the ensemble server details in zoo.cfg file. But with dynamic > reconfig, it has to consider the dynamic add/remove/update servers and need > to discuss the ways to handle dynamic cases. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ZOOKEEPER-2793) [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic reconfig servers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16213715#comment-16213715 ] Rakesh R edited comment on ZOOKEEPER-2793 at 10/21/17 4:42 AM: --- Thanks [~phunt] for the comments. Below is my reply, bq. Perhaps it should be /zookeeper/ensemble/authorized_hosts ? Agreed. bq. Can you be more explicit in the definition of a valid value. e.g. do we allow IP addresses or just host names or Following are the two ways supported for the principal names. 1) Single shared Kerberos principal name configured in all the servers >>> servicename/localh...@example.com 2) Host based Kerberos principal name with _HOST wildcard >>>> servicename/fully.qualified.domain.n...@example.com Below are the possible values of {{/zookeeper/ensemble/authorized_hosts}}. {{localhost}} or {{FQDN1,FQDN2,FQDN3,..}} bq. "Admin can update" - what specifically will that map to (acl/id/...) and will operators be able to change this? I missed this case. How about introducing add/removal/list ZooKeeperAdmin apis similar to reconfig {{"/zookeeper/config"}} and add default ZooDefs.Ids.READ_ACL_UNSAFE to znode {{/zookeeper/ensemble/authorized_hosts}}? bq. What will happen if the authorized_hosts scope is reduced, say a host that's already part of the quorum is removed from this list, what (if anything, perhaps nothing) will happen? {{authorized_hosts}} is used during FLE and the authorized_hosts is no more used after FLE process. Assume, quorum lost and also host entry is removed from authorized_hosts, now during re-election authorized_hosts will be again used and throws exception saying "host is not authorized to connect". To handle this, we can restrict removing quorum's host from {{authorized_hosts}}. Should we ? bq. Is it possible to log changes to this znode in particular? So that we capture in the server logs. auditing purposes, etc... I think, it is possible and let me try including this. bq. What keeps me from accidentally messing up the value and then not being able to change the value thereafter. Say the value is corrupted and no peers can form a quorum. How will I update the value of the znode at this stage. Apart from the following case any other way the corruption can occur - if we allows to remove and then add back the host which is part of the quorum, isn't it? Proposed Fix : How about restricting removal of a host which is presently part of the quorum so that the quorum won't be disturbed. was (Author: rakeshr): Thanks [~phunt] for the comments. Below is my reply, bq. Perhaps it should be /zookeeper/ensemble/authorized_hosts ? Agreed. bq. Can you be more explicit in the definition of a valid value. e.g. do we allow IP addresses or just host names or Following are the two ways supported for the principal names. 1) Single shared Kerberos principal name configured in all the servers >>> servicename/localh...@example.com 2) Host based Kerberos principal name with _HOST wildcard >>>> servicename/fully.qualified.domain.n...@example.com Below are the possible values of {{/zookeeper/ensemble/authorized_hosts}}. {{localhost}} or {{FQDN1,FQDN2,FQDN3,..}} bq. "Admin can update" - what specifically will that map to (acl/id/...) and will operators be able to change this? I missed this case. How about introducing add/removal/list ZooKeeperAdmin apis similar to reconfig {{"/zookeeper/config"}} and add default ZooDefs.Ids.READ_ACL_UNSAFE to znode {{/zookeeper/ensemble/authorized_hosts}}? bq. What will happen if the authorized_hosts scope is reduced, say a host that's already part of the quorum is removed from this list, what (if anything, perhaps nothing) will happen? {{authorized_hosts}} is used during FLE and the authorized_hosts is no more used after FLE process. Assume, if the quorum lost then during re-election {{authorized_hosts}} will be again used and throws exception saying "host is not authorized to connect". Should we restrict removing a used host from {{authorized_hosts}} ? bq. Is it possible to log changes to this znode in particular? So that we capture in the server logs. auditing purposes, etc... Let me consider this point. I think, it is possible. bq. What keeps me from accidentally messing up the value and then not being able to change the value thereafter. Say the value is corrupted and no peers can form a quorum. How will I update the value of the znode at this stage. Apart from the following case any other case the corruption can occur - if we allows to remove and then add back the host which is part of the quorum, isn't it? Proposed Fix : How about restricting removal of a host which is presently part of the quorum so that the quorum won't be dist
[jira] [Commented] (ZOOKEEPER-2793) [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic reconfig servers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16213715#comment-16213715 ] Rakesh R commented on ZOOKEEPER-2793: - Thanks [~phunt] for the comments. Below is my reply, bq. Perhaps it should be /zookeeper/ensemble/authorized_hosts ? Agreed. bq. Can you be more explicit in the definition of a valid value. e.g. do we allow IP addresses or just host names or Following are the two ways supported for the principal names. 1) Single shared Kerberos principal name configured in all the servers >>> servicename/localh...@example.com 2) Host based Kerberos principal name with _HOST wildcard >>>> servicename/fully.qualified.domain.n...@example.com Below are the possible values of {{/zookeeper/ensemble/authorized_hosts}}. {{localhost}} or {{FQDN1,FQDN2,FQDN3,..}} bq. "Admin can update" - what specifically will that map to (acl/id/...) and will operators be able to change this? I missed this case. How about introducing add/removal/list ZooKeeperAdmin apis similar to reconfig {{"/zookeeper/config"}} and add default ZooDefs.Ids.READ_ACL_UNSAFE to znode {{/zookeeper/ensemble/authorized_hosts}}? bq. What will happen if the authorized_hosts scope is reduced, say a host that's already part of the quorum is removed from this list, what (if anything, perhaps nothing) will happen? {{authorized_hosts}} is used during FLE and the authorized_hosts is no more used after FLE process. Assume, if the quorum lost then during re-election {{authorized_hosts}} will be again used and throws exception saying "host is not authorized to connect". Should we restrict removing a used host from {{authorized_hosts}} ? bq. Is it possible to log changes to this znode in particular? So that we capture in the server logs. auditing purposes, etc... Let me consider this point. I think, it is possible. bq. What keeps me from accidentally messing up the value and then not being able to change the value thereafter. Say the value is corrupted and no peers can form a quorum. How will I update the value of the znode at this stage. Apart from the following case any other case the corruption can occur - if we allows to remove and then add back the host which is part of the quorum, isn't it? Proposed Fix : How about restricting removal of a host which is presently part of the quorum so that the quorum won't be disturbed. > [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic > reconfig servers > - > > Key: ZOOKEEPER-2793 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2793 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.5.4, 3.6.0 > > > {{QuorumServer}} will do the authorization checks against configured > authorized hosts. During LE, QuorumLearner will send an authentication packet > to QuorumServer. Now, QuorumServer will check that the connecting > QuorumLearner’s hostname exists in the authorized hosts. If not exists then > connecting peer is not authorized to join this ensemble and the request will > be rejected immediately. > In {{branch-3.4}} building {{authzHosts}} list is pretty straight forward, > can use the ensemble server details in zoo.cfg file. But with dynamic > reconfig, it has to consider the dynamic add/remove/update servers and need > to discuss the ways to handle dynamic cases. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2793) [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic reconfig servers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208133#comment-16208133 ] Rakesh R commented on ZOOKEEPER-2793: - Also, thanks a lot [~shralex] for the #reconfig discussions during ZK-1045 implementation in {{branch-3.4}}. I'm thinking to programatically build the {{/zookeeper/authorized_hosts}} while upgrading from 3.4 to 3.5 from {{zoo.cfg}} file. Later on, before #reconfig user has to call the #setData or cli commands to update the hosts. > [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic > reconfig servers > - > > Key: ZOOKEEPER-2793 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2793 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.5.4, 3.6.0 > > > {{QuorumServer}} will do the authorization checks against configured > authorized hosts. During LE, QuorumLearner will send an authentication packet > to QuorumServer. Now, QuorumServer will check that the connecting > QuorumLearner’s hostname exists in the authorized hosts. If not exists then > connecting peer is not authorized to join this ensemble and the request will > be rejected immediately. > In {{branch-3.4}} building {{authzHosts}} list is pretty straight forward, > can use the ensemble server details in zoo.cfg file. But with dynamic > reconfig, it has to consider the dynamic add/remove/update servers and need > to discuss the ways to handle dynamic cases. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ZOOKEEPER-2793) [QP MutualAuth]: Build a mechanism to build "authzHosts" for dynamic reconfig servers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2793: --- Assignee: Rakesh R > [QP MutualAuth]: Build a mechanism to build "authzHosts" for dynamic reconfig > servers > - > > Key: ZOOKEEPER-2793 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2793 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.5.4, 3.6.0 > > > {{QuorumServer}} will do the authorization checks against configured > authorized hosts. During LE, QuorumLearner will send an authentication packet > to QuorumServer. Now, QuorumServer will check that the connecting > QuorumLearner’s hostname exists in the authorized hosts. If not exists then > connecting peer is not authorized to join this ensemble and the request will > be rejected immediately. > In {{branch-3.4}} building {{authzHosts}} list is pretty straight forward, > can use the ensemble server details in zoo.cfg file. But with dynamic > reconfig, it has to consider the dynamic add/remove/update servers and need > to discuss the ways to handle dynamic cases. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ZOOKEEPER-2793) [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic reconfig servers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2793: Summary: [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic reconfig servers (was: [QP MutualAuth]: Build a mechanism to build "authzHosts" for dynamic reconfig servers) > [QP MutualAuth]: Implement a mechanism to build "authzHosts" for dynamic > reconfig servers > - > > Key: ZOOKEEPER-2793 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2793 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security > Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.5.4, 3.6.0 > > > {{QuorumServer}} will do the authorization checks against configured > authorized hosts. During LE, QuorumLearner will send an authentication packet > to QuorumServer. Now, QuorumServer will check that the connecting > QuorumLearner’s hostname exists in the authorized hosts. If not exists then > connecting peer is not authorized to join this ensemble and the request will > be rejected immediately. > In {{branch-3.4}} building {{authzHosts}} list is pretty straight forward, > can use the ensemble server details in zoo.cfg file. But with dynamic > reconfig, it has to consider the dynamic add/remove/update servers and need > to discuss the ways to handle dynamic cases. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2793) [QP MutualAuth]: Build a mechanism to build "authzHosts" for dynamic reconfig servers
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208025#comment-16208025 ] Rakesh R commented on ZOOKEEPER-2793: - Below is the proposal to add the {{authorized_hosts}} information to the ZK ensemble. # Introduce reserved path : {{/zookeeper/authorized_hosts}}, which will store the host details like {{"host1,host2,host3"}}. Before invoking the #reconfig call the authorized_hosts has to be updated with the newly joining hosts Validation logic will use these pre-authorized hosts and reject any host which doesn't exists in this list. # Admin can update the authorized_hosts via ZooKeeper.setData("/zookeeper/authorized_hosts", ...) // user can call existing set/get/delete client APIs. # Expose zkCli.sh commands for better user experience, - setAuthorizedHosts host1,host2 - listAuthorizedHosts - delAuthorizedHosts host1,host2 [~phunt], IIRC, the above idea is same as we discussed some time back. Please feel free to edit if I missed anything. Thanks! > [QP MutualAuth]: Build a mechanism to build "authzHosts" for dynamic reconfig > servers > - > > Key: ZOOKEEPER-2793 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2793 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security >Reporter: Rakesh R > Fix For: 3.5.4, 3.6.0 > > > {{QuorumServer}} will do the authorization checks against configured > authorized hosts. During LE, QuorumLearner will send an authentication packet > to QuorumServer. Now, QuorumServer will check that the connecting > QuorumLearner’s hostname exists in the authorized hosts. If not exists then > connecting peer is not authorized to join this ensemble and the request will > be rejected immediately. > In {{branch-3.4}} building {{authzHosts}} list is pretty straight forward, > can use the ensemble server details in zoo.cfg file. But with dynamic > reconfig, it has to consider the dynamic add/remove/update servers and need > to discuss the ways to handle dynamic cases. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2792) [QP MutualAuth]: Port ZOOKEEPER-1045 implementation from branch-3.4 to branch-3.5
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097106#comment-16097106 ] Rakesh R commented on ZOOKEEPER-2792: - bq. provide an upgrade path from 3.4 to 3.5, so what it aimed to test (the upgrade logic) is only applicable to branch-3.4 IIUC, server will only permit a quorum connection from another server, if the other server has a lower sid. Please refer [QuorumCnxManager.java#L354|https://github.com/apache/zookeeper/blob/branch-3.5/src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java#L354] logic. {{QuorumCnxManagerTest}} test class contains many test cases related to this. Also, we have other set of test cases verifying the auth connection behaviors based on the {{serverRequireSasl}} and {{learnerRequireSasl}} flags. I think, {{QuorumCnxManagerTest}} cases would be a good addition by covering these cases in {{3.5}} as well. Since the current merge patch is large, I agree to push this via separate su-task. > [QP MutualAuth]: Port ZOOKEEPER-1045 implementation from branch-3.4 to > branch-3.5 > - > > Key: ZOOKEEPER-2792 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2792 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security >Reporter: Rakesh R >Assignee: Michael Han > Fix For: 3.5.4, 3.6.0 > > Attachments: ZOOKEEPER-2792.patch > > > This jira is to merge the basic working patch covering the authentication and > authorization of static(zoo.cfg) ZooKeeper servers from {{branch-3.4}} code > base. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2792) [QP MutualAuth]: Port ZOOKEEPER-1045 implementation from branch-3.4 to branch-3.5
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096417#comment-16096417 ] Rakesh R commented on ZOOKEEPER-2792: - Thanks [~hanm] for the efforts. I ran unit tests cases in my env and all are passing. Mostly changes looks good to me and I've few observations. Following changes needs to be included. I'm OK to merge these via separate sub-tasks. # Any specific reason to not merging {{QuorumCnxManagerTest}} test cases? # Need to merge below changes: https://issues.apache.org/jira/browse/ZOOKEEPER-2650 https://issues.apache.org/jira/browse/ZOOKEEPER-2759 > [QP MutualAuth]: Port ZOOKEEPER-1045 implementation from branch-3.4 to > branch-3.5 > - > > Key: ZOOKEEPER-2792 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2792 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security >Reporter: Rakesh R >Assignee: Michael Han > Fix For: 3.5.4, 3.6.0 > > Attachments: ZOOKEEPER-2792.patch > > > This jira is to merge the basic working patch covering the authentication and > authorization of static(zoo.cfg) ZooKeeper servers from {{branch-3.4}} code > base. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2755) Allow to subclass ClientCnxnSocketNetty and NettyServerCnxn in order to use Netty Local transport
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079676#comment-16079676 ] Rakesh R commented on ZOOKEEPER-2755: - bq. Embedding ZooKeeper server + client on the same JVM has many risks [~eolivelli] Could you elaborate on the potential risks that you anticipate. > Allow to subclass ClientCnxnSocketNetty and NettyServerCnxn in order to use > Netty Local transport > - > > Key: ZOOKEEPER-2755 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2755 > Project: ZooKeeper > Issue Type: New Feature > Components: java client, server >Affects Versions: 3.5.2 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli > > ClientCnxnSocketNetty and NettyServerCnxn use explicitly InetSocketAddress > class to work with network addresses. > We can do a little refactoring to use only SocketAddress and make it possible > to create subclasses of ClientCnxnSocketNetty and NettyServerCnxn which > leverage built-in Netty 'local' channels. > Such Netty local channels do not create real sockets and so allow a simple > ZooKeeper server + ZooKeeper client to be run on the same JVM without binding > to real TCP endpoints. > Usecases: > Ability to run concurrently on the same machine tests of projects which use > ZooKeeper (usually in unit tests the server and the client run inside the > same JVM) without dealing with random ports and in general using less network > resources > Run simplified (standalone, all processes in the same JVM) versions of > applications which need a working ZooKeeper ensemble to run. > Note: > Embedding ZooKeeper server + client on the same JVM has many risks and in > general I think we should encourage users to do so, so I in this patch I will > not provide official implementations of ClientCnxnSocketNetty and > NettyServerCnxn. There will be implementations only inside the test packages, > in order to test that most of the features are working with custom socket > factories and in particular with the 'LocalAddress' specific subclass of > SocketAddress. > Note: > the 'Local' sockets feature will be available on Netty 4 too -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ZOOKEEPER-2818) Improve the ZooKeeper#setACL java doc
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2818: --- Assignee: Brahma Reddy Battula > Improve the ZooKeeper#setACL java doc > -- > > Key: ZOOKEEPER-2818 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2818 > Project: ZooKeeper > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Fix For: 3.5.4, 3.6.0, 3.4.11 > > > As per discussion in > [mailinglist|http://mail-archives.apache.org/mod_mbox/zookeeper-user/201706.mbox/browser],It's > better improve Java doc or argument which might not mislead for new users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2792) [QP MutualAuth]: Merge basic working patch from branch-3.4
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071492#comment-16071492 ] Rakesh R commented on ZOOKEEPER-2792: - [~hanm], there is one last NPE with the patch, please take care. I hope we will be good after fixing this - [QuorumKerberosHostBasedAuthTest_testConnectBadServer|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/820//testReport/org.apache.zookeeper.server.quorum.auth/QuorumKerberosHostBasedAuthTest/testConnectBadServer/] {code} 2017-06-22 01:10:25,774 [myid:1] - ERROR [QuorumConnectionThread-[myid=1]-1:NIOServerCnxnFactory$1@92] - Thread Thread[QuorumConnectionThread-[myid=1]-1,5,FailOnTimeoutGroup] died java.lang.NullPointerException at org.apache.zookeeper.util.SecurityUtils.replacePattern(SecurityUtils.java:296) {code} > [QP MutualAuth]: Merge basic working patch from branch-3.4 > -- > > Key: ZOOKEEPER-2792 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2792 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security > Reporter: Rakesh R >Assignee: Michael Han > Fix For: 3.5.4, 3.6.0 > > Attachments: ZOOKEEPER-2792.patch > > > This jira is to merge the basic working patch covering the authentication and > authorization of static(zoo.cfg) ZooKeeper servers from {{branch-3.4}} code > base. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ZOOKEEPER-2728) Clean up findbug warnings in branch-3.4
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2728. - Resolution: Fixed Findbugs in {{branch-3.4}} has been cleaned up completely. I'm closing this jira. Thanks everyone for the good work! > Clean up findbug warnings in branch-3.4 > --- > > Key: ZOOKEEPER-2728 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2728 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.9 > Reporter: Rakesh R > Fix For: 3.4.11 > > Attachments: Branch-3.4 FindBugs Report.html, > zookeeper-findbugs-warnings-present-in-3.4.9.html > > > This jira to cleanup findbug warnings reported in branch-3.4 > [Branch3.4 > FindbugsWarnings.html|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/build/test/findbugs/newPatchFindbugsWarnings.html] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ZOOKEEPER-1976) address internationalization issues identified by findbugs 2.0.3
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-1976: Affects Version/s: 3.4.11 > address internationalization issues identified by findbugs 2.0.3 > > > Key: ZOOKEEPER-1976 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1976 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.5.0, 3.4.11 >Reporter: Patrick Hunt > > Findbugs 2.0.3 found a number of internationalization issues with the code, > we ignored these for the time being in ZOOKEEPER-1975, however we should > address them one by one eventually. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ZOOKEEPER-2730) Cleanup findbug warnings in branch-3.4: Disable Internationalization Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2730: --- Assignee: Janek P > Cleanup findbug warnings in branch-3.4: Disable Internationalization Warnings > - > > Key: ZOOKEEPER-2730 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2730 > Project: ZooKeeper > Issue Type: Sub-task > Reporter: Rakesh R >Assignee: Janek P > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > DmFound reliance on default encoding in > org.apache.jute.compiler.CGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.CppGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genCsharpCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genJavaCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in new > org.apache.jute.XmlOutputArchive(OutputStream): new > java.io.PrintStream(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): new java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): new > java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.generateDigest(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.handleAuthentication(ServerCnxn, > byte[]): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateQuotaForPath(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NettyServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileSnap. FileSnap>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileTxnLog. FileTxnLog>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.readLongFromFile(String): new > java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(String, long): > new java.io.OutputStreamWriter(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(Properties): > new java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.Request.toString(): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.ServerCnxn.(): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.TraceFormatter.main(String[]): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.util.OSMXBean.getMaxFileDescriptorCount(): new > java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apa
[jira] [Resolved] (ZOOKEEPER-2730) Cleanup findbug warnings in branch-3.4: Disable Internationalization Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2730. - Resolution: Fixed Issue resolved by pull request 285 [https://github.com/apache/zookeeper/pull/285] > Cleanup findbug warnings in branch-3.4: Disable Internationalization Warnings > - > > Key: ZOOKEEPER-2730 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2730 > Project: ZooKeeper > Issue Type: Sub-task > Reporter: Rakesh R > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > DmFound reliance on default encoding in > org.apache.jute.compiler.CGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.CppGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genCsharpCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genJavaCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in new > org.apache.jute.XmlOutputArchive(OutputStream): new > java.io.PrintStream(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): new java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): new > java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.generateDigest(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.handleAuthentication(ServerCnxn, > byte[]): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateQuotaForPath(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NettyServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileSnap. FileSnap>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileTxnLog. FileTxnLog>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.readLongFromFile(String): new > java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(String, long): > new java.io.OutputStreamWriter(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(Properties): > new java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.Request.toString(): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.ServerCnxn.(): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.TraceFormatter.main(String[]): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.util.OSMXBean.getMaxFileDescriptorCount(): new > java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding
[jira] [Updated] (ZOOKEEPER-2730) Cleanup findbug warnings in branch-3.4: Disable Internationalization Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2730: Summary: Cleanup findbug warnings in branch-3.4: Disable Internationalization Warnings (was: Cleanup findbug warnings in branch-3.4: Internationalization Warnings) > Cleanup findbug warnings in branch-3.4: Disable Internationalization Warnings > - > > Key: ZOOKEEPER-2730 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2730 > Project: ZooKeeper > Issue Type: Sub-task > Reporter: Rakesh R > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > DmFound reliance on default encoding in > org.apache.jute.compiler.CGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.CppGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genCsharpCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genJavaCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in new > org.apache.jute.XmlOutputArchive(OutputStream): new > java.io.PrintStream(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): new java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): new > java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.generateDigest(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.handleAuthentication(ServerCnxn, > byte[]): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateQuotaForPath(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NettyServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileSnap. FileSnap>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileTxnLog. FileTxnLog>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.readLongFromFile(String): new > java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(String, long): > new java.io.OutputStreamWriter(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(Properties): > new java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.Request.toString(): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.ServerCnxn.(): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.TraceFormatter.main(String[]): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.util.OSMXBean.getMaxFileDescrip
[jira] [Assigned] (ZOOKEEPER-2808) ACL with index 1 might be removed if it's only being used once
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2808: --- Assignee: Fangmin Lv > ACL with index 1 might be removed if it's only being used once > -- > > Key: ZOOKEEPER-2808 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2808 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.6.0 >Reporter: Fangmin Lv >Assignee: Fangmin Lv >Priority: Critical > Labels: upgrade > Fix For: 3.5.4, 3.6.0 > > > When Zeus start up, it will create DataTree instance, in which the empty > config znode is created with READ_UNSAFE acl, the acl will be stored in a map > with index 1. Then it's going to load the snapshot from disk, the nodes and > acl map will be cleared, but the reconfig znode is still reference to acl > index 1. The reconfig znode will be reused, so actually it may reference to a > different ACL stored in the snasphot. After leader-follower syncing, the > reconfig znode will be added back again (if it doesn't exist), which will > remove the previous reference to ACL index 1, if the index 1 has 0 reference > it will be removed from the ACL map, which could cause that ACL un-usable, > and that znode will not be readable. > Error logs related: > - > 2017-06-12 12:02:21,443 [myid:2] - ERROR > [CommitProcWorkThread-14:DataTree@249] - ERROR: ACL not available for long 1 > 2017-06-12 12:02:21,444 [myid:2] - ERROR > [CommitProcWorkThread-14:FinalRequestProcessor@567] - Failed to process > sessionid:0x201035cc882002d type:getChildren cxid:0x1 zxid:0xfffe > txntype:unknown reqpath:n/a > java.lang.RuntimeException: Failed to fetch acls for 1 > at org.apache.zookeeper.server.DataTree.convertLong(DataTree.java:250) > at org.apache.zookeeper.server.DataTree.getACL(DataTree.java:799) > at org.apache.zookeeper.server.ZKDatabase.getACL(ZKDatabase.java:574) > at > org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:463) > at > org.apache.zookeeper.server.quorum.CommitProcessor$CommitWorkRequest.doWork(CommitProcessor.java:439) > at > org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:151) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2792) [QP MutualAuth]: Merge basic working patch from branch-3.4
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052658#comment-16052658 ] Rakesh R commented on ZOOKEEPER-2792: - [~hanm], Its much better now and close to get a good build. {{QuorumKerberosHostBasedAuthTest}} hits NPE, please take care this as well. {code} 2017-06-16 18:01:48,411 [myid:1] - ERROR [QuorumConnectionThread-[myid=1]-1:NIOServerCnxnFactory$1@92] - Thread Thread[QuorumConnectionThread-[myid=1]-1,5,FailOnTimeoutGroup] died java.lang.NullPointerException at org.apache.zookeeper.util.SecurityUtils.replacePattern(SecurityUtils.java:296) at org.apache.zookeeper.util.SecurityUtils.getServerPrincipal(SecurityUtils.java:285) {code} > [QP MutualAuth]: Merge basic working patch from branch-3.4 > -- > > Key: ZOOKEEPER-2792 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2792 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security > Reporter: Rakesh R >Assignee: Michael Han > Fix For: 3.5.4, 3.6.0 > > Attachments: ZOOKEEPER-2792.patch > > > This jira is to merge the basic working patch covering the authentication and > authorization of static(zoo.cfg) ZooKeeper servers from {{branch-3.4}} code > base. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2792) [QP MutualAuth]: Merge basic working patch from branch-3.4
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050273#comment-16050273 ] Rakesh R commented on ZOOKEEPER-2792: - Thanks [~hanm] for the efforts. bq. Currently these tests are failing. >From the logs, I could see NPE and hope will get a clean test report once we >fix this. Please take care. Thanks! # {{QuorumKerberosHostBasedAuthTes#testConnectBadServer}} {code} 2017-06-01 22:57:22,656 [myid:0] - ERROR [QuorumConnectionThread-[myid=0]-2:NIOServerCnxnFactory$1@92] - Thread Thread[QuorumConnectionThread-[myid=0]-2,5,FailOnTimeoutGroup] died java.lang.NullPointerException at org.apache.zookeeper.util.SecurityUtils.replacePattern(SecurityUtils.java:296) {code} # {{ReconfigRecoveryTest#testCurrentObserverIsParticipantInNewConfig}} {code} java.lang.NullPointerException at org.apache.zookeeper.server.quorum.QuorumCnxManager.startConnection(QuorumCnxManager.java:413) at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:326) at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:637) {code} # {{ReconfigTest#testRemoveAddOne}} {code} Exception in thread "WorkerSender[myid=2]" java.lang.NullPointerException at org.apache.zookeeper.server.quorum.QuorumCnxManager.startConnection(QuorumCnxManager.java:413) at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:326) {code} > [QP MutualAuth]: Merge basic working patch from branch-3.4 > -- > > Key: ZOOKEEPER-2792 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2792 > Project: ZooKeeper > Issue Type: Sub-task > Components: quorum, security >Reporter: Rakesh R >Assignee: Michael Han > Fix For: 3.5.4, 3.6.0 > > Attachments: ZOOKEEPER-2792.patch > > > This jira is to merge the basic working patch covering the authentication and > authorization of static(zoo.cfg) ZooKeeper servers from {{branch-3.4}} code > base. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ZOOKEEPER-2730) Cleanup findbug warnings in branch-3.4: Internationalization Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047837#comment-16047837 ] Rakesh R commented on ZOOKEEPER-2730: - Can we take this task ahead and looks like this is the only findbug pending task. > Cleanup findbug warnings in branch-3.4: Internationalization Warnings > - > > Key: ZOOKEEPER-2730 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2730 > Project: ZooKeeper > Issue Type: Sub-task > Reporter: Rakesh R > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > DmFound reliance on default encoding in > org.apache.jute.compiler.CGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.CppGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genCsharpCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genJavaCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in new > org.apache.jute.XmlOutputArchive(OutputStream): new > java.io.PrintStream(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): new java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): new > java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.generateDigest(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.handleAuthentication(ServerCnxn, > byte[]): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateQuotaForPath(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NettyServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileSnap. FileSnap>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileTxnLog. FileTxnLog>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.readLongFromFile(String): new > java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(String, long): > new java.io.OutputStreamWriter(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(Properties): > new java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.Request.toString(): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.ServerCnxn.(): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.TraceFormatter.main(String[]): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.util.OSMXBean.getMaxFileDescriptorCount(): new > java.io.InputStreamReader(InputStream) > DmFound reliance on default
[jira] [Commented] (ZOOKEEPER-2592) Zookeeper is not recoverable once running system( machine on which zookeeper is running) is out of space
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047365#comment-16047365 ] Rakesh R commented on ZOOKEEPER-2592: - bq. But this issue is there in standalone mode. [~rakeshsingh], mostly not an issue in standalone mode as ZK server registers [ZooKeeperServerShutdownHandler_L130|https://github.com/apache/zookeeper/blob/master/src/java/main/org/apache/zookeeper/server/ZooKeeperServerMain.java#L130] to watch server errors and take necessary actions similar to QP. Please let me know if you see any corner cases where it misses server error and not shutdown. > Zookeeper is not recoverable once running system( machine on which zookeeper > is running) is out of space > > > Key: ZOOKEEPER-2592 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2592 > Project: ZooKeeper > Issue Type: Bug > Components: server >Affects Versions: 3.5.1, 3.5.2 >Reporter: Rakesh Kumar Singh >Priority: Critical > > Zookeeper is not recoverable once running system( machine on which zookeeper > is running) is out of space > Steps to reproduce:- > 1. Install zookeeper on standalone mode and start zookeeper > 2. Make the machine physical memory full > 3. Connect through client to zookeeper and trying create some znodes with > some data. > 4. After sometime creating further znode will not happened as complete memory > is occupied > 5. Now start creating space in that machine > 6. Again connect through a client. Connection is fine. Now try to execute any > command like "ls / " it fails even though now space is more than 11gb > Client log:- > BLR107042:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin > # df -h > Filesystem Size Used Avail Use% Mounted on > /dev/xvda2 36G 24G 11G 70% / > udev1.9G 116K 1.9G 1% /dev > tmpfs 1.9G 0 1.9G 0% /dev/shm > BLR107042:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin > # ./zkCli.sh > Connecting to localhost:2181 > 2016-09-19 22:50:20,227 [myid:] - INFO [main:Environment@109] - Client > environment:zookeeper.version=3.5.1-alpha--1, built on 08/18/2016 08:20 GMT > 2016-09-19 22:50:20,231 [myid:] - INFO [main:Environment@109] - Client > environment:host.name=BLR107042 > 2016-09-19 22:50:20,231 [myid:] - INFO [main:Environment@109] - Client > environment:java.version=1.7.0_79 > 2016-09-19 22:50:20,234 [myid:] - INFO [main:Environment@109] - Client > environment:java.vendor=Oracle Corporation > 2016-09-19 22:50:20,234 [myid:] - INFO [main:Environment@109] - Client > environment:java.home=/usr/java/jdk1.7.0_79/jre > 2016-09-19 22:50:20,234 [myid:] - INFO [main:Environment@109] - Client > environment:java.class.path=/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../build/classes:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../build/lib/*.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/slf4j-log4j12-1.7.5.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/slf4j-api-1.7.5.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/servlet-api-2.5-20081211.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/netty-3.7.0.Final.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/log4j-1.2.16.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/jline-2.11.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/jetty-util-6.1.26.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/jetty-6.1.26.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/javacc.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/jackson-mapper-asl-1.9.11.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/jackson-core-asl-1.9.11.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/commons-cli-1.2.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../lib/ant-eclipse-1.0-jvm1.2.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../zookeeper-3.5.1-alpha.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../src/java/lib/ant-eclipse-1.0-jvm1.2.jar:/home/Rakesh/Zookeeper/18_Aug/cluster/zookeeper-3.5.1-alpha/bin/../conf:/usr/java/jdk1.7.0_79/lib > 2016-09-19 22:50:20,234 [myid:] - INFO [main:Environment@109] - Client > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > 2016-09-19
[jira] [Comment Edited] (ZOOKEEPER-2355) Ephemeral node is never deleted if follower fails while reading the proposal packet
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16044225#comment-16044225 ] Rakesh R edited comment on ZOOKEEPER-2355 at 6/9/17 9:44 AM: - [~fpj], I didn't see your comment, looks like we both commented at the same time:) Thanks for pitches in. Yes, the [PR_112|https://github.com/apache/zookeeper/pull/112] has to be rebased. [~arshad.mohammad], would you mind rebasing the pull req. was (Author: rakeshr): [~fpj], I didn't see your comment, looks like we both commented at the same time:) Thanks for pitching in. Yes, the [PR_112|https://github.com/apache/zookeeper/pull/112] has to be rebased. [~arshad.mohammad], would you mind rebasing the pull req. > Ephemeral node is never deleted if follower fails while reading the proposal > packet > --- > > Key: ZOOKEEPER-2355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2355 > Project: ZooKeeper > Issue Type: Bug > Components: quorum, server >Reporter: Mohammad Arshad >Assignee: Mohammad Arshad >Priority: Critical > Fix For: 3.5.4, 3.6.0 > > Attachments: ZOOKEEPER-2355-01.patch, ZOOKEEPER-2355-02.patch, > ZOOKEEPER-2355-03.patch, ZOOKEEPER-2355-04.patch, ZOOKEEPER-2355-05.patch > > > ZooKeeper ephemeral node is never deleted if follower fail while reading the > proposal packet > The scenario is as follows: > # Configure three node ZooKeeper cluster, lets say nodes are A, B and C, > start all, assume A is leader, B and C are follower > # Connect to any of the server and create ephemeral node /e1 > # Close the session, ephemeral node /e1 will go for deletion > # While receiving delete proposal make Follower B to fail with > {{SocketTimeoutException}}. This we need to do to reproduce the scenario > otherwise in production environment it happens because of network fault. > # Remove the fault, just check that faulted Follower is now connected with > quorum > # Connect to any of the server, create the same ephemeral node /e1, created > is success. > # Close the session, ephemeral node /e1 will go for deletion > # {color:red}/e1 is not deleted from the faulted Follower B, It should have > been deleted as it was again created with another session{color} > # {color:green}/e1 is deleted from Leader A and other Follower C{color} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2355) Ephemeral node is never deleted if follower fails while reading the proposal packet
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16044225#comment-16044225 ] Rakesh R commented on ZOOKEEPER-2355: - [~fpj], I didn't see your comment, looks like we both commented at the same time:) Thanks for pitching in. Yes, the [PR_112|https://github.com/apache/zookeeper/pull/112] has to be rebased. [~arshad.mohammad], would you mind rebasing the pull req. > Ephemeral node is never deleted if follower fails while reading the proposal > packet > --- > > Key: ZOOKEEPER-2355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2355 > Project: ZooKeeper > Issue Type: Bug > Components: quorum, server >Reporter: Mohammad Arshad >Assignee: Mohammad Arshad >Priority: Critical > Fix For: 3.5.4, 3.6.0 > > Attachments: ZOOKEEPER-2355-01.patch, ZOOKEEPER-2355-02.patch, > ZOOKEEPER-2355-03.patch, ZOOKEEPER-2355-04.patch, ZOOKEEPER-2355-05.patch > > > ZooKeeper ephemeral node is never deleted if follower fail while reading the > proposal packet > The scenario is as follows: > # Configure three node ZooKeeper cluster, lets say nodes are A, B and C, > start all, assume A is leader, B and C are follower > # Connect to any of the server and create ephemeral node /e1 > # Close the session, ephemeral node /e1 will go for deletion > # While receiving delete proposal make Follower B to fail with > {{SocketTimeoutException}}. This we need to do to reproduce the scenario > otherwise in production environment it happens because of network fault. > # Remove the fault, just check that faulted Follower is now connected with > quorum > # Connect to any of the server, create the same ephemeral node /e1, created > is success. > # Close the session, ephemeral node /e1 will go for deletion > # {color:red}/e1 is not deleted from the faulted Follower B, It should have > been deleted as it was again created with another session{color} > # {color:green}/e1 is deleted from Leader A and other Follower C{color} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2355) Ephemeral node is never deleted if follower fails while reading the proposal packet
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16044196#comment-16044196 ] Rakesh R commented on ZOOKEEPER-2355: - bq. Can this bug be fixed in 3.4.11??? Thanks for the interest [~fjcyue], [~jiangjiafu], will include this in 3.4.11, if there is no objection from anyone. Since the priority is critical, I'd like to see one more +1 from a committer. Hi [~hanm], do you have some time to review this pull request and give your vote. Thanks! > Ephemeral node is never deleted if follower fails while reading the proposal > packet > --- > > Key: ZOOKEEPER-2355 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2355 > Project: ZooKeeper > Issue Type: Bug > Components: quorum, server >Reporter: Mohammad Arshad >Assignee: Mohammad Arshad >Priority: Critical > Fix For: 3.5.4, 3.6.0 > > Attachments: ZOOKEEPER-2355-01.patch, ZOOKEEPER-2355-02.patch, > ZOOKEEPER-2355-03.patch, ZOOKEEPER-2355-04.patch, ZOOKEEPER-2355-05.patch > > > ZooKeeper ephemeral node is never deleted if follower fail while reading the > proposal packet > The scenario is as follows: > # Configure three node ZooKeeper cluster, lets say nodes are A, B and C, > start all, assume A is leader, B and C are follower > # Connect to any of the server and create ephemeral node /e1 > # Close the session, ephemeral node /e1 will go for deletion > # While receiving delete proposal make Follower B to fail with > {{SocketTimeoutException}}. This we need to do to reproduce the scenario > otherwise in production environment it happens because of network fault. > # Remove the fault, just check that faulted Follower is now connected with > quorum > # Connect to any of the server, create the same ephemeral node /e1, created > is success. > # Close the session, ephemeral node /e1 will go for deletion > # {color:red}/e1 is not deleted from the faulted Follower B, It should have > been deleted as it was again created with another session{color} > # {color:green}/e1 is deleted from Leader A and other Follower C{color} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2800) zookeeper ephemeral node not deleted after server restart and consistency is not hold
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16044049#comment-16044049 ] Rakesh R commented on ZOOKEEPER-2800: - bq. I think this must be a bug, because the PR happens again in my environment. bq. do lost connection to the leader while wring data, and then many transcations are lost too(including the closeSession transcation). >From your comment, I think this is reproducible in your env. One easy way is >to apply 2355 patch and quickly re-test to understand the impact of 2355 fix. > zookeeper ephemeral node not deleted after server restart and consistency is > not hold > - > > Key: ZOOKEEPER-2800 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2800 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.4.11 > Environment: Centos6.5 java8 >Reporter: JiangJiafu >Priority: Critical > Attachments: zoo.cfg, zookeeper2.out, zookeeper3.out, zookeeper.out > > > I deploy a cluster of ZooKeeper with three nodes: > ofs_zk1:30.0.0.72 > ofs_zk2:30.0.0.73 > ofs_zk3:30.0.0.99 > On 2017-06-02, use the c zk client to create some ephemeral sequential nodes,: > /adm_election/rolemgr/rolemgr08, > /adm_election/rolemgr/rolemgr11, > /adm_election/rolemgr/rolemgr12, > with sesstion timeout 2 ms. > Then I restart ofs_zk1 and ofs_zk2. > On 2017-06-05, I found that, these ephemeral nodes still exist on ofs_zk1. > I can check the nodes by zkCli.sh get command on ofs_zk1. > But these nodes doesn't not exist on ofs_zk2 and ofs_zk3. > Is it odd? > I have upload the whole deploy directory of three nodes to: > https://pan.baidu.com/s/1miohiCo , > The log is printed in log/zookeeper.out > log of ofs_zk3 is too large, so I only show the head 1000 lines. > Since I find this PR a little late, some snapshot and log may be deleted. > I hope anyone can help find the reason. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2800) zookeeper ephemeral node not deleted after server restart and consistency is not hold
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16044003#comment-16044003 ] Rakesh R commented on ZOOKEEPER-2800: - I could see ZOOKEEPER-2355 jira talking about similar case of inconsistency in the Follower server. Could you please take a look at this issue analysis. Perhaps [~arshad.mohammad] can share more details on that issue faced in his env. > zookeeper ephemeral node not deleted after server restart and consistency is > not hold > - > > Key: ZOOKEEPER-2800 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2800 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.4.11 > Environment: Centos6.5 java8 >Reporter: JiangJiafu >Priority: Critical > Attachments: zoo.cfg, zookeeper2.out, zookeeper3.out, zookeeper.out > > > I deploy a cluster of ZooKeeper with three nodes: > ofs_zk1:30.0.0.72 > ofs_zk2:30.0.0.73 > ofs_zk3:30.0.0.99 > On 2017-06-02, use the c zk client to create some ephemeral sequential nodes,: > /adm_election/rolemgr/rolemgr08, > /adm_election/rolemgr/rolemgr11, > /adm_election/rolemgr/rolemgr12, > with sesstion timeout 2 ms. > Then I restart ofs_zk1 and ofs_zk2. > On 2017-06-05, I found that, these ephemeral nodes still exist on ofs_zk1. > I can check the nodes by zkCli.sh get command on ofs_zk1. > But these nodes doesn't not exist on ofs_zk2 and ofs_zk3. > Is it odd? > I have upload the whole deploy directory of three nodes to: > https://pan.baidu.com/s/1miohiCo , > The log is printed in log/zookeeper.out > log of ofs_zk3 is too large, so I only show the head 1000 lines. > Since I find this PR a little late, some snapshot and log may be deleted. > I hope anyone can help find the reason. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (ZOOKEEPER-1748) TCP keepalive for leader election connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-1748: --- Assignee: Ben Sherman (was: Daniel Peon) > TCP keepalive for leader election connections > - > > Key: ZOOKEEPER-1748 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1748 > Project: ZooKeeper > Issue Type: Improvement > Components: leaderElection >Affects Versions: 3.4.5, 3.5.0 > Environment: Linux, Java 1.7 >Reporter: Antal Sasvári >Assignee: Ben Sherman >Priority: Minor > Fix For: 3.4.11 > > Attachments: Zookeeper-1748-add_tcp_keepalive.patch > > > In our system we encountered the following problem: > If the system is stable, and there is no leader election, the leader election > port connections are open for very long time without any packets being sent > on them. > Some network elements silently drop the established TCP connection after a > timeout if there are no packets being sent on it. In this case the ZK servers > will not notice the connection loss. This causes additional delay later when > the next leader election is started, as the TCP connections are not alive any > more. > We would like to be able to enable TCP keepalive on the leader election > sockets in order to prevent the connection timeout in some network elements > due to connection inactivity. > This could be controlled by adding a new config parameter called tcpKeepAlive > in the ZooKeeper configuration file. It would be only applicable in case of > algorithm 3 (TCP based fast leader election), having the default value false. > If tcpKeepAlive is set to true, the TCP keepalive flag should be enabled for > the leader election sockets in QuorumCnxManager.setSockOpts() by calling > sock.setKeepAlive(true). > We have tested this change successfully in our environment. > Please comment whether you see any problem with this. If not, I am going to > submit a patch. > I've been told that e.g. Apache ActiveMQ also has a config option for similar > purpose called transport.keepalive. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (ZOOKEEPER-2797) Invalid TTL from misbehaving client nukes zookeeper
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2797: --- Assignee: Patrick White > Invalid TTL from misbehaving client nukes zookeeper > --- > > Key: ZOOKEEPER-2797 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2797 > Project: ZooKeeper > Issue Type: Bug > Components: security, server >Affects Versions: 3.6.0 >Reporter: Patrick White >Assignee: Patrick White > Fix For: 3.5.4, 3.6.0 > > > I was adding container and TTL support to kazoo, and managed to screw > something up which set the TTL to a negative value. This invalid TTL blew up > the commit processor, and got written to the log, preventing the zookeepers > from starting back up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2730) Cleanup findbug warnings in branch-3.4: Internationalization Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028524#comment-16028524 ] Rakesh R commented on ZOOKEEPER-2730: - bq. 1) my idea is to define everywhere the encoding to be 'UTF-8'. This was discussed quite some time back in [ZOOKEEPER-1976|https://issues.apache.org/jira/browse/ZOOKEEPER-1976?focusedCommentId=14068986=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14068986]. How about disabling this now and make the discussion alive in ZOOKEEPER-1976 to find a good solution applicable to all the branches. bq. 2) which version of FindBugs are you using? FindBugs version: 3.0.1. Please refer the [latest findbug report|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/746/artifact/build/test/findbugs/newPatchFindbugsWarnings.html] in our build. bq. And one more question - to which branch should this PR be made? Thanks in advance. Please bear with me, that is my first Zookeeper contribution. Please create a PR using github repo [branch-3.4|https://github.com/apache/zookeeper/tree/branch-3.4]. > Cleanup findbug warnings in branch-3.4: Internationalization Warnings > - > > Key: ZOOKEEPER-2730 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2730 > Project: ZooKeeper > Issue Type: Sub-task > Reporter: Rakesh R > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > DmFound reliance on default encoding in > org.apache.jute.compiler.CGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.CppGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genCsharpCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genJavaCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in new > org.apache.jute.XmlOutputArchive(OutputStream): new > java.io.PrintStream(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): new java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): new > java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.generateDigest(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.handleAuthentication(ServerCnxn, > byte[]): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateQuotaForPath(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NettyServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileSnap. FileSnap>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileTxnLog. FileTxnLog>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.readLongFromFile(String): new > java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quoru
[jira] [Updated] (ZOOKEEPER-2639) Port Quorum Peer mutual authentication SASL feature to branch-3.5 and trunk
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2639: Summary: Port Quorum Peer mutual authentication SASL feature to branch-3.5 and trunk (was: Port Quorum Peer mutual authentication feature to branch-3.5 and trunk) > Port Quorum Peer mutual authentication SASL feature to branch-3.5 and trunk > --- > > Key: ZOOKEEPER-2639 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2639 > Project: ZooKeeper > Issue Type: Task > Components: quorum, security > Reporter: Rakesh R >Assignee: Rakesh R >Priority: Critical > Fix For: 3.5.4, 3.6.0 > > > ZooKeeper server-server mutual authentication is implemented in > {{branch-3.4}} using ZOOKEEPER-1045 jira. The feature code is not directly > portable to other branches due to code difference. This jira can be used to > "forward" port the code changes to {{branch-3.5}} and {{trunk}}. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ZOOKEEPER-2794) [QP MutualAuth]: Revisit auth logic to handle dynamically add/remove servers
Rakesh R created ZOOKEEPER-2794: --- Summary: [QP MutualAuth]: Revisit auth logic to handle dynamically add/remove servers Key: ZOOKEEPER-2794 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2794 Project: ZooKeeper Issue Type: Sub-task Reporter: Rakesh R This jira is to revisit the basic authn/authz logic to handle the dynamically joining server to an ensemble. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ZOOKEEPER-2793) [QP MutualAuth]: Build a mechanism to build "authzHosts" for dynamic reconfig servers
Rakesh R created ZOOKEEPER-2793: --- Summary: [QP MutualAuth]: Build a mechanism to build "authzHosts" for dynamic reconfig servers Key: ZOOKEEPER-2793 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2793 Project: ZooKeeper Issue Type: Sub-task Reporter: Rakesh R {{QuorumServer}} will do the authorization checks against configured authorized hosts. During LE, QuorumLearner will send an authentication packet to QuorumServer. Now, QuorumServer will check that the connecting QuorumLearner’s hostname exists in the authorized hosts. If not exists then connecting peer is not authorized to join this ensemble and the request will be rejected immediately. In {{branch-3.4}} building {{authzHosts}} list is pretty straight forward, can use the ensemble server details in zoo.cfg file. But with dynamic reconfig, it has to consider the dynamic add/remove/update servers and need to discuss the ways to handle dynamic cases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ZOOKEEPER-2792) [QP MutualAuth]: Merge basic working patch from branch-3.4
Rakesh R created ZOOKEEPER-2792: --- Summary: [QP MutualAuth]: Merge basic working patch from branch-3.4 Key: ZOOKEEPER-2792 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2792 Project: ZooKeeper Issue Type: Sub-task Reporter: Rakesh R Assignee: Michael Han This jira is to merge the basic working patch covering the authentication and authorization of static(zoo.cfg) ZooKeeper servers from {{branch-3.4}} code base. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2639) Port Quorum Peer mutual authentication feature to branch-3.5 and trunk
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2639: Fix Version/s: 3.5.4 > Port Quorum Peer mutual authentication feature to branch-3.5 and trunk > -- > > Key: ZOOKEEPER-2639 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2639 > Project: ZooKeeper > Issue Type: Task > Components: quorum, security > Reporter: Rakesh R >Assignee: Rakesh R >Priority: Critical > Fix For: 3.5.4, 3.6.0 > > > ZooKeeper server-server mutual authentication is implemented in > {{branch-3.4}} using ZOOKEEPER-1045 jira. The feature code is not directly > portable to other branches due to code difference. This jira can be used to > "forward" port the code changes to {{branch-3.5}} and {{trunk}}. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ZOOKEEPER-2762) Multithreaded correctness Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2762. - Resolution: Fixed Issue resolved by pull request 239 [https://github.com/apache/zookeeper/pull/239] > Multithreaded correctness Warnings > -- > > Key: ZOOKEEPER-2762 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2762 > Project: ZooKeeper > Issue Type: Sub-task >Reporter: Abraham Fine >Assignee: Abraham Fine > Fix For: 3.4.11 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ZOOKEEPER-2731) Cleanup findbug warnings in branch-3.4: Malicious code vulnerability Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2731. - Resolution: Fixed Issue resolved by pull request 232 [https://github.com/apache/zookeeper/pull/232] > Cleanup findbug warnings in branch-3.4: Malicious code vulnerability Warnings > - > > Key: ZOOKEEPER-2731 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2731 > Project: ZooKeeper > Issue Type: Sub-task >Affects Versions: 3.4.9 > Reporter: Rakesh R >Assignee: Abraham Fine > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > MSorg.apache.zookeeper.Environment.JAAS_CONF_KEY isn't final but should be > Bug type MS_SHOULD_BE_FINAL (click for details) > In class org.apache.zookeeper.Environment > Field org.apache.zookeeper.Environment.JAAS_CONF_KEY > At Environment.java:[line 34] > MSorg.apache.zookeeper.server.ServerCnxn.cmd2String is a mutable > collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class org.apache.zookeeper.server.ServerCnxn > Field org.apache.zookeeper.server.ServerCnxn.cmd2String > At ServerCnxn.java:[line 230] > MSorg.apache.zookeeper.ZooDefs$Ids.OPEN_ACL_UNSAFE is a mutable collection > Bug type MS_MUTABLE_COLLECTION (click for details) > In class org.apache.zookeeper.ZooDefs$Ids > Field org.apache.zookeeper.ZooDefs$Ids.OPEN_ACL_UNSAFE > At ZooDefs.java:[line 100] > MSorg.apache.zookeeper.ZooKeeperMain.commandMap is a mutable collection > which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class org.apache.zookeeper.ZooKeeperMain > Field org.apache.zookeeper.ZooKeeperMain.commandMap > At ZooKeeperMain.java:[line 53] > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2730) Cleanup findbug warnings in branch-3.4: Internationalization Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023060#comment-16023060 ] Rakesh R commented on ZOOKEEPER-2730: - [~gosubpl], I'd appreciate if you could post a pull request fixing the warnings. Thanks! Probably, you can refer ZK cwiki section - [Contribute Work via Github Pull Request|https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute] > Cleanup findbug warnings in branch-3.4: Internationalization Warnings > - > > Key: ZOOKEEPER-2730 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2730 > Project: ZooKeeper > Issue Type: Sub-task > Reporter: Rakesh R > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > DmFound reliance on default encoding in > org.apache.jute.compiler.CGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.CppGenerator.genCode(): new java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genCsharpCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in > org.apache.jute.compiler.JRecord.genJavaCode(File): new > java.io.FileWriter(File) > DmFound reliance on default encoding in new > org.apache.jute.XmlOutputArchive(OutputStream): new > java.io.PrintStream(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): new java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, > String, int): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): new > java.io.InputStreamReader(InputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.generateDigest(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.auth.DigestAuthenticationProvider.handleAuthentication(ServerCnxn, > byte[]): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateBytes(String, long): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): new > String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateCount(String, int): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.DataTree.updateQuotaForPath(String): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NettyServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(boolean): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileSnap. FileSnap>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.persistence.FileTxnLog. FileTxnLog>(): String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.readLongFromFile(String): new > java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(String, long): > new java.io.OutputStreamWriter(OutputStream) > DmFound reliance on default encoding in > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(Properties): > new java.io.FileReader(File) > DmFound reliance on default encoding in > org.apache.zookeeper.server.Request.toString(): new String(byte[]) > DmFound reliance on default encoding in > org.apache.zookeeper.server.ServerCnxn.(): > String.getBytes() > DmFound reliance on default encoding in > org.apache.zookeeper.server.TraceFormatter.main(String[]): new String(byte[]) > DmFound reliance on
[jira] [Updated] (ZOOKEEPER-2749) Cleanup findbug warnings in branch-3.4: Experimental Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2749: Affects Version/s: 3.4.10 > Cleanup findbug warnings in branch-3.4: Experimental Warnings > - > > Key: ZOOKEEPER-2749 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2749 > Project: ZooKeeper > Issue Type: Sub-task >Affects Versions: 3.4.10 >Reporter: Abraham Fine >Assignee: Abraham Fine > Fix For: 3.4.11 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2733) Cleanup findbug warnings in branch-3.4: Dodgy code Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2733: Affects Version/s: 3.4.10 > Cleanup findbug warnings in branch-3.4: Dodgy code Warnings > --- > > Key: ZOOKEEPER-2733 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2733 > Project: ZooKeeper > Issue Type: Sub-task >Affects Versions: 3.4.10 > Reporter: Rakesh R >Assignee: Abraham Fine > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > DB > org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner.send(DataOutputStream, > byte[]) uses the same code for two branches > DLS Dead store to txn in > org.apache.zookeeper.server.quorum.LearnerHandler.packetToString(QuorumPacket) > NPLoad of known null value in > org.apache.zookeeper.server.PrepRequestProcessor.pRequest(Request) > NPPossible null pointer dereference in > org.apache.zookeeper.server.PurgeTxnLog.purgeOlderSnapshots(FileTxnSnapLog, > File) due to return value of called method > NPPossible null pointer dereference in > org.apache.zookeeper.server.PurgeTxnLog.purgeOlderSnapshots(FileTxnSnapLog, > File) due to return value of called method > NPLoad of known null value in > org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner.send(DataOutputStream, > byte[]) > NPLoad of known null value in > org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer.send(DataOutputStream, > byte[], QuorumAuth$Status) > NPPossible null pointer dereference in > org.apache.zookeeper.server.upgrade.UpgradeMain.copyFiles(File, File, String) > due to return value of called method > RCN Redundant nullcheck of bytes, which is known to be non-null in > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next() > SFSwitch statement found in > org.apache.zookeeper.server.PrepRequestProcessor.pRequest(Request) where > default case is missing > SFSwitch statement found in > org.apache.zookeeper.server.PrepRequestProcessor.pRequest2Txn(int, long, > Request, Record, boolean) where default case is missing > SFSwitch statement found in > org.apache.zookeeper.server.quorum.AuthFastLeaderElection$Messenger$WorkerReceiver.run() > where default case is missing > SFSwitch statement found in > org.apache.zookeeper.server.quorum.AuthFastLeaderElection$Messenger$WorkerSender.process(AuthFastLeaderElection$ToSend) > where default case is missing > SFSwitch statement found in > org.apache.zookeeper.server.quorum.Follower.processPacket(QuorumPacket) where > default case is missing > SFSwitch statement found in > org.apache.zookeeper.server.quorum.Observer.processPacket(QuorumPacket) where > default case is missing > STWrite to static field > org.apache.zookeeper.server.SyncRequestProcessor.randRoll from instance > method org.apache.zookeeper.server.SyncRequestProcessor.run() > UrF Unread public/protected field: > org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.err > UrF Unread public/protected field: > org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.path > UrF Unread public/protected field: > org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.stat > UrF Unread public/protected field: > org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.type > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ZOOKEEPER-2733) Cleanup findbug warnings in branch-3.4: Dodgy code Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2733. - Resolution: Fixed > Cleanup findbug warnings in branch-3.4: Dodgy code Warnings > --- > > Key: ZOOKEEPER-2733 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2733 > Project: ZooKeeper > Issue Type: Sub-task > Reporter: Rakesh R >Assignee: Abraham Fine > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > DB > org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner.send(DataOutputStream, > byte[]) uses the same code for two branches > DLS Dead store to txn in > org.apache.zookeeper.server.quorum.LearnerHandler.packetToString(QuorumPacket) > NPLoad of known null value in > org.apache.zookeeper.server.PrepRequestProcessor.pRequest(Request) > NPPossible null pointer dereference in > org.apache.zookeeper.server.PurgeTxnLog.purgeOlderSnapshots(FileTxnSnapLog, > File) due to return value of called method > NPPossible null pointer dereference in > org.apache.zookeeper.server.PurgeTxnLog.purgeOlderSnapshots(FileTxnSnapLog, > File) due to return value of called method > NPLoad of known null value in > org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner.send(DataOutputStream, > byte[]) > NPLoad of known null value in > org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer.send(DataOutputStream, > byte[], QuorumAuth$Status) > NPPossible null pointer dereference in > org.apache.zookeeper.server.upgrade.UpgradeMain.copyFiles(File, File, String) > due to return value of called method > RCN Redundant nullcheck of bytes, which is known to be non-null in > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next() > SFSwitch statement found in > org.apache.zookeeper.server.PrepRequestProcessor.pRequest(Request) where > default case is missing > SFSwitch statement found in > org.apache.zookeeper.server.PrepRequestProcessor.pRequest2Txn(int, long, > Request, Record, boolean) where default case is missing > SFSwitch statement found in > org.apache.zookeeper.server.quorum.AuthFastLeaderElection$Messenger$WorkerReceiver.run() > where default case is missing > SFSwitch statement found in > org.apache.zookeeper.server.quorum.AuthFastLeaderElection$Messenger$WorkerSender.process(AuthFastLeaderElection$ToSend) > where default case is missing > SFSwitch statement found in > org.apache.zookeeper.server.quorum.Follower.processPacket(QuorumPacket) where > default case is missing > SFSwitch statement found in > org.apache.zookeeper.server.quorum.Observer.processPacket(QuorumPacket) where > default case is missing > STWrite to static field > org.apache.zookeeper.server.SyncRequestProcessor.randRoll from instance > method org.apache.zookeeper.server.SyncRequestProcessor.run() > UrF Unread public/protected field: > org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.err > UrF Unread public/protected field: > org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.path > UrF Unread public/protected field: > org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.stat > UrF Unread public/protected field: > org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.type > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ZOOKEEPER-2732) Cleanup findbug warnings in branch-3.4: Performance Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2732. - Resolution: Fixed Issue resolved by pull request 258 [https://github.com/apache/zookeeper/pull/258] > Cleanup findbug warnings in branch-3.4: Performance Warnings > > > Key: ZOOKEEPER-2732 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2732 > Project: ZooKeeper > Issue Type: Sub-task > Reporter: Rakesh R >Assignee: Abraham Fine > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > BxBoxing/unboxing to parse a primitive new > org.apache.zookeeper.server.quorum.QuorumCnxManager(long, Map, > QuorumAuthServer, QuorumAuthLearner, int, boolean, int, boolean) > Bxnew org.apache.zookeeper.server.quorum.QuorumCnxManager(long, Map, > QuorumAuthServer, QuorumAuthLearner, int, boolean, int, boolean) invokes > inefficient new Integer(String) constructor; use Integer.valueOf(String) > instead > Dm > org.apache.zookeeper.server.quorum.FastLeaderElection$Notification.toString() > invokes inefficient new String(String) constructor > WMI org.apache.zookeeper.server.DataTree.dumpEphemerals(PrintWriter) makes > inefficient use of keySet iterator instead of entrySet iterator > WMI > org.apache.zookeeper.server.quorum.flexible.QuorumHierarchical.computeGroupWeight() > makes inefficient use of keySet iterator instead of entrySet iterator > WMI > org.apache.zookeeper.server.quorum.flexible.QuorumHierarchical.containsQuorum(HashSet) > makes inefficient use of keySet iterator instead of entrySet iterator > WMI org.apache.zookeeper.ZooKeeperMain.usage() makes inefficient use of > keySet iterator instead of entrySet iterator > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (ZOOKEEPER-2665) Port QA github pull request build to branch 3.4 and 3.5
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2665: --- Assignee: Enrico Olivelli > Port QA github pull request build to branch 3.4 and 3.5 > --- > > Key: ZOOKEEPER-2665 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2665 > Project: ZooKeeper > Issue Type: Test > Components: build >Reporter: Flavio Junqueira >Assignee: Enrico Olivelli > Fix For: 3.4.10, 3.5.3 > > > We have QA build for pull requests against master but not against branches > 3.4 and 3.5. We need to port the necessary wiring to do it, it shouldn't be > difficult. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (ZOOKEEPER-2617) correct a few spelling typos
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2617: --- Assignee: tony mancill > correct a few spelling typos > > > Key: ZOOKEEPER-2617 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2617 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.9 >Reporter: tony mancill >Assignee: tony mancill >Priority: Trivial > Labels: newbie, patch > Fix For: 3.4.10, 3.5.3, 3.6.0 > > Attachments: JIRA-ZOOKEEPER-2617.patch > > > While working on the Debian packaging of ZooKeeper, some misspellings were > detected in the source that affect the documentation, logging, and program > output. > There is a PR against github containing the patch here: > https://github.com/apache/zookeeper/pull/87 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (ZOOKEEPER-2460) Remove javacc dependency from public Maven pom
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2460: --- Assignee: Enrico Olivelli > Remove javacc dependency from public Maven pom > -- > > Key: ZOOKEEPER-2460 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2460 > Project: ZooKeeper > Issue Type: Bug > Components: java client >Affects Versions: 3.5.2 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli >Priority: Critical > Fix For: 3.5.3, 3.6.0 > > > during the vote of 3.5.2-ALPHA RC 0 we found a Maven dependency to javacc in > published pom for zookeeper > {code} > > net.java.dev.javacc > javacc > 5.0compile > > {code} > this dependency appears not to be useful and should be removed > this was the tested pom: > https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.2-alpha/zookeeper-3.5.2-alpha.pom -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (ZOOKEEPER-2742) Few test cases of org.apache.zookeeper.ZooKeeperTest fails in Windows
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2742: --- Assignee: Abhishek Kumar > Few test cases of org.apache.zookeeper.ZooKeeperTest fails in Windows > - > > Key: ZOOKEEPER-2742 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2742 > Project: ZooKeeper > Issue Type: Test > Components: tests > Environment: Windows >Reporter: Abhishek Kumar >Assignee: Abhishek Kumar >Priority: Trivial > Fix For: 3.5.4, 3.6.0 > > Attachments: ZOOKEEPER-2742-01.patch > > > Following test cases fail in Windows environment: > 1. org.apache.zookeeper.ZooKeeperTest.testLsrRootCommand() > 2. org.apache.zookeeper.ZooKeeperTest.testLsrCommand() > It seems that failure is related to use of "\n" (System dependent new line > char)in org.apache.zookeeper.ZooKeeperTest.runCommandExpect(CliCommand, > List) > .. > .. > String result = byteStream.toString(); > assertTrue(result, result.contains( > StringUtils.joinStrings(expectedResults, "\n"))); > .. > .. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2731) Cleanup findbug warnings in branch-3.4: Malicious code vulnerability Warnings
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2731: Description: Please refer the attached sheet in parent jira. Below is the details of findbug warnings. {code} MS org.apache.zookeeper.Environment.JAAS_CONF_KEY isn't final but should be Bug type MS_SHOULD_BE_FINAL (click for details) In class org.apache.zookeeper.Environment Field org.apache.zookeeper.Environment.JAAS_CONF_KEY At Environment.java:[line 34] MS org.apache.zookeeper.server.ServerCnxn.cmd2String is a mutable collection which should be package protected Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) In class org.apache.zookeeper.server.ServerCnxn Field org.apache.zookeeper.server.ServerCnxn.cmd2String At ServerCnxn.java:[line 230] MS org.apache.zookeeper.ZooDefs$Ids.OPEN_ACL_UNSAFE is a mutable collection Bug type MS_MUTABLE_COLLECTION (click for details) In class org.apache.zookeeper.ZooDefs$Ids Field org.apache.zookeeper.ZooDefs$Ids.OPEN_ACL_UNSAFE At ZooDefs.java:[line 100] MS org.apache.zookeeper.ZooKeeperMain.commandMap is a mutable collection which should be package protected Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) In class org.apache.zookeeper.ZooKeeperMain Field org.apache.zookeeper.ZooKeeperMain.commandMap At ZooKeeperMain.java:[line 53] {code} was: Please refer the attached sheet in parent jira. Below is the details of findbug warnings. {code} AT Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.zookeeper.server.quorum.QuorumCnxManager.handleConnection(Socket, DataInputStream) AT Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.zookeeper.server.quorum.QuorumCnxManager.startConnection(Socket, Long) AT Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(Long, ByteBuffer) VO Increment of volatile field org.apache.zookeeper.server.quorum.AuthFastLeaderElection.logicalclock in org.apache.zookeeper.server.quorum.AuthFastLeaderElection.leaveInstance() VO Increment of volatile field org.apache.zookeeper.server.quorum.AuthFastLeaderElection.logicalclock in org.apache.zookeeper.server.quorum.AuthFastLeaderElection.lookForLeader() VO Increment of volatile field org.apache.zookeeper.server.quorum.FastLeaderElection.logicalclock in org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader() VO Increment of volatile field org.apache.zookeeper.server.quorum.QuorumPeer.tick in org.apache.zookeeper.server.quorum.Leader.lead() VO Increment of volatile field org.apache.zookeeper.server.quorum.QuorumPeer.tick in org.apache.zookeeper.server.quorum.Leader.lead() {code} > Cleanup findbug warnings in branch-3.4: Malicious code vulnerability Warnings > - > > Key: ZOOKEEPER-2731 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2731 > Project: ZooKeeper > Issue Type: Sub-task >Affects Versions: 3.4.9 > Reporter: Rakesh R >Assignee: Abraham Fine > Fix For: 3.4.11 > > > Please refer the attached sheet in parent jira. Below is the details of > findbug warnings. > {code} > MSorg.apache.zookeeper.Environment.JAAS_CONF_KEY isn't final but should be > Bug type MS_SHOULD_BE_FINAL (click for details) > In class org.apache.zookeeper.Environment > Field org.apache.zookeeper.Environment.JAAS_CONF_KEY > At Environment.java:[line 34] > MSorg.apache.zookeeper.server.ServerCnxn.cmd2String is a mutable > collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class org.apache.zookeeper.server.ServerCnxn > Field org.apache.zookeeper.server.ServerCnxn.cmd2String > At ServerCnxn.java:[line 230] > MSorg.apache.zookeeper.ZooDefs$Ids.OPEN_ACL_UNSAFE is a mutable collection > Bug type MS_MUTABLE_COLLECTION (click for details) > In class org.apache.zookeeper.ZooDefs$Ids > Field org.apache.zookeeper.ZooDefs$Ids.OPEN_ACL_UNSAFE > At ZooDefs.java:[line 100] > MSorg.apache.zookeeper.ZooKeeperMain.commandMap is a mutable collection > which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class org.apache.zookeeper.ZooKeeperMain > Field org.apache.zookeeper.ZooKeeperMain.commandMap > At ZooKeeperMain.java:[line 53] > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (ZOOKEEPER-2756) Add CMake build system for better cross-platform support
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2756: --- Assignee: Andrew Schwartzmeyer > Add CMake build system for better cross-platform support > > > Key: ZOOKEEPER-2756 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2756 > Project: ZooKeeper > Issue Type: Improvement > Components: build, c client >Affects Versions: 3.5.2 > Environment: Windows and Linux >Reporter: Andrew Schwartzmeyer >Assignee: Andrew Schwartzmeyer > Labels: build, windows > Attachments: ZOOKEEPER-2756.patch > > > The C bindings primary build system is Autotools. This obviously does not > work for Windows, and so the original port to Windows simply added a Visual > Studio solution to the project, splitting the build system. As new versions > of Visual Studio have come along, new (probably auto-converted) solutions > have come along (see zookeeper.sln vs zookeeper-vs2013.sln). When Mesos > started being ported to Windows, a Visual Studio 2015 solution was needed, > and the previous developer created yet another solution, and setup Mesos' > build to patch ZooKeeper and add the 2015 solution. Now Visual Studio 2017 > was released, and in the process of moving Mesos ahead, I realized that I > would either have to make *yet another* converted solution for ZooKeeper. So > instead I tackled the root problem, and ported the Autotools build to CMake, > which is a meta-build system which generates files for the in-use platform > (whether it be Linux or Solaris or MacOS or Windows). > NOTE: I already have this patch, and will submit it. It has a couple TODOs, > and some other things in it that were necessary for Mesos that may need to be > pulled into separate patches. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2763) Utils.toCsvBuffer() omits leading 0 for bytes < 0x10
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15975129#comment-15975129 ] Rakesh R commented on ZOOKEEPER-2763: - bq. Can you add Alburt to JIRA contributor list? Done. [~weitang], please create a PR with the fix. All the best! > Utils.toCsvBuffer() omits leading 0 for bytes < 0x10 > > > Key: ZOOKEEPER-2763 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2763 > Project: ZooKeeper > Issue Type: Bug > Components: jute >Affects Versions: 3.5.2 >Reporter: Brandon Berg >Assignee: Alburt Hoffman >Priority: Minor > > org.apache.jute.Utils.toCsvBuffer(), which converts a byte array to a string > containing the hex representation of that byte array, omits the leading zero > for any byte less than 0x10, due to its use of Integer.toHexString, which has > the same behavior. > https://github.com/apache/zookeeper/blob/master/src/java/main/org/apache/jute/Utils.java#L234 > One consequence of this is that the hex strings printed by > ClientCnxn.Packet.toString(), used in the debug logging for > ClientCnxn.readResponse(), cannot be parsed to determine the result of a > Zookeeper request from client debug logs. > Utils.toXmlBuffer() appears to have the same issue. > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (ZOOKEEPER-2763) Utils.toCsvBuffer() omits leading 0 for bytes < 0x10
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2763: --- Assignee: Alburt Hoffman > Utils.toCsvBuffer() omits leading 0 for bytes < 0x10 > > > Key: ZOOKEEPER-2763 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2763 > Project: ZooKeeper > Issue Type: Bug > Components: jute >Affects Versions: 3.5.2 >Reporter: Brandon Berg >Assignee: Alburt Hoffman >Priority: Minor > > org.apache.jute.Utils.toCsvBuffer(), which converts a byte array to a string > containing the hex representation of that byte array, omits the leading zero > for any byte less than 0x10, due to its use of Integer.toHexString, which has > the same behavior. > https://github.com/apache/zookeeper/blob/master/src/java/main/org/apache/jute/Utils.java#L234 > One consequence of this is that the hex strings printed by > ClientCnxn.Packet.toString(), used in the debug logging for > ClientCnxn.readResponse(), cannot be parsed to determine the result of a > Zookeeper request from client debug logs. > Utils.toXmlBuffer() appears to have the same issue. > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2722) Flaky Test: org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2722: Fix Version/s: 3.4.11 > Flaky Test: > org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment > --- > > Key: ZOOKEEPER-2722 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2722 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Michael Han >Assignee: Michael Han > Labels: flaky > Fix For: 3.5.4, 3.6.0, 3.4.11 > > > {noformat} > Error Message > KeeperErrorCode = ConnectionLoss for /test > Stacktrace > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss for /test > at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1423) > at > org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment(ReadOnlyModeTest.java:238) > at > org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Looks like we should retry before giving up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ZOOKEEPER-2706) checkstyle broken on branch-3.4
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2706. - Resolution: Fixed Fix Version/s: 3.4.10 Issue resolved by pull request 181 [https://github.com/apache/zookeeper/pull/181] > checkstyle broken on branch-3.4 > --- > > Key: ZOOKEEPER-2706 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2706 > Project: ZooKeeper > Issue Type: Bug >Reporter: Abraham Fine >Assignee: Abraham Fine > Fix For: 3.4.10 > > > While working on ZOOKEEPER-2696, [~rakeshr] and I noticed that checkstyle is > failing to execute on branch-3.4 with the following error: > {code} > BUILD FAILED > /Users/abefine/cloudera_code/zookeeper/build.xml:1595: Unable to create a > Checker: cannot initialize module PackageHtml - Unable to instantiate > PackageHtml > {code} > This should essentially be a backport of ZOOKEEPER-412 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2728) Clean up findbug warnings in branch-3.4
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15935652#comment-15935652 ] Rakesh R commented on ZOOKEEPER-2728: - I'm adding some of the [findbugs related discussion from ZK-2712 jira|https://issues.apache.org/jira/browse/ZOOKEEPER-2712?focusedCommentId=15933929=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15933929] for reference. Hi [~hanm], how about moving this task to 3.4.11 release ? > Clean up findbug warnings in branch-3.4 > --- > > Key: ZOOKEEPER-2728 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2728 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.9 > Reporter: Rakesh R > Attachments: Branch-3.4 FindBugs Report.html, > zookeeper-findbugs-warnings-present-in-3.4.9.html > > > This jira to cleanup findbug warnings reported in branch-3.4 > [Branch3.4 > FindbugsWarnings.html|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/build/test/findbugs/newPatchFindbugsWarnings.html] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2712. - Resolution: Fixed Issue resolved by pull request 200 [https://github.com/apache/zookeeper/pull/200] > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Critical > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt, > zookeeper-findbugs-warnings-present-in-3.4.9.html > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (ZOOKEEPER-2728) Clean up findbug warnings in branch-3.4
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15934099#comment-15934099 ] Rakesh R edited comment on ZOOKEEPER-2728 at 3/21/17 5:04 AM: -- Attached sheet contains the findbugs to be cleaned up. I've created sub-tasks for each {{Warning Type}} under this parent jira, could use respective sub-tasks for cleaning up the same. was (Author: rakeshr): Attached sheet contains the findbugs to be cleaned up. > Clean up findbug warnings in branch-3.4 > --- > > Key: ZOOKEEPER-2728 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2728 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.9 > Reporter: Rakesh R > Attachments: Branch-3.4 FindBugs Report.html, > zookeeper-findbugs-warnings-present-in-3.4.9.html > > > This jira to cleanup findbug warnings reported in branch-3.4 > [Branch3.4 > FindbugsWarnings.html|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/build/test/findbugs/newPatchFindbugsWarnings.html] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2728) Clean up findbug warnings in branch-3.4
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2728: Attachment: Branch-3.4 FindBugs Report.html Attached sheet contains the findbugs to be cleaned up. > Clean up findbug warnings in branch-3.4 > --- > > Key: ZOOKEEPER-2728 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2728 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.9 > Reporter: Rakesh R > Attachments: Branch-3.4 FindBugs Report.html, > zookeeper-findbugs-warnings-present-in-3.4.9.html > > > This jira to cleanup findbug warnings reported in branch-3.4 > [Branch3.4 > FindbugsWarnings.html|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/build/test/findbugs/newPatchFindbugsWarnings.html] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2728) Clean up findbug warnings in branch-3.4
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2728: Attachment: zookeeper-findbugs-warnings-present-in-3.4.9.html Attached sheet contains the warnings present in previous 3.4.x release. Compare to the previous, we have 8 new {{Medium Priority Warnings}} - Correctness Warnings(2), Dodgy code Warnings(3), Malicious code vulnerability Warnings(3) > Clean up findbug warnings in branch-3.4 > --- > > Key: ZOOKEEPER-2728 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2728 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.9 > Reporter: Rakesh R > Attachments: zookeeper-findbugs-warnings-present-in-3.4.9.html > > > This jira to cleanup findbug warnings reported in branch-3.4 > [Branch3.4 > FindbugsWarnings.html|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/build/test/findbugs/newPatchFindbugsWarnings.html] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15934007#comment-15934007 ] Rakesh R commented on ZOOKEEPER-2712: - Thanks [~hanm], Attached sheet contains the warnings present in previous 3.4.x release. Compare to the previous, we have 8 new {{Medium Priority Warnings}} - Correctness Warnings(2), Dodgy code Warnings(3), Malicious code vulnerability Warnings(3) > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Critical > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt, > zookeeper-findbugs-warnings-present-in-3.4.9.html > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2712: Attachment: zookeeper-findbugs-warnings-present-in-3.4.9.html > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Critical > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt, > zookeeper-findbugs-warnings-present-in-3.4.9.html > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932383#comment-15932383 ] Rakesh R commented on ZOOKEEPER-2712: - FYI, I've raised issue ZOOKEEPER-2728 to cleanup {{br-3.4}} findbugs warnings. > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Critical > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ZOOKEEPER-2733) Cleanup findbug warnings in branch-3.4: Dodgy code Warnings
Rakesh R created ZOOKEEPER-2733: --- Summary: Cleanup findbug warnings in branch-3.4: Dodgy code Warnings Key: ZOOKEEPER-2733 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2733 Project: ZooKeeper Issue Type: Sub-task Reporter: Rakesh R Please refer the attached sheet in parent jira. Below is the details of findbug warnings. {code} DB org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner.send(DataOutputStream, byte[]) uses the same code for two branches DLS Dead store to txn in org.apache.zookeeper.server.quorum.LearnerHandler.packetToString(QuorumPacket) NP Load of known null value in org.apache.zookeeper.server.PrepRequestProcessor.pRequest(Request) NP Possible null pointer dereference in org.apache.zookeeper.server.PurgeTxnLog.purgeOlderSnapshots(FileTxnSnapLog, File) due to return value of called method NP Possible null pointer dereference in org.apache.zookeeper.server.PurgeTxnLog.purgeOlderSnapshots(FileTxnSnapLog, File) due to return value of called method NP Load of known null value in org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner.send(DataOutputStream, byte[]) NP Load of known null value in org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer.send(DataOutputStream, byte[], QuorumAuth$Status) NP Possible null pointer dereference in org.apache.zookeeper.server.upgrade.UpgradeMain.copyFiles(File, File, String) due to return value of called method RCN Redundant nullcheck of bytes, which is known to be non-null in org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next() SF Switch statement found in org.apache.zookeeper.server.PrepRequestProcessor.pRequest(Request) where default case is missing SF Switch statement found in org.apache.zookeeper.server.PrepRequestProcessor.pRequest2Txn(int, long, Request, Record, boolean) where default case is missing SF Switch statement found in org.apache.zookeeper.server.quorum.AuthFastLeaderElection$Messenger$WorkerReceiver.run() where default case is missing SF Switch statement found in org.apache.zookeeper.server.quorum.AuthFastLeaderElection$Messenger$WorkerSender.process(AuthFastLeaderElection$ToSend) where default case is missing SF Switch statement found in org.apache.zookeeper.server.quorum.Follower.processPacket(QuorumPacket) where default case is missing SF Switch statement found in org.apache.zookeeper.server.quorum.Observer.processPacket(QuorumPacket) where default case is missing ST Write to static field org.apache.zookeeper.server.SyncRequestProcessor.randRoll from instance method org.apache.zookeeper.server.SyncRequestProcessor.run() UrF Unread public/protected field: org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.err UrF Unread public/protected field: org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.path UrF Unread public/protected field: org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.stat UrF Unread public/protected field: org.apache.zookeeper.server.upgrade.DataTreeV1$ProcessTxnResult.type {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ZOOKEEPER-2732) Cleanup findbug warnings in branch-3.4: Performance Warnings
Rakesh R created ZOOKEEPER-2732: --- Summary: Cleanup findbug warnings in branch-3.4: Performance Warnings Key: ZOOKEEPER-2732 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2732 Project: ZooKeeper Issue Type: Sub-task Reporter: Rakesh R Please refer the attached sheet in parent jira. Below is the details of findbug warnings. {code} Bx Boxing/unboxing to parse a primitive new org.apache.zookeeper.server.quorum.QuorumCnxManager(long, Map, QuorumAuthServer, QuorumAuthLearner, int, boolean, int, boolean) Bx new org.apache.zookeeper.server.quorum.QuorumCnxManager(long, Map, QuorumAuthServer, QuorumAuthLearner, int, boolean, int, boolean) invokes inefficient new Integer(String) constructor; use Integer.valueOf(String) instead Dm org.apache.zookeeper.server.quorum.FastLeaderElection$Notification.toString() invokes inefficient new String(String) constructor WMI org.apache.zookeeper.server.DataTree.dumpEphemerals(PrintWriter) makes inefficient use of keySet iterator instead of entrySet iterator WMI org.apache.zookeeper.server.quorum.flexible.QuorumHierarchical.computeGroupWeight() makes inefficient use of keySet iterator instead of entrySet iterator WMI org.apache.zookeeper.server.quorum.flexible.QuorumHierarchical.containsQuorum(HashSet) makes inefficient use of keySet iterator instead of entrySet iterator WMI org.apache.zookeeper.ZooKeeperMain.usage() makes inefficient use of keySet iterator instead of entrySet iterator {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ZOOKEEPER-2731) Cleanup findbug warnings in branch-3.4: Malicious code vulnerability Warnings
Rakesh R created ZOOKEEPER-2731: --- Summary: Cleanup findbug warnings in branch-3.4: Malicious code vulnerability Warnings Key: ZOOKEEPER-2731 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2731 Project: ZooKeeper Issue Type: Sub-task Affects Versions: 3.4.9 Reporter: Rakesh R Please refer the attached sheet in parent jira. Below is the details of findbug warnings. {code} AT Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.zookeeper.server.quorum.QuorumCnxManager.handleConnection(Socket, DataInputStream) AT Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.zookeeper.server.quorum.QuorumCnxManager.startConnection(Socket, Long) AT Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(Long, ByteBuffer) VO Increment of volatile field org.apache.zookeeper.server.quorum.AuthFastLeaderElection.logicalclock in org.apache.zookeeper.server.quorum.AuthFastLeaderElection.leaveInstance() VO Increment of volatile field org.apache.zookeeper.server.quorum.AuthFastLeaderElection.logicalclock in org.apache.zookeeper.server.quorum.AuthFastLeaderElection.lookForLeader() VO Increment of volatile field org.apache.zookeeper.server.quorum.FastLeaderElection.logicalclock in org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader() VO Increment of volatile field org.apache.zookeeper.server.quorum.QuorumPeer.tick in org.apache.zookeeper.server.quorum.Leader.lead() VO Increment of volatile field org.apache.zookeeper.server.quorum.QuorumPeer.tick in org.apache.zookeeper.server.quorum.Leader.lead() {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ZOOKEEPER-2730) Cleanup findbug warnings in branch-3.4: Internationalization Warnings
Rakesh R created ZOOKEEPER-2730: --- Summary: Cleanup findbug warnings in branch-3.4: Internationalization Warnings Key: ZOOKEEPER-2730 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2730 Project: ZooKeeper Issue Type: Sub-task Reporter: Rakesh R Please refer the attached sheet in parent jira. Below is the details of findbug warnings. {code} Dm Found reliance on default encoding in org.apache.jute.compiler.CGenerator.genCode(): new java.io.FileWriter(File) Dm Found reliance on default encoding in org.apache.jute.compiler.CppGenerator.genCode(): new java.io.FileWriter(File) Dm Found reliance on default encoding in org.apache.jute.compiler.JRecord.genCsharpCode(File): new java.io.FileWriter(File) Dm Found reliance on default encoding in org.apache.jute.compiler.JRecord.genJavaCode(File): new java.io.FileWriter(File) Dm Found reliance on default encoding in new org.apache.jute.XmlOutputArchive(OutputStream): new java.io.PrintStream(OutputStream) Dm Found reliance on default encoding in org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, String, int): new java.io.InputStreamReader(InputStream) Dm Found reliance on default encoding in org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(String, int, String, int): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): new java.io.InputStreamReader(InputStream) Dm Found reliance on default encoding in org.apache.zookeeper.ClientCnxn$SendThread.pingRwServer(): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.server.auth.DigestAuthenticationProvider.generateDigest(String): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.server.auth.DigestAuthenticationProvider.handleAuthentication(ServerCnxn, byte[]): new String(byte[]) Dm Found reliance on default encoding in org.apache.zookeeper.server.DataTree.updateBytes(String, long): new String(byte[]) Dm Found reliance on default encoding in org.apache.zookeeper.server.DataTree.updateBytes(String, long): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.server.DataTree.updateCount(String, int): new String(byte[]) Dm Found reliance on default encoding in org.apache.zookeeper.server.DataTree.updateCount(String, int): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.server.DataTree.updateQuotaForPath(String): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.server.NettyServerCnxn$SendBufferWriter.checkFlush(boolean): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.server.NIOServerCnxn$SendBufferWriter.checkFlush(boolean): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.server.persistence.FileSnap.(): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.server.persistence.FileTxnLog.(): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.server.quorum.QuorumPeer.readLongFromFile(String): new java.io.FileReader(File) Dm Found reliance on default encoding in org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(String, long): new java.io.OutputStreamWriter(OutputStream) Dm Found reliance on default encoding in org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(Properties): new java.io.FileReader(File) Dm Found reliance on default encoding in org.apache.zookeeper.server.Request.toString(): new String(byte[]) Dm Found reliance on default encoding in org.apache.zookeeper.server.ServerCnxn.(): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.server.TraceFormatter.main(String[]): new String(byte[]) Dm Found reliance on default encoding in org.apache.zookeeper.server.util.OSMXBean.getMaxFileDescriptorCount(): new java.io.InputStreamReader(InputStream) Dm Found reliance on default encoding in org.apache.zookeeper.server.util.OSMXBean.getOpenFileDescriptorCount(): new java.io.InputStreamReader(InputStream) Dm Found reliance on default encoding in org.apache.zookeeper.ServerAdminClient.dump(String, int): new String(byte[]) Dm Found reliance on default encoding in org.apache.zookeeper.ServerAdminClient.dump(String, int): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.ServerAdminClient.getTraceMask(String, int): String.getBytes() Dm Found reliance on default encoding in org.apache.zookeeper.ServerAdminClient.kill(String, int): new String(byte[]) Dm Found reliance on default encoding in org.apache.zookeeper.ServerAdminClient.kill(String, int
[jira] [Created] (ZOOKEEPER-2729) Cleanup findbug warnings in branch-3.4: Correctness Warnings
Rakesh R created ZOOKEEPER-2729: --- Summary: Cleanup findbug warnings in branch-3.4: Correctness Warnings Key: ZOOKEEPER-2729 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2729 Project: ZooKeeper Issue Type: Sub-task Affects Versions: 3.4.9 Reporter: Rakesh R {code} (1) INT Bad comparison of nonnegative value with 0 in org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner.send(DataOutputStream, byte[]) Bug type INT_BAD_COMPARISON_WITH_NONNEGATIVE_VALUE (click for details) In class org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner In method org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthLearner.send(DataOutputStream, byte[]) Value 0 At SaslQuorumAuthLearner.java:[line 176] (2) INT Bad comparison of nonnegative value with 0 in org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer.send(DataOutputStream, byte[], QuorumAuth$Status) Bug type INT_BAD_COMPARISON_WITH_NONNEGATIVE_VALUE (click for details) In class org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer In method org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer.send(DataOutputStream, byte[], QuorumAuth$Status) Value 0 At SaslQuorumAuthServer.java:[line 170] {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ZOOKEEPER-2728) Clean up findbug warnings in branch-3.4
Rakesh R created ZOOKEEPER-2728: --- Summary: Clean up findbug warnings in branch-3.4 Key: ZOOKEEPER-2728 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2728 Project: ZooKeeper Issue Type: Bug Affects Versions: 3.4.9 Reporter: Rakesh R This jira to cleanup findbug warnings reported in branch-3.4 [Branch3.4 FindbugsWarnings.html|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/build/test/findbugs/newPatchFindbugsWarnings.html] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931855#comment-15931855 ] Rakesh R commented on ZOOKEEPER-2712: - bq. Regarding find bug warnings - I tried find bugs (3.0.1) locally and I also get 97 warnings. I suspect we did not get such warnings either because we did not run find bugs at all for branch-3.4, or find bug version updates. I think we should fix the warnings before releasing of 3.4.10, not sure if you agree with this or not. Regardless of this, I think fixing findbug warning is out scope of this particular JIRA and we could use another JIRA to track it (same for the javadoc warnings). Thank you [~hanm] for bringing the findbug warnings point. Its true, Jenkins was not running findbug tool for branch-3.4 and the report was not visible. I personally would like to avoid postponing the 3.4.10 release even further as there are many critical, security defects fixed and quorum auth sasl feature available in this. IMHO, we should fix findbugs if anything critical and should cut a {{3.4.10}} release and all the other warnings can be pushed to {{3.4.11}}? I have quickly gone through the warnings and didn't notice any critical items. Please feel free to add if you would like to discuss any specific case. Probably, could raise a jira to fix those and continue discussion over there. > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Critical > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931573#comment-15931573 ] Rakesh R commented on ZOOKEEPER-2712: - Thanks a lot [~hanm] for the unit test results and comments. bq. I am curious what exactly cause the races though - from the description it was not very clear to me. Do you mind to elaborate a little bit with regards to what code in test case that uses what function in the dependency library and what the race condition is? I hope the following explanation will help to understand the concurrency flow. Below is the auth failed exception which frequently hits in our {{KerberosSecurityTestcase}} related unit test cases. {code} 2017-03-17 15:55:51,397 [myid:] - WARN [NioProcessor-3:KerberosProtocolHandler@241] - Server not found in Kerberos database (7) 2017-03-17 15:55:51,398 [myid:] - WARN [NioProcessor-3:KerberosProtocolHandler@242] - Server not found in Kerberos database (7) [Krb5LoginModule] authentication failed Server not found in Kerberos database (7) - Server not found in Kerberos database 2017-03-17 15:55:51,409 [myid:1] - ERROR [Thread-3:QuorumPeerTestBase$MainThread@145] - unexpected exception in run javax.security.sasl.SaslException: Failed to initialize authentication mechanism using SASL [Caused by javax.security.auth.login.LoginException: Server not found in Kerberos database (7) - Server not found in Kerberos database] at org.apache.zookeeper.server.quorum.auth.SaslQuorumAuthServer.(SaslQuorumAuthServer.java:69) at org.apache.zookeeper.server.quorum.QuorumPeer.initialize(QuorumPeer.java:570) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:162) {code} As we know, test case is creating a ZK cluster of size 3 and uses [QuorumAuthTestBase.startServer() #L186|https://github.com/apache/zookeeper/blob/branch-3.4/src/java/test/org/apache/zookeeper/server/quorum/auth/QuorumAuthTestBase.java#L186] function to start server in a separate thread. Now, we have three servers starting parallel in three different threads. During startup, each server will initialize SaslQuorumAuthServer and SaslQuorumAuthLearner [QuorumPeer#init|https://github.com/apache/zookeeper/blob/branch-3.4/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java#L570] and does auth login. For Krb login, it internally uses ApacheDS library as this test case is based on {{KerberosSecurityTestcase}}. I have experimented a test scenario of doing multiple Krb {{javax.security.auth.login.LoginContext#login()}} simultaneously and hits exactly the same error {{server not found in Kerberos database}}. Later, I made the login in a sequential fashion and never hits server not found problem. I personally feel, that ApacheDS login module is sharing some resources and resulting in concurrency failure. IMHO, fixing ApacheDS is not our scope and the sequential login changes makes the test case more consistent, does this make sense to you? > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Critical > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2712: Priority: Critical (was: Major) > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Critical > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930308#comment-15930308 ] Rakesh R edited comment on ZOOKEEPER-2712 at 3/17/17 5:34 PM: -- Thanks [~hanm] for the offline discussion about the static tool warnings. {code} -1 javadoc. The javadoc tool appears to have generated 1 warning messages. {code} Jira has broken links, am adding [patchJavadocWarnings.txt|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/patchprocess/patchJavadocWarnings.txt] link. {code} -1 findbugs. The patch appears to introduce 97 new Findbugs (version 3.0.1) warnings. {code} Jira has broken links, am adding [newPatchFindbugsWarnings.html|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/build/test/findbugs/newPatchFindbugsWarnings.html] link. It looks like entire project items are showing and those are unrelated to the patch. The proposed fix is majorily touching test code apart from one source code QuorumPeer.java. So, anything required here? was (Author: rakeshr): Thanks [~hanm] for the offline discussion about the static tool warnings. {code} -1 javadoc. The javadoc tool appears to have generated 1 warning messages. {code} Jira has broken links, am adding [patchJavadocWarnings.txt|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/patchprocess/patchJavadocWarnings.txt] link. {code} -1 findbugs. The patch appears to introduce 97 new Findbugs (version 3.0.1) warnings. {code} Jira has broken links, am adding [newPatchFindbugsWarnings.html|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/build/test/findbugs/newPatchFindbugsWarnings.html] link. It looks like entire project items are showing. The proposed fix is majorily touching test code apart from one source code QuorumPeer.java. So, anything required here? > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930308#comment-15930308 ] Rakesh R commented on ZOOKEEPER-2712: - Thanks [~hanm] for the offline discussion about the static tool warnings. {code} -1 javadoc. The javadoc tool appears to have generated 1 warning messages. {code} Jira has broken links, am adding [patchJavadocWarnings.txt|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/patchprocess/patchJavadocWarnings.txt] link. {code} -1 findbugs. The patch appears to introduce 97 new Findbugs (version 3.0.1) warnings. {code} Jira has broken links, am adding [newPatchFindbugsWarnings.html|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/444/artifact/build/test/findbugs/newPatchFindbugsWarnings.html] link. It looks like entire project items are showing. The proposed fix is majorily touching test code apart from one source code QuorumPeer.java. So, anything required here? > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930227#comment-15930227 ] Rakesh R commented on ZOOKEEPER-2712: - [~hanm], Yes the test cases are happily running in my Windows env. Later, I have tried running in Linux and it is failing. After putting the delay between the server startup, I could see it is passing consistently. [~hanm], [~phunt], It would be really helpful if you could post your feedback on the proposed fix. Thanks! > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2712: Fix Version/s: (was: 3.4.11) 3.4.10 > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.4.10 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927849#comment-15927849 ] Rakesh R commented on ZOOKEEPER-2712: - [~hanm], Yes test case was consistently passing with Kerby dependency. Also, it was pretty simple compares to ApacheDS but Kerby has build requirement of jdk1.7+, http://directory.apache.org/kerby/developer-guide.html]. Since branch-3.4.x support Java 1.6 or higher, Apache Kerby dependency is not possible, right? I'm still searching the root cause of this test failure, kerb related logs are not helping much to identify the real problem. > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.4.11 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ZOOKEEPER-2716) Flaky Test: org.apache.zookeeper.server.SessionTrackerTest.testAddSessionAfterSessionExpiry
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2716. - Resolution: Fixed Fix Version/s: (was: 3.5.4) 3.5.3 3.4.10 > Flaky Test: > org.apache.zookeeper.server.SessionTrackerTest.testAddSessionAfterSessionExpiry > --- > > Key: ZOOKEEPER-2716 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2716 > Project: ZooKeeper > Issue Type: Test > Components: server, tests >Affects Versions: 3.4.9, 3.5.2 >Reporter: Michael Han >Assignee: Michael Han > Labels: flaky, flaky-build, flaky-test > Fix For: 3.4.10, 3.5.3, 3.6.0 > > > This test fail once in a while because the test logic has time assumptions > that should be fixed. PR in a minute. Sample log when it fails: > {noformat} > Error Message > Duplicate session expiry request has been generated expected:<1> but was:<0> > Stacktrace > junit.framework.AssertionFailedError: Duplicate session expiry request has > been generated expected:<1> but was:<0> > at > org.apache.zookeeper.server.SessionTrackerTest.testAddSessionAfterSessionExpiry(SessionTrackerTest.java:82) > at > org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.lang.Thread.run(Thread.java:745) > Standard Output > 2017-03-04 19:25:16,141 [myid:] - INFO [main:JUnit4ZKTestRunner@47] - No > test.method specified. using default methods. > 2017-03-04 19:25:16,358 [myid:] - INFO [main:JUnit4ZKTestRunner@47] - No > test.method specified. using default methods. > 2017-03-04 19:25:16,419 [myid:] - INFO [main:ZKTestCase$1@58] - STARTING > testAddSessionAfterSessionExpiry > 2017-03-04 19:25:16,430 [myid:] - INFO [Time-limited > test:JUnit4ZKTestRunner$LoggedInvokeMethod@77] - RUNNING TEST METHOD > testAddSessionAfterSessionExpiry > 2017-03-04 19:25:16,581 [myid:] - INFO [Time-limited test:Environment@109] - > Server > environment:zookeeper.version=3.5.3-alpha-SNAPSHOT-6d9fc04c052adbc791c63f3f00c816fb8e56, > built on 03/04/2017 19:24 GMT > 2017-03-04 19:25:16,581 [myid:] - INFO [Time-limited test:Environment@109] - > Server environment:host.name=jenkins-ubuntu3.apache.org > 2017-03-04 19:25:16,581 [myid:] - INFO [Time-limited test:Environment@109] - > Server environment:java.version=1.8.0_121 > 2017-03-04 19:25:16,581 [myid:] - INFO [Time-limited test:Environment@109] - > Server environment:java.vendor=Oracle Corporation > 2017-03-04 19:25:16,582 [myid:] - INFO [Time-limited test:Environment@109] - > Server environment:java.home=/usr/local/asfpackages/java/jdk1.8.0_121/jre > 2017-03-04 19:25:16,587 [myid:] - INFO [Time-limited test:Environment@109] - > Server > environment:java.class.path=/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/classes:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/antlr-2.7.7.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/antlr4-runtime-4.5.1-1.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/checkstyle-6.13.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/commons-beanutils-1.9.2.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/commons-cli-1.3.1.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/commons-collections-3.2.2.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/commons-lang3-3.4.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/commons-logging-1.1.1.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/guava-18.0.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/hamcrest-core-1.3.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/junit-4.12.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/test/lib/mockito-all-1.8.2.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/classes:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/src/java/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/latest/lib/ant.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/lib/commons-cli-1.2.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch35_jdk8/build/lib/jackson-core-asl-1.9.11.jar:/home/jenkins/jenkins-slave/works
[jira] [Reopened] (ZOOKEEPER-2722) Flaky Test: org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reopened ZOOKEEPER-2722: - > Flaky Test: > org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment > --- > > Key: ZOOKEEPER-2722 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2722 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Affects Versions: 3.4.9, 3.5.2 >Reporter: Michael Han >Assignee: Michael Han > Labels: flaky, flaky-build, flaky-test > Fix For: 3.5.3, 3.6.0 > > > {noformat} > Error Message > KeeperErrorCode = ConnectionLoss for /test > Stacktrace > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss for /test > at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1423) > at > org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment(ReadOnlyModeTest.java:238) > at > org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Looks like we should retry before giving up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2722) Flaky Test: org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2722: Fix Version/s: (was: 3.4.10) > Flaky Test: > org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment > --- > > Key: ZOOKEEPER-2722 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2722 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Affects Versions: 3.4.9, 3.5.2 >Reporter: Michael Han >Assignee: Michael Han > Labels: flaky, flaky-build, flaky-test > Fix For: 3.5.3, 3.6.0 > > > {noformat} > Error Message > KeeperErrorCode = ConnectionLoss for /test > Stacktrace > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss for /test > at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1423) > at > org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment(ReadOnlyModeTest.java:238) > at > org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Looks like we should retry before giving up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ZOOKEEPER-2722) Flaky Test: org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2722. - Resolution: Fixed Fix Version/s: 3.4.10 > Flaky Test: > org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment > --- > > Key: ZOOKEEPER-2722 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2722 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Affects Versions: 3.4.9, 3.5.2 >Reporter: Michael Han >Assignee: Michael Han > Labels: flaky, flaky-build, flaky-test > Fix For: 3.4.10, 3.5.3, 3.6.0 > > > {noformat} > Error Message > KeeperErrorCode = ConnectionLoss for /test > Stacktrace > org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode > = ConnectionLoss for /test > at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1423) > at > org.apache.zookeeper.test.ReadOnlyModeTest.testSessionEstablishment(ReadOnlyModeTest.java:238) > at > org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Looks like we should retry before giving up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2713) Create CVE text for ZOOKEEPER-2693 "DOS attack on wchp/wchc four letter words (4lw)"
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901513#comment-15901513 ] Rakesh R commented on ZOOKEEPER-2713: - Hi [~phunt], Some time back, I've cut [release candidate for {{3.4.10}}|https://qnalist.com/questions/8011199/apache-zookeeper-release-3-4-10-candidate-0] including ZOOKEEPER-2693. Do I need to wait to settle down this jira and cut another candidate-1 ? > Create CVE text for ZOOKEEPER-2693 "DOS attack on wchp/wchc four letter words > (4lw)" > > > Key: ZOOKEEPER-2713 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2713 > Project: ZooKeeper > Issue Type: Task > Components: security >Reporter: Patrick Hunt >Priority: Blocker > Fix For: 3.4.10, 3.5.3 > > > We need to agree to the CVE text for ZOOKEEPER-2693. Let's use the comments > here to do so. > The assigned CVE number is CVE-2017-5637 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
Rakesh R created ZOOKEEPER-2712: --- Summary: MiniKdc test case intermittently failing due to principal not found in Kerberos database Key: ZOOKEEPER-2712 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 Project: ZooKeeper Issue Type: Bug Components: tests Reporter: Rakesh R Assignee: Rakesh R Fix For: 3.4.11 MiniKdc test cases are intermittently failing due to not finding the principal. Below is the failure stacktrace. {code} 2017-03-08 13:21:10,843 [myid:] - ERROR [NioProcessor-1:AuthenticationService@187] - Error while searching for client lear...@example.com : Client not found in Kerberos database 2017-03-08 13:21:10,843 [myid:] - WARN [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos database (7) 2017-03-08 13:21:10,845 [myid:] - WARN [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos database (7) 2017-03-08 13:21:10,844 [myid:] - WARN [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos database (6) 2017-03-08 13:21:10,845 [myid:] - WARN [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos database (6) {code} Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2712) MiniKdc test case intermittently failing due to principal not found in Kerberos database
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2712: Attachment: TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt I'm attaching the failure test case log. > MiniKdc test case intermittently failing due to principal not found in > Kerberos database > > > Key: ZOOKEEPER-2712 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2712 > Project: ZooKeeper > Issue Type: Bug > Components: tests >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.4.11 > > Attachments: > TEST-org.apache.zookeeper.server.quorum.auth.QuorumKerberosAuthTest.txt > > > MiniKdc test cases are intermittently failing due to not finding the > principal. Below is the failure stacktrace. > {code} > 2017-03-08 13:21:10,843 [myid:] - ERROR > [NioProcessor-1:AuthenticationService@187] - Error while searching for client > lear...@example.com : Client not found in Kerberos database > 2017-03-08 13:21:10,843 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@241] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-2:KerberosProtocolHandler@242] - Server not found in Kerberos > database (7) > 2017-03-08 13:21:10,844 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@241] - Client not found in Kerberos > database (6) > 2017-03-08 13:21:10,845 [myid:] - WARN > [NioProcessor-1:KerberosProtocolHandler@242] - Client not found in Kerberos > database (6) > {code} > Will attach the detailed log to jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2665) Port QA github pull request build to branch 3.4 and 3.5
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900683#comment-15900683 ] Rakesh R commented on ZOOKEEPER-2665: - Hi [~fpj], please add [~eolivelli] to the contributor list and assign the issue to him. Thanks! > Port QA github pull request build to branch 3.4 and 3.5 > --- > > Key: ZOOKEEPER-2665 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2665 > Project: ZooKeeper > Issue Type: Test > Components: build >Reporter: Flavio Junqueira > Fix For: 3.4.10, 3.5.3 > > > We have QA build for pull requests against master but not against branches > 3.4 and 3.5. We need to port the necessary wiring to do it, it shouldn't be > difficult. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ZOOKEEPER-2710) Regenerate documentation for branch-3.4 release
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved ZOOKEEPER-2710. - Resolution: Fixed Fix Version/s: 3.4.10 Regenerated ZooKeeper documentation and committed to {{branch-3.4}} Branch 3.4: https://git-wip-us.apache.org/repos/asf?p=zookeeper.git;a=commit;h=09555983be93e5278b1ab8a71e3ca9f54ea6c374 > Regenerate documentation for branch-3.4 release > --- > > Key: ZOOKEEPER-2710 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2710 > Project: ZooKeeper > Issue Type: Bug > Components: documentation >Affects Versions: 3.4.10 >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.4.10 > > > This jira can be used to regenerate the documentation as some of the recent > commits didn't regenerated the doc section. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2710) Regenerate documentation for branch-3.4 release
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2710: Component/s: documentation > Regenerate documentation for branch-3.4 release > --- > > Key: ZOOKEEPER-2710 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2710 > Project: ZooKeeper > Issue Type: Bug > Components: documentation >Affects Versions: 3.4.10 >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: 3.4.10 > > > This jira can be used to regenerate the documentation as some of the recent > commits didn't regenerated the doc section. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ZOOKEEPER-2710) Regenerate documentation for branch-3.4 release
Rakesh R created ZOOKEEPER-2710: --- Summary: Regenerate documentation for branch-3.4 release Key: ZOOKEEPER-2710 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2710 Project: ZooKeeper Issue Type: Bug Affects Versions: 3.4.10 Reporter: Rakesh R Assignee: Rakesh R This jira can be used to regenerate the documentation as some of the recent commits didn't regenerated the doc section. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2693) DOS attack on wchp/wchc four letter words (4lw)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898792#comment-15898792 ] Rakesh R commented on ZOOKEEPER-2693: - I will merge this shortly. > DOS attack on wchp/wchc four letter words (4lw) > --- > > Key: ZOOKEEPER-2693 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2693 > Project: ZooKeeper > Issue Type: Bug > Components: security, server >Affects Versions: 3.4.0, 3.5.1, 3.5.2 >Reporter: Patrick Hunt >Assignee: Michael Han >Priority: Blocker > Fix For: 3.4.10, 3.5.3 > > Attachments: ZOOKEEPER-2693-01.patch > > > The wchp/wchc four letter words can be exploited in a DOS attack on the ZK > client port - typically 2181. The following POC attack was recently published > on the web: > https://webcache.googleusercontent.com/search?q=cache:_CNGIz10PRYJ:https://www.exploit-db.com/exploits/41277/+=14=en=clnk=us > The most straightforward way to block this attack is to not allow access to > the client port to non-trusted clients - i.e. firewall the ZooKeeper service > and only allow access to trusted applications using it for coordination. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ZOOKEEPER-2696) Eclipse ant task no longer determines correct classpath for tests after ZOOKEEPER-2689
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated ZOOKEEPER-2696: Component/s: build > Eclipse ant task no longer determines correct classpath for tests after > ZOOKEEPER-2689 > -- > > Key: ZOOKEEPER-2696 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2696 > Project: ZooKeeper > Issue Type: Bug > Components: build >Affects Versions: 3.4.10 >Reporter: Abraham Fine >Assignee: Abraham Fine > Fix For: 3.4.10 > > > Following the changes made in ZOOKEEPER-2689 IDE's using the .classpath file > generated by the eclipse ant task (I tested both idea and eclipse) cannot > compile the tests. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2696) Eclipse ant task no longer determines correct classpath for tests after ZOOKEEPER-2689
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885210#comment-15885210 ] Rakesh R commented on ZOOKEEPER-2696: - Thanks [~abrahamfine]. Committed to branch-3.4: https://git-wip-us.apache.org/repos/asf?p=zookeeper.git;a=commit;h=b26eaf3492a43821f648d8a8f87d8ac3488f1cef > Eclipse ant task no longer determines correct classpath for tests after > ZOOKEEPER-2689 > -- > > Key: ZOOKEEPER-2696 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2696 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.4.10 >Reporter: Abraham Fine >Assignee: Abraham Fine > Fix For: 3.4.10 > > > Following the changes made in ZOOKEEPER-2689 IDE's using the .classpath file > generated by the eclipse ant task (I tested both idea and eclipse) cannot > compile the tests. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2693) DOS attack on wchp/wchc four letter words (4lw)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881932#comment-15881932 ] Rakesh R commented on ZOOKEEPER-2693: - I'm trying an attempt to unblock 3.4.10 and 3.5.3 releases. Following are few proposals to make the release happen by including the reported issue. *3.4.10 requirement:* Expose whitelist configuration with the default values, {{4lw.commands.whitelist=ruok,srvr,crst,srst,isro,mntr}} Also, properly documenting the {{Publicly accessible deployment}} part from the current PR_179 changes. *3.5.3 requirement:* IMHO, there are two possible proposals: # +Proposal-1)+ Expose whitelist configuration with the default values, {{4lw.commands.whitelist=srvr,isro}} Also, recommend users to use admin server rather than 4lw cmds considering that 4lw will deprecated in future. Then, later in 3.5.4, we could make this whitelist and while deprecating will replace srvr, isro calls with better solution. That way, we would get enough time to think about better solutions. # +Proposal-2)+ Expose whitelist configuration with the empty default value, {{4lw.commands.whitelist=}} * (a) How about zkServer.sh usage of "srvr" can be achieved like, Introduce a new admin API, admin#getServerStatus("host:port"), which will return running stat of that server(probably, the same string format where srvr command is returning) {code} "version= read_only=false server_stats= node_count=" {code} * (b) Client#pingRwServer=> Just a plain thought, probably, readonly client code can internally tries to establish new client session with all other servers round robin fashion. Then, check whether that the connected server is in {{rw}} mode and act upon. > DOS attack on wchp/wchc four letter words (4lw) > --- > > Key: ZOOKEEPER-2693 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2693 > Project: ZooKeeper > Issue Type: Bug > Components: security, server >Affects Versions: 3.4.0, 3.5.1, 3.5.2 >Reporter: Patrick Hunt >Assignee: Michael Han >Priority: Blocker > Fix For: 3.4.10, 3.5.3 > > Attachments: ZOOKEEPER-2693-01.patch > > > The wchp/wchc four letter words can be exploited in a DOS attack on the ZK > client port - typically 2181. The following POC attack was recently published > on the web: > https://webcache.googleusercontent.com/search?q=cache:_CNGIz10PRYJ:https://www.exploit-db.com/exploits/41277/+=14=en=clnk=us > The most straightforward way to block this attack is to not allow access to > the client port to non-trusted clients - i.e. firewall the ZooKeeper service > and only allow access to trusted applications using it for coordination. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2693) DOS attack on wchp/wchc four letter words (4lw)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877512#comment-15877512 ] Rakesh R commented on ZOOKEEPER-2693: - Thanks [~hanm] for making good progress. {quote} I will once I get this landed in 3.5. PR to 3.4 will not be much different, but I'd like to finalize this PR first to avoid potential duplicated efforts. Meanwhile, I'll create a set of follow JIRAs to address concerns of abusing 4lw in general: {quote} Makes sense to me. If I understand your comment correctly, we could push {{whitelist config}} changes first which will unblock 3.4.10 releasing. But for 3.5.3 releasing, we need both {{whitelist config}} and {{deprecating 4lw tasks}} together. Probably, {{4lw rate limiting task}} is nice to have feature(keep it low priority compare to other tasks), right? > DOS attack on wchp/wchc four letter words (4lw) > --- > > Key: ZOOKEEPER-2693 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2693 > Project: ZooKeeper > Issue Type: Bug > Components: security, server >Affects Versions: 3.4.0, 3.5.1, 3.5.2 >Reporter: Patrick Hunt >Assignee: Michael Han >Priority: Blocker > Fix For: 3.4.10, 3.5.3 > > Attachments: ZOOKEEPER-2693-01.patch > > > The wchp/wchc four letter words can be exploited in a DOS attack on the ZK > client port - typically 2181. The following POC attack was recently published > on the web: > https://webcache.googleusercontent.com/search?q=cache:_CNGIz10PRYJ:https://www.exploit-db.com/exploits/41277/+=14=en=clnk=us > The most straightforward way to block this attack is to not allow access to > the client port to non-trusted clients - i.e. firewall the ZooKeeper service > and only allow access to trusted applications using it for coordination. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2693) DOS attack on wchp/wchc four letter words (4lw)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876313#comment-15876313 ] Rakesh R commented on ZOOKEEPER-2693: - bq. There are some 4lw commands which ZooKeeper is using by itself Good catch, [~arshad.mohammad]. bq. even though we are not doing much work in these commands but still the connections will be created for every call. IIUC, these are two problems -> {{case-1)}} restrict 4lw cmd execution as few cmds taking too much time for execution. {{case-2)}} protection against overuse because it creates many connections. I think, {{case-1}} is matching with this jira reported issue and we could provide {{whitelist}} config in both branches 3.4 and 3.5+ to solve this problem. In branch-3.4, we could give a simple fix by exposing {{whitelist}} configuration and include {{srvr, isro}} cmds in default list. Since we have plans to deprecate 4lws in branch-3.5, we could find alternative ways instead of using {{srvr, isro}} cmd internally. Probably, we could raise separate jira task and handle this case. [~hanm], could you create a PR for branch-3.4, if no objection from anyone about {{whitelist}} idea and that would help to unblock 3.4.10 releasing. {{case-2}}, I like [~phunt]'s idea of introducing configuration to limit the number of 4lw that would be allowed to run in parallel. For example, allows only 1-n number of 4lw cmds to run in parallel. How about raising another jira to implement this instead of clubbing with this issue? > DOS attack on wchp/wchc four letter words (4lw) > --- > > Key: ZOOKEEPER-2693 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2693 > Project: ZooKeeper > Issue Type: Bug > Components: security, server >Affects Versions: 3.4.0, 3.5.1, 3.5.2 >Reporter: Patrick Hunt >Assignee: Michael Han >Priority: Blocker > Fix For: 3.4.10, 3.5.3 > > Attachments: ZOOKEEPER-2693-01.patch > > > The wchp/wchc four letter words can be exploited in a DOS attack on the ZK > client port - typically 2181. The following POC attack was recently published > on the web: > https://webcache.googleusercontent.com/search?q=cache:_CNGIz10PRYJ:https://www.exploit-db.com/exploits/41277/+=14=en=clnk=us > The most straightforward way to block this attack is to not allow access to > the client port to non-trusted clients - i.e. firewall the ZooKeeper service > and only allow access to trusted applications using it for coordination. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869254#comment-15869254 ] Rakesh R commented on ZOOKEEPER-2260: - FYI, I've triggered [build_338|https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/338/] to get the report of https://github.com/apache/zookeeper/pull/50 recent changes. > Paginated getChildren call > -- > > Key: ZOOKEEPER-2260 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260 > Project: ZooKeeper > Issue Type: New Feature >Affects Versions: 3.4.6, 3.5.0 >Reporter: Marco P. >Assignee: Marco P. > Labels: api, features > Fix For: 3.5.3, 3.6.0 > > Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch > > > Add pagination support to the getChildren() call, allowing clients to iterate > over children N at the time. > Motivations for this include: > - Getting out of a situation where so many children were created that > listing them exceeded the network buffer sizes (making it impossible to > recover by deleting)[1] > - More efficient traversal of nodes with large number of children [2] > I do have a patch (for 3.4.6) we've been using successfully for a while, but > I suspect much more work is needed for this to be accepted. > [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272 > [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ZOOKEEPER-2260) Paginated getChildren call
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869027#comment-15869027 ] Rakesh R commented on ZOOKEEPER-2260: - bq. Jenkins test this please Thanks [~mprime] for your time and continuous effort in making this feature. I could see Jenkins PreCommit build is failing due to {{Unsupported major.minor version 52.0}}. I've filed INFRA-13526 jira to resolve this. Will trigger the Precommit build once the INFRA problem is resolved. > Paginated getChildren call > -- > > Key: ZOOKEEPER-2260 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2260 > Project: ZooKeeper > Issue Type: New Feature >Affects Versions: 3.4.6, 3.5.0 >Reporter: Marco P. >Assignee: Marco P. > Labels: api, features > Fix For: 3.5.3, 3.6.0 > > Attachments: ZOOKEEPER-2260.patch, ZOOKEEPER-2260.patch > > > Add pagination support to the getChildren() call, allowing clients to iterate > over children N at the time. > Motivations for this include: > - Getting out of a situation where so many children were created that > listing them exceeded the network buffer sizes (making it impossible to > recover by deleting)[1] > - More efficient traversal of nodes with large number of children [2] > I do have a patch (for 3.4.6) we've been using successfully for a while, but > I suspect much more work is needed for this to be accepted. > [1] https://issues.apache.org/jira/browse/ZOOKEEPER-272 > [2] https://issues.apache.org/jira/browse/ZOOKEEPER-282 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (ZOOKEEPER-2597) Add script to merge PR from Apache git repo to Github
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned ZOOKEEPER-2597: --- Assignee: Rakesh R (was: Edward Ribeiro) > Add script to merge PR from Apache git repo to Github > - > > Key: ZOOKEEPER-2597 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2597 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Edward Ribeiro > Assignee: Rakesh R >Priority: Minor > Attachments: ZOOKEEPER-2597.patch, ZOOKEEPER-2597.patch > > > A port of kafka-merge-pr.py to workon on ZooKeeper repo. -- This message was sent by Atlassian JIRA (v6.3.15#6346)