[jira] [Commented] (ZOOKEEPER-3501) unify the method:op2String()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934648#comment-16934648 ] Hudson commented on ZOOKEEPER-3501: --- SUCCESS: Integrated in Jenkins build Zookeeper-trunk-single-thread #540 (See [https://builds.apache.org/job/Zookeeper-trunk-single-thread/540/]) ZOOKEEPER-3501: unify the method:op2String() (nkalmar: rev a179caa668882d6129ebb000ee60455de749d34f) * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/TraceFormatter.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/persistence/TxnLogToolkit.java * (edit) zookeeper-server/src/test/java/org/apache/zookeeper/test/LocalSessionRequestTest.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/LogFormatter.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/Request.java > unify the method:op2String() > > > Key: ZOOKEEPER-3501 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3501 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: maoling >Assignee: maoling >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 50m > Remaining Estimate: 0h > > there were two duplicated method > *public static String op2String(int op)* > in the code base: > > {code:java} > org.apache.zookeeper.server.TraceFormatter#op2String > org.apache.zookeeper.server.Request#op2String > {code} > > and they are inconsistency, we should unify it and remain only one > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ZOOKEEPER-3506) correct the SessionTrackerImpl#initializeNextSession's javaDoc about how to generate the sessionId
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934647#comment-16934647 ] Hudson commented on ZOOKEEPER-3506: --- SUCCESS: Integrated in Jenkins build Zookeeper-trunk-single-thread #540 (See [https://builds.apache.org/job/Zookeeper-trunk-single-thread/540/]) ZOOKEEPER-3506: correct the javaDoc of the (nkalmar: rev 2c1034f8fe4e6e437f7c71478153f41180970542) * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/SessionTrackerImpl.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/LearnerSessionTracker.java > correct the SessionTrackerImpl#initializeNextSession's javaDoc about how to > generate the sessionId > -- > > Key: ZOOKEEPER-3506 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3506 > Project: ZooKeeper > Issue Type: Improvement > Components: documentation, server >Reporter: maoling >Assignee: maoling >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 40m > Remaining Estimate: 0h > > > {code:java} > /** > * Generates an initial sessionId. High order byte is serverId, next 5 > * 5 bytes are from timestamp, and low order 2 bytes are 0s. > */ > public static long initializeNextSession(long id) { > long nextSid; > nextSid = (Time.currentElapsedTime() << 24) >>> 8; > nextSid = nextSid | (id <<56); > if (nextSid == EphemeralType.CONTAINER_EPHEMERAL_OWNER) { > ++nextSid; // this is an unlikely edge case, but check it just in > case > } > return nextSid; > } > {code} > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ZOOKEEPER-3506) correct the SessionTrackerImpl#initializeNextSession's javaDoc about how to generate the sessionId
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934579#comment-16934579 ] Hudson commented on ZOOKEEPER-3506: --- SUCCESS: Integrated in Jenkins build ZooKeeper-trunk #703 (See [https://builds.apache.org/job/ZooKeeper-trunk/703/]) ZOOKEEPER-3506: correct the javaDoc of the (nkalmar: rev 2c1034f8fe4e6e437f7c71478153f41180970542) * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/SessionTrackerImpl.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/LearnerSessionTracker.java > correct the SessionTrackerImpl#initializeNextSession's javaDoc about how to > generate the sessionId > -- > > Key: ZOOKEEPER-3506 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3506 > Project: ZooKeeper > Issue Type: Improvement > Components: documentation, server >Reporter: maoling >Assignee: maoling >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 40m > Remaining Estimate: 0h > > > {code:java} > /** > * Generates an initial sessionId. High order byte is serverId, next 5 > * 5 bytes are from timestamp, and low order 2 bytes are 0s. > */ > public static long initializeNextSession(long id) { > long nextSid; > nextSid = (Time.currentElapsedTime() << 24) >>> 8; > nextSid = nextSid | (id <<56); > if (nextSid == EphemeralType.CONTAINER_EPHEMERAL_OWNER) { > ++nextSid; // this is an unlikely edge case, but check it just in > case > } > return nextSid; > } > {code} > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ZOOKEEPER-3501) unify the method:op2String()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934580#comment-16934580 ] Hudson commented on ZOOKEEPER-3501: --- SUCCESS: Integrated in Jenkins build ZooKeeper-trunk #703 (See [https://builds.apache.org/job/ZooKeeper-trunk/703/]) ZOOKEEPER-3501: unify the method:op2String() (nkalmar: rev a179caa668882d6129ebb000ee60455de749d34f) * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/TraceFormatter.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/persistence/TxnLogToolkit.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/LogFormatter.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/Request.java * (edit) zookeeper-server/src/test/java/org/apache/zookeeper/test/LocalSessionRequestTest.java > unify the method:op2String() > > > Key: ZOOKEEPER-3501 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3501 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: maoling >Assignee: maoling >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 50m > Remaining Estimate: 0h > > there were two duplicated method > *public static String op2String(int op)* > in the code base: > > {code:java} > org.apache.zookeeper.server.TraceFormatter#op2String > org.apache.zookeeper.server.Request#op2String > {code} > > and they are inconsistency, we should unify it and remain only one > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ZOOKEEPER-102) Need to replace Jute with supported code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934484#comment-16934484 ] tison edited comment on ZOOKEEPER-102 at 9/20/19 3:13 PM: -- Here is a important question: shall we want an intermediate stage support both jute and the other serialization framework? I have visited jute abstraction and its {{readInt}}/{{writeInt}} etc. are hard to adopt in other frameworks. kyro may have a near abstraction(for {{serialize}}/{{deserialize}}) but neither protobuf nor thrift seems compatible with jute abstractly. was (Author: tison): Here is a important question: shall we want an immediate stage support both jute and the other serialization framework? I have visited jute abstraction and its {{readInt}}/{{writeInt}} etc. are hard to adopt in other frameworks. kyro may have a near abstraction(for {{serialize}}/{{deserialize}}) but neither protobuf nor thrift seems compatible with jute abstractly. > Need to replace Jute with supported code > > > Key: ZOOKEEPER-102 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Benjamin Reed >Priority: Major > Fix For: 4.0.0 > > > ZooKeeper currently uses Jute to serialize objects to put on the wire and on > disk. We pulled Jute out of Hadoop and added a C binding. Both versions of > Jute have evolved (although Hadoop still doesn't have a C binding). It would > be nice to use a more standard serialization library. Some options include > Thrift or Google's protocol buffers. > Our main requirements would be Java and C bindings and good performance. (For > example, serializing to XML would give us incredibly bad performance and > would not be acceptible!) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ZOOKEEPER-102) Need to replace Jute with supported code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934484#comment-16934484 ] tison commented on ZOOKEEPER-102: - Here is a important question: shall we want an immediate stage support both jute and the other serialization framework? I have visited jute abstraction and its {{readInt}}/{{writeInt}} etc. are hard to adopt in other frameworks. kyro may have a near abstraction(for {{serialize}}/{{deserialize}}) but neither protobuf nor thrift seems compatible with jute abstractly. > Need to replace Jute with supported code > > > Key: ZOOKEEPER-102 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Benjamin Reed >Priority: Major > Fix For: 4.0.0 > > > ZooKeeper currently uses Jute to serialize objects to put on the wire and on > disk. We pulled Jute out of Hadoop and added a C binding. Both versions of > Jute have evolved (although Hadoop still doesn't have a C binding). It would > be nice to use a more standard serialization library. Some options include > Thrift or Google's protocol buffers. > Our main requirements would be Java and C bindings and good performance. (For > example, serializing to XML would give us incredibly bad performance and > would not be acceptible!) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ZOOKEEPER-3470) Flaky test: LearnerMetricsTest.testLearnerMetricsTest()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934423#comment-16934423 ] Mate Szalay-Beko commented on ZOOKEEPER-3470: - I just submitted a PR. It is definitely fixing an issue I found in the jenkins logs, let's hope there are no more problems :) (I executed the fixed test a couple of hundred times, succeeded all the time) > Flaky test: LearnerMetricsTest.testLearnerMetricsTest() > --- > > Key: ZOOKEEPER-3470 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3470 > Project: ZooKeeper > Issue Type: Sub-task > Components: tests >Affects Versions: 3.6.0 >Reporter: Andor Molnar >Assignee: Mate Szalay-Beko >Priority: Major > Labels: flaky, flaky-test, newbie, pull-request-available, test > Fix For: 3.6.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Hi team, > New test testLearnerMetricsTest() added by the following commit failed 2 > times on master with the same error: > junit.framework.AssertionFailedError: expected:<10> but was:<9> > at > org.apache.zookeeper.server.quorum.LearnerMetricsTest.testLearnerMetricsTest(LearnerMetricsTest.java:88) > at > org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:80) > [https://github.com/apache/zookeeper/pull/856] > Submitted by: jhuan31 > Maybe we should just blame the Thread.sleep(200) in line:77 and replace it > with some clever logic. > Please take a look. > Regards, > Andor -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ZOOKEEPER-3553) Create Performance Test Tool
David Mollitor created ZOOKEEPER-3553: - Summary: Create Performance Test Tool Key: ZOOKEEPER-3553 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3553 Project: ZooKeeper Issue Type: New Feature Affects Versions: 3.6.0 Reporter: David Mollitor Create a tool similar to [Dynamometer|https://github.com/linkedin/dynamometer] for ZooKeeper. To use this tool, the operator should collect transaction log files from a live instance of ZooKeeper and supply them to the tool. The tool then simply replays all of the commands it finds in all of the supplied transaction logs, in order. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ZOOKEEPER-3496) Transaction larger than jute.maxbuffer makes ZooKeeper unavailable
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934358#comment-16934358 ] Hadoop QA commented on ZOOKEEPER-3496: -- +1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version ) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/4148//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/4148//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/4148//console This message is automatically generated. > Transaction larger than jute.maxbuffer makes ZooKeeper unavailable > -- > > Key: ZOOKEEPER-3496 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3496 > Project: ZooKeeper > Issue Type: Bug >Affects Versions: 3.5.5, 3.4.14 >Reporter: Mohammad Arshad >Assignee: Mohammad Arshad >Priority: Critical > Labels: pull-request-available > Fix For: 3.6.0, 3.4.15, 3.5.7 > > Attachments: ZOOKEEPER-3496-001.patch > > Time Spent: 3.5h > Remaining Estimate: 0h > > *Problem:* > ZooKeeper server fails to start, logs following error > {code:java} > Exception in thread "main" java.io.IOException: Unreasonable length = 1001025 > at > org.apache.jute.BinaryInputArchive.checkLength(BinaryInputArchive.java:127) > at > org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:92) > {code} > This indicates that one of the transactions size is more than the configured > jute.maxbuffer values. But how transaction more than jute.maxbuffer size is > allowed to write? > *Analysis:* > At ZooKeeper server jute.maxbuffer specifies the maximum size of a > transaction. By default it is 1 MB at the server > jute.maxbuffer is used for following: > # Size sanity check of incoming request. Incoming requests size must not be > more than jute.maxbuffer > # Size sanity check of the transaction while reading from transaction or > snapshot file. Transaction size must not be more than jute.maxbuffer+1024 > # Size sanity check of transaction while reading data from the leader. > Transaction size must not be more than jute.maxbuffer+1024 > Request size sanity check is done in the beginning of a request processing > but later request processing adds additional information into request then > writes to transaction file. This additional information size is not > considered in sanity check. This is how transaction larger than > jute.maxbuffer are accepted into ZooKeeper. > If this additional information size is less than 1024 Bytes then it is OK as > ZooKeeper already takes care of it. > But if this additional information size is more than 1024 bytes it allows the > request, But while reading from transaction/snapshot file and while reading > from leader it fails and make the ZooKeeper service unavailable > +Example:+ > Suppose incoming request size is 100 Bytes > Configured jute.maxbuffer is 100 > After processing the request ZooKeeper server adds 1025 more bytes > In this case, request will be processed successfully, and 100+1025 bytes > will be written to transaction file > But while reading from the transaction log 100+1025 bytes cannot be read > as max allowed length is 100(effectively 100+1024). > *Solutions:* > If incoming request size sanity check is done after populating all additional > information then this problem is solved. But doing sanity check in the later > stage of request processing will defeat the purpose of sanity check itself. > So this we can not do > Currently additional information size is constant 1024 Bytes [Code > Reference|https://github.com/apache/zookeeper/blob/branch-3.5/zookeeper-jute/src/main/java/org/apache/jute/BinaryInputArchive.java#L126]. > We should increase this value and make it more reasonable. I propose to make > this additional information size to same as the jute.maxbuffer. Also make > additional information size configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ZOOKEEPER-3536) On Windows maven build generates corrupted tarball
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Arshad resolved ZOOKEEPER-3536. Resolution: Invalid It is not an issue > On Windows maven build generates corrupted tarball > -- > > Key: ZOOKEEPER-3536 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3536 > Project: ZooKeeper > Issue Type: Bug > Components: build >Affects Versions: 3.5.5 >Reporter: Mohammad Arshad >Priority: Minor > > On windows maven command {code}mvn clean install -DskipTests{code} creates > corrupted tarballs. > In zookeeper-assembly/pom.xml posix > causing the problem. Many use Windows as development environment. it would > be better if we can make tarLongFileMode property configurable or select > based on OS. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ZOOKEEPER-3524) LearnerMetricsTest.testLearnerMetricsTest is flaky again
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Szalay-Beko resolved ZOOKEEPER-3524. - Resolution: Duplicate original Jira was reopened: https://issues.apache.org/jira/browse/ZOOKEEPER-3470 > LearnerMetricsTest.testLearnerMetricsTest is flaky again > - > > Key: ZOOKEEPER-3524 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3524 > Project: ZooKeeper > Issue Type: Bug >Reporter: Mate Szalay-Beko >Assignee: Mate Szalay-Beko >Priority: Major > > We had an earlier fix in ZOOKEEPER-3470 for this test, but it looks failing > again. > I haven't found any failures on the [zookeeper trunk > job|https://builds.apache.org/view/ZK%20All/job/ZooKeeper-trunk], but it does > fail from time to time on the [precommit maven jobs > |https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build-maven/org.apache.zookeeper$zookeeper/1171/testReport/junit/org.apache.zookeeper.server.quorum/LearnerMetricsTest/testLearnerMetricsTest/] > or on > [zookeeper-master-maven|https://builds.apache.org/view/ZK%20All/job/zookeeper-master-maven/lastCompletedBuild/org.apache.zookeeper$zookeeper/testReport/org.apache.zookeeper.server.quorum/LearnerMetricsTest/testLearnerMetricsTest/]. > > Can this be maven related somehow? would be strange... -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ZOOKEEPER-3552) Source tarbal for branch-3.5 does not set execution permissions to "configure" file
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Norbert Kalmar resolved ZOOKEEPER-3552. --- Resolution: Fixed > Source tarbal for branch-3.5 does not set execution permissions to > "configure" file > --- > > Key: ZOOKEEPER-3552 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3552 > Project: ZooKeeper > Issue Type: Bug > Components: build, c client >Affects Versions: 3.5.6 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli >Priority: Blocker > Labels: pull-request-available > Fix For: 3.5.6 > > Time Spent: 50m > Remaining Estimate: 0h > > During the Rc0 VOTE of 3.5.6 we found that the 'configure' file inside the > source tarball does not have the right permissions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ZOOKEEPER-3501) unify the method:op2String()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Norbert Kalmar resolved ZOOKEEPER-3501. --- Resolution: Fixed > unify the method:op2String() > > > Key: ZOOKEEPER-3501 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3501 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: maoling >Assignee: maoling >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 50m > Remaining Estimate: 0h > > there were two duplicated method > *public static String op2String(int op)* > in the code base: > > {code:java} > org.apache.zookeeper.server.TraceFormatter#op2String > org.apache.zookeeper.server.Request#op2String > {code} > > and they are inconsistency, we should unify it and remain only one > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ZOOKEEPER-3506) correct the SessionTrackerImpl#initializeNextSession's javaDoc about how to generate the sessionId
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Norbert Kalmar resolved ZOOKEEPER-3506. --- Resolution: Fixed > correct the SessionTrackerImpl#initializeNextSession's javaDoc about how to > generate the sessionId > -- > > Key: ZOOKEEPER-3506 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3506 > Project: ZooKeeper > Issue Type: Improvement > Components: documentation, server >Reporter: maoling >Assignee: maoling >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 40m > Remaining Estimate: 0h > > > {code:java} > /** > * Generates an initial sessionId. High order byte is serverId, next 5 > * 5 bytes are from timestamp, and low order 2 bytes are 0s. > */ > public static long initializeNextSession(long id) { > long nextSid; > nextSid = (Time.currentElapsedTime() << 24) >>> 8; > nextSid = nextSid | (id <<56); > if (nextSid == EphemeralType.CONTAINER_EPHEMERAL_OWNER) { > ++nextSid; // this is an unlikely edge case, but check it just in > case > } > return nextSid; > } > {code} > > -- This message was sent by Atlassian Jira (v8.3.4#803005)