[jira] [Created] (ZOOKEEPER-3536) On Windows maven build generates corrupted tarball
Mohammad Arshad created ZOOKEEPER-3536:
--
Summary: On Windows maven build generates corrupted tarball
Key: ZOOKEEPER-3536
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3536
Project: ZooKeeper
Issue Type: Bug
Components: build
Affects Versions: 3.5.5
Reporter: Mohammad Arshad
On windows maven command {code}mvn clean install -DskipTests{code} creates
corrupted tarballs.
In zookeeper-assembly/pom.xml posix causing
the problem. Many use Windows as development environment. it would be better
if we can make tarLongFileMode property configurable or select based on OS.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (ZOOKEEPER-3535) add a new property:initial-cluster-token to identify a ensemble
maoling created ZOOKEEPER-3535: -- Summary: add a new property:initial-cluster-token to identify a ensemble Key: ZOOKEEPER-3535 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3535 Project: ZooKeeper Issue Type: New Feature Components: documentation, server Reporter: maoling Assignee: maoling Fix For: 3.6.0 Every new zk cluster generates a new cluster ID based on the initial cluster configuration and a user-provided unique initial-cluster-token value. By having unique cluster ID’s, zk is protected from cross-cluster interaction which could corrupt the cluster. Usually this warning happens after tearing down an old cluster, then reusing some of the peer addresses for the new cluster. If any zk process from the old cluster is still running it will try to contact the new cluster. The new cluster will recognize a cluster ID mismatch, then ignore the request and emit this warning. This warning is often cleared by ensuring peer addresses among distinct clusters are disjoint. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (ZOOKEEPER-3522) Consistency guarantees discussion.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923851#comment-16923851 ] Hudson commented on ZOOKEEPER-3522: --- SUCCESS: Integrated in Jenkins build ZooKeeper-trunk #681 (See [https://builds.apache.org/job/ZooKeeper-trunk/681/]) ZOOKEEPER-3522: Reword Zookeeper consistency doc for clarity (hanm: rev 614224e318551a4ba332618cbd377050f19e1f3f) * (edit) zookeeper-docs/src/main/resources/markdown/zookeeperInternals.md > Consistency guarantees discussion. > -- > > Key: ZOOKEEPER-3522 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3522 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Karolos Antoniadis >Assignee: Karolos Antoniadis >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > > It seems there is some confusion on the exact consistency guarantees of > ZooKeeper. > Goal of this issue is to add a few paragraphs on the subject in [ZooKeeper > internals|[https://zookeeper.apache.org/doc/r3.5.5/zookeeperInternals.html]]. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (ZOOKEEPER-3522) Consistency guarantees discussion.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923806#comment-16923806 ] Hudson commented on ZOOKEEPER-3522: --- SUCCESS: Integrated in Jenkins build Zookeeper-trunk-single-thread #516 (See [https://builds.apache.org/job/Zookeeper-trunk-single-thread/516/]) ZOOKEEPER-3522: Reword Zookeeper consistency doc for clarity (hanm: rev 614224e318551a4ba332618cbd377050f19e1f3f) * (edit) zookeeper-docs/src/main/resources/markdown/zookeeperInternals.md > Consistency guarantees discussion. > -- > > Key: ZOOKEEPER-3522 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3522 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Karolos Antoniadis >Assignee: Karolos Antoniadis >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > > It seems there is some confusion on the exact consistency guarantees of > ZooKeeper. > Goal of this issue is to add a few paragraphs on the subject in [ZooKeeper > internals|[https://zookeeper.apache.org/doc/r3.5.5/zookeeperInternals.html]]. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (ZOOKEEPER-3528) Revisit AsyncCallback javadoc
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923804#comment-16923804 ] Hudson commented on ZOOKEEPER-3528: --- SUCCESS: Integrated in Jenkins build Zookeeper-trunk-single-thread #516 (See [https://builds.apache.org/job/Zookeeper-trunk-single-thread/516/]) ZOOKEEPER-3528: Revisit AsyncCallback javadoc (hanm: rev 47385c75e0e0d0a48e71122df989effd55f0fe59) * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/AsyncCallback.java > Revisit AsyncCallback javadoc > - > > Key: ZOOKEEPER-3528 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3528 > Project: ZooKeeper > Issue Type: Sub-task > Components: documentation >Affects Versions: 3.6.0 >Reporter: TisonKun >Assignee: TisonKun >Priority: Major > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (ZOOKEEPER-3492) Add weights to server side connection throttling
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923805#comment-16923805 ] Hudson commented on ZOOKEEPER-3492: --- SUCCESS: Integrated in Jenkins build Zookeeper-trunk-single-thread #516 (See [https://builds.apache.org/job/Zookeeper-trunk-single-thread/516/]) ZOOKEEPER-3492: Add weights to server side connection throttling (hanm: rev eecd9e7ce046083bd40cd6134bdb2b405d01fe67) * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/UpgradeableSessionTracker.java * (edit) zookeeper-server/src/test/java/org/apache/zookeeper/server/PrepRequestProcessorTest.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/SessionTrackerImpl.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/LeaderSessionTracker.java * (edit) zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md * (edit) zookeeper-server/src/test/java/org/apache/zookeeper/server/BlueThrottleTest.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/SessionTracker.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/BlueThrottle.java > Add weights to server side connection throttling > > > Key: ZOOKEEPER-3492 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3492 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Jie Huang >Assignee: Jie Huang >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > In ZOOKEEPER-3242, we introduced connection throttling to protect the server > from being overloaded. We realize that the costs for creating a local > session, creating a global session, and reconnecting are different. So we > associate weights to the costs when throttling. For example, for the same > setting, the throttler will allow more connections to be created if they are > local. This allows the server resources to be fully utilized. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (ZOOKEEPER-3491) Specify commitLogCount value using a system property
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ZOOKEEPER-3491: -- Labels: pull-request-available (was: ) > Specify commitLogCount value using a system property > > > Key: ZOOKEEPER-3491 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3491 > Project: ZooKeeper > Issue Type: Improvement > Components: quorum, server >Affects Versions: 3.6.0 >Reporter: Vladimir Ivić >Assignee: Vladimir Ivić >Priority: Major > Labels: pull-request-available > > Currently the commit log count value is set to 500. This can cause busy > servers to snapshot transactions too often. > Override default commitLogCount=500 through the system property > zookeeper.commitLogCount. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (ZOOKEEPER-3492) Add weights to server side connection throttling
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923755#comment-16923755 ] Hudson commented on ZOOKEEPER-3492: --- SUCCESS: Integrated in Jenkins build ZooKeeper-trunk #680 (See [https://builds.apache.org/job/ZooKeeper-trunk/680/]) ZOOKEEPER-3492: Add weights to server side connection throttling (hanm: rev eecd9e7ce046083bd40cd6134bdb2b405d01fe67) * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/SessionTrackerImpl.java * (edit) zookeeper-server/src/test/java/org/apache/zookeeper/server/PrepRequestProcessorTest.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/BlueThrottle.java * (edit) zookeeper-server/src/test/java/org/apache/zookeeper/server/BlueThrottleTest.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/SessionTracker.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/UpgradeableSessionTracker.java * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/LeaderSessionTracker.java * (edit) zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md > Add weights to server side connection throttling > > > Key: ZOOKEEPER-3492 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3492 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Jie Huang >Assignee: Jie Huang >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > In ZOOKEEPER-3242, we introduced connection throttling to protect the server > from being overloaded. We realize that the costs for creating a local > session, creating a global session, and reconnecting are different. So we > associate weights to the costs when throttling. For example, for the same > setting, the throttler will allow more connections to be created if they are > local. This allows the server resources to be fully utilized. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (ZOOKEEPER-3528) Revisit AsyncCallback javadoc
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923754#comment-16923754 ] Hudson commented on ZOOKEEPER-3528: --- SUCCESS: Integrated in Jenkins build ZooKeeper-trunk #680 (See [https://builds.apache.org/job/ZooKeeper-trunk/680/]) ZOOKEEPER-3528: Revisit AsyncCallback javadoc (hanm: rev 47385c75e0e0d0a48e71122df989effd55f0fe59) * (edit) zookeeper-server/src/main/java/org/apache/zookeeper/AsyncCallback.java > Revisit AsyncCallback javadoc > - > > Key: ZOOKEEPER-3528 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3528 > Project: ZooKeeper > Issue Type: Sub-task > Components: documentation >Affects Versions: 3.6.0 >Reporter: TisonKun >Assignee: TisonKun >Priority: Major > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (ZOOKEEPER-3492) Add weights to server side connection throttling
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Han reassigned ZOOKEEPER-3492: -- Assignee: Jie Huang > Add weights to server side connection throttling > > > Key: ZOOKEEPER-3492 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3492 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Jie Huang >Assignee: Jie Huang >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > In ZOOKEEPER-3242, we introduced connection throttling to protect the server > from being overloaded. We realize that the costs for creating a local > session, creating a global session, and reconnecting are different. So we > associate weights to the costs when throttling. For example, for the same > setting, the throttler will allow more connections to be created if they are > local. This allows the server resources to be fully utilized. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (ZOOKEEPER-3492) Add weights to server side connection throttling
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Han resolved ZOOKEEPER-3492. Resolution: Fixed Issue resolved by pull request 1037 [https://github.com/apache/zookeeper/pull/1037] > Add weights to server side connection throttling > > > Key: ZOOKEEPER-3492 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3492 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Jie Huang >Priority: Minor > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > In ZOOKEEPER-3242, we introduced connection throttling to protect the server > from being overloaded. We realize that the costs for creating a local > session, creating a global session, and reconnecting are different. So we > associate weights to the costs when throttling. For example, for the same > setting, the throttler will allow more connections to be created if they are > local. This allows the server resources to be fully utilized. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (ZOOKEEPER-3528) Revisit AsyncCallback javadoc
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Han resolved ZOOKEEPER-3528. Resolution: Fixed Issue resolved by pull request 1070 [https://github.com/apache/zookeeper/pull/1070] > Revisit AsyncCallback javadoc > - > > Key: ZOOKEEPER-3528 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3528 > Project: ZooKeeper > Issue Type: Sub-task > Components: documentation >Affects Versions: 3.6.0 >Reporter: TisonKun >Assignee: TisonKun >Priority: Major > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (ZOOKEEPER-3496) Transaction larger than jute.maxbuffer makes ZooKeeper unavailable
[
https://issues.apache.org/jira/browse/ZOOKEEPER-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923151#comment-16923151
]
Mohammad Arshad commented on ZOOKEEPER-3496:
PR Created
> Transaction larger than jute.maxbuffer makes ZooKeeper unavailable
> --
>
> Key: ZOOKEEPER-3496
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3496
> Project: ZooKeeper
> Issue Type: Bug
>Affects Versions: 3.5.5, 3.4.14
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.6.0, 3.4.15, 3.5.6
>
> Attachments: ZOOKEEPER-3496-001.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *Problem:*
> ZooKeeper server fails to start, logs following error
> {code:java}
> Exception in thread "main" java.io.IOException: Unreasonable length = 1001025
> at
> org.apache.jute.BinaryInputArchive.checkLength(BinaryInputArchive.java:127)
> at
> org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:92)
> {code}
> This indicates that one of the transactions size is more than the configured
> jute.maxbuffer values. But how transaction more than jute.maxbuffer size is
> allowed to write?
> *Analysis:*
> At ZooKeeper server jute.maxbuffer specifies the maximum size of a
> transaction. By default it is 1 MB at the server
> jute.maxbuffer is used for following:
> # Size sanity check of incoming request. Incoming requests size must not be
> more than jute.maxbuffer
> # Size sanity check of the transaction while reading from transaction or
> snapshot file. Transaction size must not be more than jute.maxbuffer+1024
> # Size sanity check of transaction while reading data from the leader.
> Transaction size must not be more than jute.maxbuffer+1024
> Request size sanity check is done in the beginning of a request processing
> but later request processing adds additional information into request then
> writes to transaction file. This additional information size is not
> considered in sanity check. This is how transaction larger than
> jute.maxbuffer are accepted into ZooKeeper.
> If this additional information size is less than 1024 Bytes then it is OK as
> ZooKeeper already takes care of it.
> But if this additional information size is more than 1024 bytes it allows the
> request, But while reading from transaction/snapshot file and while reading
> from leader it fails and make the ZooKeeper service unavailable
> +Example:+
> Suppose incoming request size is 100 Bytes
> Configured jute.maxbuffer is 100
> After processing the request ZooKeeper server adds 1025 more bytes
> In this case, request will be processed successfully, and 100+1025 bytes
> will be written to transaction file
> But while reading from the transaction log 100+1025 bytes cannot be read
> as max allowed length is 100(effectively 100+1024).
> *Solutions:*
> If incoming request size sanity check is done after populating all additional
> information then this problem is solved. But doing sanity check in the later
> stage of request processing will defeat the purpose of sanity check itself.
> So this we can not do
> Currently additional information size is constant 1024 Bytes [Code
> Reference|https://github.com/apache/zookeeper/blob/branch-3.5/zookeeper-jute/src/main/java/org/apache/jute/BinaryInputArchive.java#L126].
> We should increase this value and make it more reasonable. I propose to make
> this additional information size to same as the jute.maxbuffer. Also make
> additional information size configurable.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (ZOOKEEPER-3496) Transaction larger than jute.maxbuffer makes ZooKeeper unavailable
[
https://issues.apache.org/jira/browse/ZOOKEEPER-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ZOOKEEPER-3496:
--
Labels: pull-request-available (was: )
> Transaction larger than jute.maxbuffer makes ZooKeeper unavailable
> --
>
> Key: ZOOKEEPER-3496
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3496
> Project: ZooKeeper
> Issue Type: Bug
>Affects Versions: 3.5.5, 3.4.14
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.6.0, 3.4.15, 3.5.6
>
> Attachments: ZOOKEEPER-3496-001.patch
>
>
> *Problem:*
> ZooKeeper server fails to start, logs following error
> {code:java}
> Exception in thread "main" java.io.IOException: Unreasonable length = 1001025
> at
> org.apache.jute.BinaryInputArchive.checkLength(BinaryInputArchive.java:127)
> at
> org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:92)
> {code}
> This indicates that one of the transactions size is more than the configured
> jute.maxbuffer values. But how transaction more than jute.maxbuffer size is
> allowed to write?
> *Analysis:*
> At ZooKeeper server jute.maxbuffer specifies the maximum size of a
> transaction. By default it is 1 MB at the server
> jute.maxbuffer is used for following:
> # Size sanity check of incoming request. Incoming requests size must not be
> more than jute.maxbuffer
> # Size sanity check of the transaction while reading from transaction or
> snapshot file. Transaction size must not be more than jute.maxbuffer+1024
> # Size sanity check of transaction while reading data from the leader.
> Transaction size must not be more than jute.maxbuffer+1024
> Request size sanity check is done in the beginning of a request processing
> but later request processing adds additional information into request then
> writes to transaction file. This additional information size is not
> considered in sanity check. This is how transaction larger than
> jute.maxbuffer are accepted into ZooKeeper.
> If this additional information size is less than 1024 Bytes then it is OK as
> ZooKeeper already takes care of it.
> But if this additional information size is more than 1024 bytes it allows the
> request, But while reading from transaction/snapshot file and while reading
> from leader it fails and make the ZooKeeper service unavailable
> +Example:+
> Suppose incoming request size is 100 Bytes
> Configured jute.maxbuffer is 100
> After processing the request ZooKeeper server adds 1025 more bytes
> In this case, request will be processed successfully, and 100+1025 bytes
> will be written to transaction file
> But while reading from the transaction log 100+1025 bytes cannot be read
> as max allowed length is 100(effectively 100+1024).
> *Solutions:*
> If incoming request size sanity check is done after populating all additional
> information then this problem is solved. But doing sanity check in the later
> stage of request processing will defeat the purpose of sanity check itself.
> So this we can not do
> Currently additional information size is constant 1024 Bytes [Code
> Reference|https://github.com/apache/zookeeper/blob/branch-3.5/zookeeper-jute/src/main/java/org/apache/jute/BinaryInputArchive.java#L126].
> We should increase this value and make it more reasonable. I propose to make
> this additional information size to same as the jute.maxbuffer. Also make
> additional information size configurable.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
