This is an automated email from the ASF dual-hosted git repository.
enixon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zookeeper.git
The following commit(s) were added to refs/heads/master by this push:
new ef3649f ZOOKEEPER-3593: fix the default value of jute.maxbuffer in
client side and an optimization for the documentation
ef3649f is described below
commit ef3649f599b4213a918c7a84cfe663d6cb1fa26e
Author: maoling <[email protected]>
AuthorDate: Sun Nov 3 17:01:42 2019 -0800
ZOOKEEPER-3593: fix the default value of jute.maxbuffer in client side and
an optimization for the documentation
- The default value of `jute.maxbuffer` in client side said it's 4MB, but
actually, it was never working, because when the client reads the deserialized
znode data, it also calls `checkLength(BinaryInputArchive.java:127)` where
`jute.maxbuffer` default value is 1MB. It's easy to reproduce, just read a
znode more than 1MB with any special configure client. Look at the stack trace
I attached in the JIRA
- Users also confused about that the doc said `jute.maxbuffer` must be set
on all servers and clients, but their default value is not same in the
[ZOOKEEPER-1295](https://issues.apache.org/jira/browse/ZOOKEEPER-1295)
- more details in the
[ZOOKEEPER-3593](https://issues.apache.org/jira/browse/ZOOKEEPER-3593)
Author: maoling <[email protected]>
Reviewers: [email protected], [email protected]
Closes #1129 from maoling/ZOOKEEPER-3593
---
.../src/main/resources/markdown/zookeeperAdmin.md | 24 ++++++++++++++--------
.../resources/markdown/zookeeperProgrammers.md | 7 ++++---
.../apache/zookeeper/client/ZKClientConfig.java | 2 +-
3 files changed, 21 insertions(+), 12 deletions(-)
diff --git a/zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md
b/zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md
index b0f08fe..66da6ca 100644
--- a/zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md
+++ b/zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md
@@ -1384,15 +1384,23 @@ the variable does.
set to no, ZooKeeper will not require updates to be synced to
the media.
-* *jute.maxbuffer:* :
- (Java system property:**jute.maxbuffer**)
- This option can only be set as a Java system property.
+* *jute.maxbuffer* :
+ (Java system property:**jute.maxbuffer**).
+ - This option can only be set as a Java system property.
There is no zookeeper prefix on it. It specifies the maximum
- size of the data that can be stored in a znode. The default is
- 0xfffff, or just under 1M. If this option is changed, the system
- property must be set on all servers and clients otherwise
- problems will arise. This is really a sanity check. ZooKeeper is
- designed to store data on the order of kilobytes in size.
+ size of the data that can be stored in a znode. The unit is: byte. The
default is
+ 0xfffff(1048575) bytes, or just under 1M.
+ - If this option is changed, the system property must be set on all
servers and clients otherwise
+ problems will arise.
+ - When *jute.maxbuffer* in the client side is greater than the server
side, the client wants to write the data
+ exceeds *jute.maxbuffer* in the server side, the server side will get
**java.io.IOException: Len error**
+ - When *jute.maxbuffer* in the client side is less than the server side,
the client wants to read the data
+ exceeds *jute.maxbuffer* in the client side, the client side will get
**java.io.IOException: Unreasonable length**
+ or **Packet len is out of range!**
+ - This is really a sanity check. ZooKeeper is designed to store data on
the order of kilobytes in size.
+ In the production environment, increasing this property to exceed the
default value is not recommended for the following reasons:
+ - Large size znodes cause unwarranted latency spikes, worsen the
throughput
+ - Large size znodes make the synchronization time between leader and
followers unpredictable and non-convergent(sometimes timeout), cause the quorum
unstable
* *jute.maxbuffer.extrasize*:
(Java system property: **zookeeper.jute.maxbuffer.extrasize**)
diff --git a/zookeeper-docs/src/main/resources/markdown/zookeeperProgrammers.md
b/zookeeper-docs/src/main/resources/markdown/zookeeperProgrammers.md
index 26e4aa5..03e83c0 100644
--- a/zookeeper-docs/src/main/resources/markdown/zookeeperProgrammers.md
+++ b/zookeeper-docs/src/main/resources/markdown/zookeeperProgrammers.md
@@ -1305,10 +1305,11 @@ following reference
and the password to unlock the file.
* *jute.maxbuffer* :
- It specifies the maximum size of the incoming data from the server. The
default value is 4194304
- Bytes , or just 4 MB. This is really a sanity check. The ZooKeeper server
is designed to store and send
+ In the client side, it specifies the maximum size of the incoming data
from the server. The default is 0xfffff(1048575) bytes,
+ or just under 1M. This is really a sanity check. The ZooKeeper server is
designed to store and send
data on the order of kilobytes. If incoming data length is more than this
value, an IOException
- is raised.
+ is raised. This value of client side should keep same with the server
side(Setting **System.setProperty("jute.maxbuffer", "xxxx")** in the client
side will work),
+ otherwise problems will arise.
* *zookeeper.kinit* :
Specifies path to kinit binary. Default is "/usr/bin/kinit".
diff --git
a/zookeeper-server/src/main/java/org/apache/zookeeper/client/ZKClientConfig.java
b/zookeeper-server/src/main/java/org/apache/zookeeper/client/ZKClientConfig.java
index 3ba4c6a..3012a35 100644
---
a/zookeeper-server/src/main/java/org/apache/zookeeper/client/ZKClientConfig.java
+++
b/zookeeper-server/src/main/java/org/apache/zookeeper/client/ZKClientConfig.java
@@ -57,7 +57,7 @@ public class ZKClientConfig extends ZKConfig {
*/
@SuppressWarnings("deprecation")
public static final String SECURE_CLIENT = ZooKeeper.SECURE_CLIENT;
- public static final int CLIENT_MAX_PACKET_LENGTH_DEFAULT = 4096 * 1024; /*
4 MB */
+ public static final int CLIENT_MAX_PACKET_LENGTH_DEFAULT = 0xfffff; /* 1
MB */
public static final String ZOOKEEPER_REQUEST_TIMEOUT =
"zookeeper.request.timeout";
public static final String ZOOKEEPER_SERVER_PRINCIPAL =
"zookeeper.server.principal";
/**
