Hi Team,
According to the ZooKeeper documentation (
https://zookeeper.apache.org/doc/r3.6.4/zookeeperAdmin.html), the default
*jute.maxbuffer* in ZooKeeper is 1,048,575 bytes for both the server and
client sides. However, creating a znode with a size close to this limit can
cause issues.
For example, a znode (test_node) with a size of 1,048,494 bytes can be
created on the ZooKeeper server without any problems. But you will
encounter an exception if you try to retrieve this znode using the
ZooKeeper CLI or java client. We saw this issue across four different
Zookeeper versions: 3.6.4, 3.7.2, 3.8.4, and 3.9.2.
[zk: 127.0.0.1:2181(CONNECTED) 0] stat /test_node
cZxid = 0x10
ctime = Mon Jul 22 04:45:24 UTC 2024
mZxid = 0x10
mtime = Mon Jul 22 04:45:24 UTC 2024
pZxid = 0x10
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = *1048494*
numChildren = 0
[zk: 127.0.0.1:2181(CONNECTED) 0] get /test_node
2024-07-22 04:45:25,628 [myid:127.0.0.1:2181] - WARN
[main-SendThread(127.0.0.1:2181):o.a.z.ClientCnxn$SendThread@1285] -
Session 0x100114673830008 for server localhost/127.0.0.1:2181, Closing
socket connection. Attempting reconnect except it is a
SessionExpiredException.
java.io.IOException: Packet len *1048582* is out of range!
at
org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:124)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:84)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1274)
It appears that when a client retrieves a znode from the ZooKeeper server,
the size of the znode received by the client is larger than the actual
size. This discrepancy can prevent the client from connecting to the
ZooKeeper server.
To avoid this issue, should we set the *jute.maxbuffer* on the server side,
slightly lower than the setting on the client side, contrary to the
documentation's recommendation to have the same setting on both sides? This
adjustment might be necessary because the data received by the client is
consistently larger than the data stored on the server.
Thanks,
Yifei Zhang
