[
https://issues.apache.org/jira/browse/ZOOKEEPER-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057068#comment-17057068
]
H Lu commented on ZOOKEEPER-3593:
---------------------------------
[~maoling] Thanks for the reply and explanation.
* I understand your point that 1 MB packet length would be good for a
reasonable number of children depending on the child name's length. It is
enough for most cases when ZK is appropriately used.
* In your case, the znode length 1073590 bytes = 1048.4 KB > 1 MB. Reading is
blocked on server side:
[code|[https://github.com/apache/zookeeper/blob/b4c89dc7f6083829e18fae6e446907ae0b1f22d7/zookeeper-jute/src/main/java/org/apache/jute/BinaryInputArchive.java#L166].]
because jute.maxbuffer on server side is 1 MB by default. It is a different
value from the client side in ClientCnxnSocket. It means, the failure you get
is caused by server side' default maxbuffer, not the client side's. And the log
below is actually accurate. It has nothing to do with your reading failure.
2019-10-27 18:08:41 INFO ClientCnxnSocket:237 - jute.maxbuffer value is 4194304
Bytes
If you set jute.maxbuffer = 1.5MB on ZK server, I think you can read the data.
* The ticket here is to address the reading failure issue which happens on
server. And it is expected as data length exceeds 1 MB. And the log is
accurate. And changing the default packet length breaks backward-compatibility.
If there is a need to change the default packet length for optimization reason,
I think that is more reasonable and should be in a separate ticket.
* So I don't think changing the ClientCnxnSocket's default packet length would
solve this issue in this ticket.
These are just my personal opinions. Hope you can consider. Thanks!
> fix the default value of jute.maxbuffer in client side and an optimization
> for the documentation
> ------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-3593
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3593
> Project: ZooKeeper
> Issue Type: Improvement
> Affects Versions: 3.6.0, 3.5.5, 3.5.6
> Reporter: maoling
> Assignee: maoling
> Priority: Minor
> Labels: pull-request-available
> Fix For: 3.6.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
>
> {code:java}
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client
> environment:java.io.tmpdir=/var/folders/kj/092gpj_s2hvdgx77c9ghqdv00000gp/T/
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client environment:java.compiler=<NA>
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client environment:os.name=Mac OS X
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client environment:os.arch=x86_64
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client environment:os.version=10.13.6
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client environment:user.name=wenba
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client
> environment:user.home=/Users/wenba
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client
> environment:user.dir=/Users/wenba/workspaces/workspace_learning/YCSB
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client
> environment:os.memory.free=234MB
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client
> environment:os.memory.max=3641MB
> 2019-10-27 18:08:41 INFO ZooKeeper:109 - Client
> environment:os.memory.total=245MB
> 2019-10-27 18:08:41 INFO ZooKeeper:868 - Initiating client connection,
> connectString=127.0.0.1:2180/bm sessionTimeout=30000
> watcher=site.ycsb.db.zookeeper.ZKClient$SimpleWatcher@4b506000
> 2019-10-27 18:08:41 INFO X509Util:79 - Setting -D
> jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated
> TLS renegotiation
> 2019-10-27 18:08:41 INFO ClientCnxnSocket:237 - jute.maxbuffer value is
> 4194304 Bytes
> FAILED: Count=6, Max=106559, Min=99392, Avg=103157.33, 90=105791, 99=106559,
> 99.9=106559, 99.99=106559]FAILED: Count=6, Max=106559, Min=99392,
> Avg=103157.33, 90=105791, 99=106559, 99.9=106559, 99.99=106559]2019-10-27
> 18:08:51 INFO ClientCnxn:1112 - Opening socket connection to server
> localhost/127.0.0.1:2180. Will not attempt to authenticate using SASL
> (unknown error)2019-10-27 18:08:51 INFO ClientCnxn:959 - Socket connection
> established, initiating session, client: /127.0.0.1:51795, server:
> localhost/127.0.0.1:21802019-10-27 18:08:51 INFO ClientCnxn:1394 - Session
> establishment complete on server localhost/127.0.0.1:2180, sessionid =
> 0x1000c6e0658000a, negotiated timeout = 300002019-10-27 18:08:51 WARN
> ClientCnxn:1246 - Session 0x1000c6e0658000a for server
> localhost/127.0.0.1:2180, unexpected error, closing socket connection and
> attempting reconnectjava.io.IOException: Unreasonable length = 1073590 at
> org.apache.jute.BinaryInputArchive.checkLength(BinaryInputArchive.java:127)
> at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:92)
> at
> org.apache.zookeeper.proto.GetDataResponse.deserialize(GetDataResponse.java:56)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:919)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:101)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:363)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223)2019-10-27
> 18:08:51 ERROR ZKClient:128 - Error when reading a
> path:/user5344789772525948776,tableName:usertableorg.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /user5344789772525948776
> {code}
>
>
> From the log I saw *jute.maxbuffer* in my client side is 4194304 Bytes, but
> I cannot read a znode which is almost 1MB
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
