I have a use case supporting multiple clients and multiple languages in a
So , In order to improve the language support, we want to leverage the Solr
dictionary (userdict.txt) files as large as 10MB.
I understand that ZooKeeper's default zNode file size limit is 1MB.
I'm not sure sure if someone tried increasing it before and how does that
fares in terms of performance.
Looking at - https://zookeeper.apache.org/doc/r3.2.2/zookeeperAdmin.html
It states -
The following options can be useful, but be careful when you use them. The
risk of each is explained along with the explanation of what the variable
(Java system property:* jute.maxbuffer*)
This option can only be set as a Java system property. There is no
zookeeper prefix on it. It specifies the maximum size of the data that can
be stored in a znode. The default is 0xfffff, or just under 1M. If this
option is changed, the system property must be set on all servers and
clients otherwise problems will arise. This is really a sanity check.
ZooKeeper is designed to store data on the order of kilobytes in size.
I would appreciate if someone has any suggestions on what are the best
practices for handling large config/dictionary files in ZK?