Hi Pratyush, Node size (more precisely jute-maxbuffer size) has quite a debate. Currently it is 1MB by default. I've seen it used in production as high as 1GB! That is insane, and definitely not recommended, but if you up it to a few MB it's fine IMHO. But these high numbers are mainly due to getChildren() getting too big (Kafka and Hive have a tendency to run into this). They usually still don't store big znodes. And that's a good thing, since ZK is not an in-memory database.
As for how many znodes, again, depends on a few things (probably a lot actually) and I've seen surprisingly high numbers here also. But there were many observers used, and also it was tweaked to the setup they used. Basically it depends on the hardware ZK is running on. First and foremost you need to have sufficient memory. And of course disc and write capacity to write snapshots and txn logs. A few thousands nodes are no biggie, but if we're talking millions, that's probably a little too much. So to sum it up, no exact numbers, but it's worth re-thinking your use-case, if any of these numbers are abused. Regards, Norbert On Fri, May 29, 2020 at 3:11 AM Pratyush Tarar <pratyush.ta...@gmail.com> wrote: > Hello > > As understand the key-data combination stored in zookeeper nodes should not > be too large (say < 1 MB). > > But is there a limit to the number of key-data combinations (each say < 1 > MB size) that can be stored in zookeeper node? > > Response appreciated. > > Pratyush >
