Hi, Zookeeper in current form stores all keys in memory and snapshots them periodically. I think it limits size of data Zookeeper can store.
I am investigating feasibility of patching ZK to support large number of keys. It would use off-heap storage engine with incremental snapshotting. I think this way Zookeeper could store around 100 million keys-value pairs without negative impact on performance. Technically it is feasible (I already done something similar for Hazelcast). My question is if someone would actually use this improvement. Current ZK is probably just fine for most uses, it only has problem when you put excessive amount of data inside. So my questions is: Do you use ZK as a database? And do you have problem with long crash recovery time? Thanks, Jan Kotek
