Hi, Currently we have bucketing as one of the options when the number of partitions are large. We have couple of bugs with the handling of bucketized resources (one of them is fatal).
One of the reasons to split the znode is because we use JSON to store the data in ZNode. While JSON is good for debugging, its space inefficient. A better option before going to bucketing is to support compression of Ideal state, current state and External View. This also gives good performance. I plan to add this support and make it configurable. Feedback/suggestions thanks, Kishore G
