We are attempting to use ZooKeeper to coordinate daily email thresholds.  To do 
this we created a node hierarchy of


The idea being that we only send the template to an email address once per day. 
 This is intended to support millions of email hashes per day. From the 
ZooKeeper perspective we just attempt a create and if it succeeds we proceed 
and if we get a node exists exception we stop processing.  This seems to 
operate fine for over 2 million email hashes so far in testing.  However we 
also want to prune all previous days nodes to conserve memory.  We have run 
into a hard limit while using the getChildren method for a given 
/root/template/date.  If the List of children exceeds the hardcoded 4,194,304 
byte limit ClientCnxn$SendThread.readLength() throws an exception on line 490.  
So we have an issue that we can not delete a node that has children nor is it 
possible to delete a node who has children whose total names exceed 4 Mb.  

Any feedback or guidance is appreciated.

Joshua Tuberville

Reply via email to