Re: Maximum number of children

2009-01-12 Thread Mahadev Konar
I was going to suggest bucketing with predifined hashes. /root/template/data/hashbucket/hash For the issue raised by Joshua regarding the length of the output from the server -- This is a bug. We seem to allow any number of children (< int) of a node and the getchildren call fails to return the c

RE: Maximum number of children

2009-01-12 Thread Stu Hood
To continue with your current design, you could create a trie based on shared hash prefixes. /root/template/date/ 1a5e67/2b45dc /root/template/date/ 1a5e67/3d4a1f /root/template/date/ 3d4a1f/1a5e67 /root/template/date/ 3d4a1f/2b45dc Alternatively, you could use what the maildir mail storage form

Maximum number of children

2009-01-12 Thread Joshua Tuberville
Hello, We are attempting to use ZooKeeper to coordinate daily email thresholds. To do this we created a node hierarchy of /root/template/date/email_hash The idea being that we only send the template to an email address once per day. This is intended to support millions of email hashes per da

RE: Distributed queue: how to ensure no lost items?

2009-01-12 Thread Benjamin Reed
That is a good point. you could put a child znode of queue-X that contains the processing history. Like who tried to process and what time they started. ben From: Hiram Chirino [chir...@gmail.com] Sent: Monday, January 12, 2009 8:48 AM To: zookeeper-user@

Re: Distributed queue: how to ensure no lost items?

2009-01-12 Thread Hiram Chirino
At least once is generally the case in queuing systems unless you can do a distributed transaction with your consumer. What comes in handy in an at least once case, is letting the consumer know that a message may have 'potentially' already been processed. That way he can double check first before