Todd Nine:
> [...]
> UC1: Synchronized Jobs
> 1. A job is fired across all nodes
> 2. The nodes wait until the barrier is entered by all participants
> 3. The nodes process the data and leave
> 4. On all nodes leaving the barrier, the Leader node marks the job as
> complete.
> UC2: Multiple Jobs per Node
> 1. A Job is scheduled for a future time on a specific node (usually the
> same node that's creating the trigger)
> 2. A Trigger can be overwritten and cancelled without the job firing
> 3. In the event of a node failure, the Leader will take all pending jobs
> from the failed node, and partition them across the remaining nodes.

Hi Todd,

we've implemented UC2 for an internal project with ZK. I'd love to make the 
code free, but I've to ask our product owner. It's a small company so this 
could go quickly. But I don't know how to convince them. They're so afraid of 
giving away stuff.
The basic idea is, that we've two "folders" in ZK, a work queue and a lock 
folder. The items (znodes) in the work queue a timestamp prefixed. Every node 
consuming the queue tries to create an ephemeral znode in the lock "folder" 
before starting on a work item. Work items are actually URLs and we lock on 
the domain. Since we also use a lock pool on every worker that only releases 
on overflow or timeout, we can reuse locks and also get "weak" locality for 
URLs of the same domain. - That's all the magic. Six java classes on top of 
our own ZK helper lib.

Best regards,

Thomas Koch,

Reply via email to