Re: Using zookeeper to assign a bunch of long-running tasks to nodes (without unhandled tasks and double-handled tasks)

2010-01-24 Thread Zheng Shao
the machines can update this count and alert (to the admin) if a task cannot be started or worked upon by a given count of machines. Hope this helps. Thanks Mahadev On 1/23/10 12:58 AM, Zheng Shao zsh...@gmail.com wrote: Let's say I have 100 long-running tasks and 20 nodes. I want each

Using zookeeper to assign a bunch of long-running tasks to nodes (without unhandled tasks and double-handled tasks)

2010-01-23 Thread Zheng Shao
Let's say I have 100 long-running tasks and 20 nodes. I want each of them to take up to 10 tasks. Each of the task should be taken by one and only one node. Will the following solution solve the problem? Create a directory /mytasks in zookeeper. Normally there will be 100 EPHEMERAL children in