TaskManagement Using Zookeeper Recipe
Issue Type: Task
Reporter: kishore gopalakrishna
A typical use case in distributed system is " There are T tasks and P processes
running but only T processes must be active always [ P > T ] and remaining P-T
processes acting as stand by and be ready to take up a Task with one or more
active processes fail".
Zookeeper provides an excellent service which can be used to co ordinate among
P processes and using the mechanism of locking we can ensure that there is
always T processes active. Without a central co-ordinating service generally
there will be 2T processes[ i.e atleast one back up for each process]. With
Zookeeper we can decide P based on the failure rate.
The assumption here are
1. At any time we have P > T. P can be chosen appropriately based on failure
2. The tasks are stateless. That is any process P_i that takes up a task T_j
does not know the state of the process P_k which previously processed T_j.
This is not entirely true and there are ways to over come this draw back on a
case by case basis.
This was developed for a different project S4 which is also open sourced
http://s4.io/. The communication layer and task management layer is completely
independent of S4 and can be used in any application.
This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.