Check out Apache Helix. Provides pretty much everything you need. The basic idea is that workers have a life cycle defined in terms of a state machine and there are cluster wide constraints allowed on how many workers can be in different states. There are also resources which are assigned to workers according to flexible preferences.
What these states are, what they mean and what the resources are remains comfortably abstract. The actual coordination is done using ZK, which is a good thing. For anything more than a single worker, you have to have some reliable handling of partition and resource assignment anyway, and there are few options other than implementing yet another paxos engine or using Zookeeper. I would veto the first as a huge waste of time so we are pretty much left with the Zookeeper option. On Mar 18, 2013, at 2:42 AM, David Alves (JIRA) wrote: > David Alves created DRILL-53: > -------------------------------- > > Summary: Setup cluster configuration and membership mgmt system > Key: DRILL-53 > URL: https://issues.apache.org/jira/browse/DRILL-53 > Project: Apache Drill > Issue Type: New Feature > Reporter: David Alves > > > Several configuration entries need to be managed across the cluster, namely > metastore (hive?) location and drill daemon addresses. > > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA administrators > For more information on JIRA, see: http://www.atlassian.com/software/jira
