I am working on this right now , can’t promise the dates because I have some other tasks as well. The general approach is already described in the JIRA. You basically need 3 things:
Leader election between all nimbus hosts. (zookeeper is what I am going with), checks on nimbus background threads and state changing APIS to ensure only leader is changing states. Some way to distribute the local nimbus data directory to all the other nimbus and keep it consistent. Simplest way is to not have a local directory but instead use something like HDFS, if you can take that kind of dependency. Nimbus discovery, could be done either by adding/modifying current rest/thrift APIS or by allowing supervisors or clients to read from zookeeper. I have some code on leader election part with preliminary testing and unit-test here. This is by no means ready for anyone’s use but if you want to take a look to get an idea you can take a look. Thanks Parth On Sep 16, 2014, at 6:04 AM, M.Tarkeshwar Rao <tarkeshwa...@gmail.com> wrote: > something -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
