Ah great. I did not realize LeaderLatch supported callbacks. That's should be sufficient then.
Thanks, Lenni On Wed, Sep 2, 2015 at 3:18 PM, Sravya Tirukkovalur <[email protected]> wrote: > Thanks for your response Lenni! > > Yes LeaderSelector is more flexible, but LeaderLatch also supports call > backs on leadership changes through LeaderLatchListeners. So unless we need > more flexibility, I think we can just use LeaderLatch. > > https://curator.apache.org/apidocs/org/apache/curator/framework/recipes/leader/LeaderLatchListener.html > > I will work on the design doc shortly and open a jira. > > Regards, > > > On Mon, Aug 31, 2015 at 11:29 PM, Lenni Kuff <[email protected]> wrote: > > > Good catch Sravya - these do sound like significant problems. Great job > > thinking through the design, I'll need to think through it a bit more but > > at a high level it makes sense. It might be good to put together a small > > design doc on this to review and so we all understand the protocol and > > failures points. > > A more specific point on the HMS leader election implementation - I think > > you need to use LeaderSelector rather than LeaderLatch so HMS plugin can > > get notified (via callback) when the leader changes. I don't think that's > > possible with the LeaderLatch. Are there any JIRAs I can follow to > > track/review this work? > > > > Thanks, > > Lenni > > > > On Fri, Aug 28, 2015 at 8:15 PM, Sravya Tirukkovalur < > [email protected]> > > wrote: > > > > > Hi fellow developer, > > > > > > Looks like there are some problems with current design when Sentry is > in > > HA > > > / HMS is in HA. Here are some of the problems I have identified some > > > problems and would like to propose some solutions. Please let me know > > what > > > you think. > > > > > > Problem 1: Zookeeper might blow up if HMS meta data is too big > > > See https://cwiki.apache.org/confluence/display/CURATOR/TN4 > > > > > > Problem 2: Both HMSs send full updates to sentry > > > There is a chance that these two full updates might actually be > > different. > > > This is true if there are some meta data operations while the full > update > > > is being built on one server. > > > > > > Proposed design: > > > For HMS HA: > > > We will pick a leader using curator's Leader latch and only this HMS > > would > > > be responsible for sending the path updates to Sentry > > > For the propagation of path updates from the follower, we will use > > > PathChildrenCacheListener recipe of curator, where the follower can > post > > > the updates it sees to ZK path. And the leader listens in this path, > and > > > processes these updates and sends to Sentry. > > > > > > For Sentry HA: > > > - Leader sends the path updates to both the sentry servers. > > > - And for permission updates, sentry servers use zookeeper similar to > HMS > > > to propagate updates to each other. > > > > > > Regards, > > > -- > > > Sravya Tirukkovalur > > > > > > > > > -- > Sravya Tirukkovalur >
