Jason, correct me if I am wrong. We are already releasing the watches.the problem is with zookeeper client.
After a path is deleted, we cannot remove the watch. This is a zookeeper bug and is fixed in they latest version. Thanks Kishore G On Sep 23, 2014 11:03 AM, "Sandeep Nayak" <osgig...@gmail.com> wrote: > Why not start tracking all the watches created by extending the watch > with our own implementation which can have shutdown ability? That way > we can kill all watches when we find them stale. Is the concern that > there are too many of them? > > On Tue, Sep 23, 2014 at 10:49 AM, Zhen Zhang <nehzgn...@gmail.com> wrote: > > it will remove the dead watch associated with the controller. but not the > > dead watch associated the participant and spectator. > > > > > > On Tue, Sep 23, 2014 at 10:39 AM, kishore g <g.kish...@gmail.com> wrote: > > > >> Can we restart controller as a work around > >> On Sep 23, 2014 10:11 AM, "Zhen Zhang (JIRA)" <j...@apache.org> wrote: > >> > >> > > >> > [ > >> > > >> > https://issues.apache.org/jira/browse/HELIX-504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14145049#comment-14145049 > >> > ] > >> > > >> > Zhen Zhang commented on HELIX-504: > >> > ---------------------------------- > >> > > >> > On investigating zookeeper watch leakage problem, it turns out to be a > >> > zookeeper issue: > >> > https://issues.apache.org/jira/browse/ZOOKEEPER-442 > >> > > >> > For zookeeper before 3.5.0, we can't remove watches that are no > longer > >> of > >> > interests. The only way to remove a watch is to trigger it; that is, > if > >> it > >> > is a DataWatch, we need to trigger a data change on the watching > path, or > >> > if it is a ChildWatch, we need to trigger a child change on the > watching > >> > path. Unfortunately, if we are watching a path that has been deleted, > >> > unless we re-create the path, there is no way we can remove the watch. > >> > > >> > Here are some of the most common scenarios where we will have dead > >> > zookeeper watches on zookeeper server side even though we unregister > all > >> > the listeners on the zookeeper client side: > >> > > >> > - When we drop a resource group from a cluster, we may have dead > watches > >> > on ideal-state, participant current-state, and external-view > >> > - When we remove an instance from a cluster, we may have dead watches > on > >> > current-state, participant-config, and participant messages > >> > - When we use property store with caches enabled by zookeeper > watches, we > >> > may have dead watches on all removed paths > >> > > >> > Will create separate jiras to track the issue. > >> > > >> > > Controller should avoid resetting watches on removed paths > >> > > ---------------------------------------------------------- > >> > > > >> > > Key: HELIX-504 > >> > > URL: > https://issues.apache.org/jira/browse/HELIX-504 > >> > > Project: Apache Helix > >> > > Issue Type: Bug > >> > > Reporter: Kanak Biscuitwala > >> > > Assignee: Zhen Zhang > >> > > > >> > > Right now, if a participant has a session change, the old current > state > >> > will be removed, but the controller despite unregistering the > listener, > >> > still reset the watch on the path deleted event. We should avoid > >> resetting > >> > this path on delete and let the controller explicitly set > >> watches/listeners > >> > on new sessions. > >> > > >> > > >> > > >> > -- > >> > This message was sent by Atlassian JIRA > >> > (v6.3.4#6332) > >> > > >> >
