Jordan, I see. That explains things.
Thanks, Jens On Thu, Sep 17, 2015 at 4:45 PM, Jordan Zimmerman < [email protected]> wrote: > Jens, > > Reaper and ChildReaper are now deprecated. Moving forward, Curator will > automatically use container nodes which are now supported directly by > ZooKeeper (I wrote the feature). Reaper/ChildReaper was always a hack to > work around the situation. > > -JZ > > > > On September 17, 2015 at 9:43:03 AM, Jens Rantil ([email protected]) > wrote: > > Hi again Jordan, > > Your input is highly appreciated. Thanks! > > I assume you mean I would do something like > > 1. myReaper = new ChildReaper(client, "/mylock") > 2. myReaper.start() > 3. mutex = new InterProcessSemaphoreMutex(client, "/mylock/user123"); > 4. myReaper.addPath("/mylock/user123") > 5. // Use the mutex... > > However, when reading the implementation of ChildReaper (version 2.7.1) > it looks like the paths member field never removes "/mylock/user123" if > it has been reaped by the reaper for "/mylock". This would essentially > become a memory leak if I have a lot of users, right? Or have I > misunderstood something? > > Generally this isn't a pretty abstraction since a ChildReaper user (me) > must essentially know how the internals of a recipe to know if it creates > additional hierarchies or not. A different approach would be to add a > `Reapable` interface to all recipe classes and have them implement how they > would reap themselves. Possibly they could simply delegate to a > RecursiveChildReaper that supports a configurable cleanup level (1 for > basic locks, 2 for InterProcessSemaphoreMutexes that has two level znode > hierarchies). > > Thanks, > Jens > > On Thu, Sep 17, 2015 at 3:38 PM, Jordan Zimmerman < > [email protected]> wrote: > > > > You’d need to add each level to the ChildReaper. You can also move to > ZooKeeper 3.5.1-alpha and Curator 2.9.0 and you’ll get automatic cleanup > (via Container nodes). > > > > -Jordan > > > > > > > > On September 17, 2015 at 8:11:49 AM, Jens Rantil ([email protected]) > wrote: > > > > Hi again, > > > > A follow-up regarding this: I also have a couple of > InterProcessSemaphoreMutexes that I'd like to clean up and since they are > using using a nested hierachy (containing "locks" and "leases"), they > aren't being cleaned up my ChildReaper. Is there a way to clean these up > automatically? > > > > Thanks, > > Jens > > > > On Mon, Sep 14, 2015 at 3:47 PM, Jordan Zimmerman < > [email protected]> wrote: > >> > >> ChildReaper uses a Reaper internally. So, just use whichever one works > for you. > >> > >> -Jordan > >> > >> > >> > >> On September 14, 2015 at 8:30:19 AM, Jens Rantil ([email protected]) > wrote: > >> > >> Hi again, > >> > >> A small follow-up question. What are the pros and cons of using Reaper > vs. ChildReaper? It seems to me that using ChildReaper will involve a lot > less intrusive code. Are there performance implications of one or the other? > >> > >> Thanks, > >> Jens > >> > >> On Sun, Sep 13, 2015 at 10:27 PM, Jordan Zimmerman < > [email protected]> wrote: > >>> > >>> This has long been an issue in ZooKeeper. The good news is that there > are solutions: > >>> > >>> * Current releases: Curator has utilities for cleaning up old parent > nodes. Look at Reaper and ChildReaper > >>> * New releases: ZooKeeper 3.5.1 + Curator 2.9.0 (just released) > support the new Container node feature. Curator 2.9.0 will automatically > create container nodes which are automatically cleaned up when empty. > >>> > >>> -Jordan > >>> > >>> > >>> On September 13, 2015 at 11:41:41 AM, Jens Rantil ([email protected]) > wrote: > >>> > >>> Hi again, > >>> > >>> I have a distributed application which has > InterProcessSemaphoreMutexes which occasionally are taken per user. To do > this, the mutex path is appended with a user's unique ID (example: > /mylock/{userid}). All is fine and dandy and this generally works. However, > I've noticed that the paths given to the InterProcessSemaphoreMutexes > aren't cleaned up after I'm done with the locks. > >>> > >>> This means, my Zookeeper ensemble has every single one of my previous > mutexes held in memory. When I list my znodes I see: > >>> /mylock/1 > >>> /mylock/2 > >>> ... > >>> /mylock/XXX > >>> > >>> Given that I generally only take ~20 locks at a time it feels > unnecessary to keep all these in memory. We have had to increase initLimit > in Zookeeper configuration, most certainly due to all of these znodes. > >>> > >>> Two questions: > >>> > >>> Is there a reason for keeping these znodes hanging around? > >>> Given that I need to use mutexes, what are the best practise for > taking locks per user like this? Hash the user IDs and bucket them into a > limitted number of mutexes (enough to avoid contention)? Have a separate > cleaning thread that cleans up old znodes? How are you guys handling this? > >>> > >>> Thanks, > >>> Jens > >>> > >>> -- > >>> Jens Rantil > >>> Backend engineer > >>> Tink AB > >>> > >>> Email: [email protected] > >>> Phone: +46 708 84 18 32 > >>> Web: www.tink.se > >>> > >>> Facebook Linkedin Twitter > >> > >> > >> > >> > >> -- > >> Jens Rantil > >> Backend engineer > >> Tink AB > >> > >> Email: [email protected] > >> Phone: +46 708 84 18 32 > >> Web: www.tink.se > >> > >> Facebook Linkedin Twitter > > > > > > > > > > -- > > Jens Rantil > > Backend engineer > > Tink AB > > > > Email: [email protected] > > Phone: +46 708 84 18 32 > > Web: www.tink.se > > > > Facebook Linkedin Twitter > > > > > -- > Jens Rantil > Backend engineer > Tink AB > > Email: [email protected] > Phone: +46 708 84 18 32 > Web: www.tink.se > > Facebook Linkedin Twitter > > -- Jens Rantil Backend engineer Tink AB Email: [email protected] Phone: +46 708 84 18 32 Web: www.tink.se Facebook <https://www.facebook.com/#!/tink.se> Linkedin <http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary> Twitter <https://twitter.com/tink>
