Jens, Reaper and ChildReaper are now deprecated. Moving forward, Curator will automatically use container nodes which are now supported directly by ZooKeeper (I wrote the feature). Reaper/ChildReaper was always a hack to work around the situation.
-JZ On September 17, 2015 at 9:43:03 AM, Jens Rantil ([email protected]) wrote: Hi again Jordan, Your input is highly appreciated. Thanks! I assume you mean I would do something like myReaper = new ChildReaper(client, "/mylock") myReaper.start() mutex = new InterProcessSemaphoreMutex(client, "/mylock/user123"); myReaper.addPath("/mylock/user123") // Use the mutex... However, when reading the implementation of ChildReaper (version 2.7.1) it looks like the paths member field never removes "/mylock/user123" if it has been reaped by the reaper for "/mylock". This would essentially become a memory leak if I have a lot of users, right? Or have I misunderstood something? Generally this isn't a pretty abstraction since a ChildReaper user (me) must essentially know how the internals of a recipe to know if it creates additional hierarchies or not. A different approach would be to add a `Reapable` interface to all recipe classes and have them implement how they would reap themselves. Possibly they could simply delegate to a RecursiveChildReaper that supports a configurable cleanup level (1 for basic locks, 2 for InterProcessSemaphoreMutexes that has two level znode hierarchies). Thanks, Jens On Thu, Sep 17, 2015 at 3:38 PM, Jordan Zimmerman <[email protected]> wrote: > > You’d need to add each level to the ChildReaper. You can also move to > ZooKeeper 3.5.1-alpha and Curator 2.9.0 and you’ll get automatic cleanup (via > Container nodes). > > -Jordan > > > > On September 17, 2015 at 8:11:49 AM, Jens Rantil ([email protected]) wrote: > > Hi again, > > A follow-up regarding this: I also have a couple of > InterProcessSemaphoreMutexes that I'd like to clean up and since they are > using using a nested hierachy (containing "locks" and "leases"), they aren't > being cleaned up my ChildReaper. Is there a way to clean these up > automatically? > > Thanks, > Jens > > On Mon, Sep 14, 2015 at 3:47 PM, Jordan Zimmerman > <[email protected]> wrote: >> >> ChildReaper uses a Reaper internally. So, just use whichever one works for >> you. >> >> -Jordan >> >> >> >> On September 14, 2015 at 8:30:19 AM, Jens Rantil ([email protected]) wrote: >> >> Hi again, >> >> A small follow-up question. What are the pros and cons of using Reaper vs. >> ChildReaper? It seems to me that using ChildReaper will involve a lot less >> intrusive code. Are there performance implications of one or the other? >> >> Thanks, >> Jens >> >> On Sun, Sep 13, 2015 at 10:27 PM, Jordan Zimmerman >> <[email protected]> wrote: >>> >>> This has long been an issue in ZooKeeper. The good news is that there are >>> solutions: >>> >>> * Current releases: Curator has utilities for cleaning up old parent nodes. >>> Look at Reaper and ChildReaper >>> * New releases: ZooKeeper 3.5.1 + Curator 2.9.0 (just released) support the >>> new Container node feature. Curator 2.9.0 will automatically create >>> container nodes which are automatically cleaned up when empty. >>> >>> -Jordan >>> >>> >>> On September 13, 2015 at 11:41:41 AM, Jens Rantil ([email protected]) >>> wrote: >>> >>> Hi again, >>> >>> I have a distributed application which has InterProcessSemaphoreMutexes >>> which occasionally are taken per user. To do this, the mutex path is >>> appended with a user's unique ID (example: /mylock/{userid}). All is fine >>> and dandy and this generally works. However, I've noticed that the paths >>> given to the InterProcessSemaphoreMutexes aren't cleaned up after I'm done >>> with the locks. >>> >>> This means, my Zookeeper ensemble has every single one of my previous >>> mutexes held in memory. When I list my znodes I see: >>> /mylock/1 >>> /mylock/2 >>> ... >>> /mylock/XXX >>> >>> Given that I generally only take ~20 locks at a time it feels unnecessary >>> to keep all these in memory. We have had to increase initLimit in Zookeeper >>> configuration, most certainly due to all of these znodes. >>> >>> Two questions: >>> >>> Is there a reason for keeping these znodes hanging around? >>> Given that I need to use mutexes, what are the best practise for taking >>> locks per user like this? Hash the user IDs and bucket them into a limitted >>> number of mutexes (enough to avoid contention)? Have a separate cleaning >>> thread that cleans up old znodes? How are you guys handling this? >>> >>> Thanks, >>> Jens >>> >>> -- >>> Jens Rantil >>> Backend engineer >>> Tink AB >>> >>> Email: [email protected] >>> Phone: +46 708 84 18 32 >>> Web: www.tink.se >>> >>> Facebook Linkedin Twitter >> >> >> >> >> -- >> Jens Rantil >> Backend engineer >> Tink AB >> >> Email: [email protected] >> Phone: +46 708 84 18 32 >> Web: www.tink.se >> >> Facebook Linkedin Twitter > > > > > -- > Jens Rantil > Backend engineer > Tink AB > > Email: [email protected] > Phone: +46 708 84 18 32 > Web: www.tink.se > > Facebook Linkedin Twitter -- Jens Rantil Backend engineer Tink AB Email: [email protected] Phone: +46 708 84 18 32 Web: www.tink.se Facebook Linkedin Twitter
