Thanks, Jordan. The ChildReaper is exactly what I was looking for. I'll hold off regarding upgrading of Zookeeper and Curator for now.
Cheers, Jens On Sun, Sep 13, 2015 at 10:27 PM, Jordan Zimmerman < [email protected]> wrote: > This has long been an issue in ZooKeeper. The good news is that there are > solutions: > > * Current releases: Curator has utilities for cleaning up old parent > nodes. Look at Reaper and ChildReaper > * New releases: ZooKeeper 3.5.1 + Curator 2.9.0 (just released) support > the new Container node feature. Curator 2.9.0 will automatically create > container nodes which are automatically cleaned up when empty. > > -Jordan > > > On September 13, 2015 at 11:41:41 AM, Jens Rantil ([email protected]) > wrote: > > Hi again, > > I have a distributed application which has InterProcessSemaphoreMutexes which > occasionally are taken per user. To do this, the mutex path is appended > with a user's unique ID (example: /mylock/{userid}). All is fine and > dandy and this generally works. However, I've noticed that the paths given > to the InterProcessSemaphoreMutexes aren't cleaned up after I'm done with > the locks. > > This means, my Zookeeper ensemble has every single one of my previous > mutexes held in memory. When I list my znodes I see: > /mylock/1 > /mylock/2 > ... > /mylock/XXX > > Given that I generally only take ~20 locks at a time it feels unnecessary > to keep all these in memory. We have had to increase initLimit in > Zookeeper configuration, most certainly due to all of these znodes. > > Two questions: > > - Is there a reason for keeping these znodes hanging around? > - Given that I need to use mutexes, what are the best practise for > taking locks per user like this? Hash the user IDs and bucket them into a > limitted number of mutexes (enough to avoid contention)? Have a separate > cleaning thread that cleans up old znodes? How are you guys handling this? > > Thanks, > Jens > > -- > Jens Rantil > Backend engineer > Tink AB > > Email: [email protected] > Phone: +46 708 84 18 32 > Web: www.tink.se > > Facebook <https://www.facebook.com/#!/tink.se> Linkedin > <http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary> > Twitter <https://twitter.com/tink> > > -- Jens Rantil Backend engineer Tink AB Email: [email protected] Phone: +46 708 84 18 32 Web: www.tink.se Facebook <https://www.facebook.com/#!/tink.se> Linkedin <http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary> Twitter <https://twitter.com/tink>
