Hi again Jordan,
Your input is highly appreciated. Thanks!
I assume you mean I would do something like
1. myReaper = new ChildReaper(client, "/mylock")
2. myReaper.start()
3. mutex = new InterProcessSemaphoreMutex(client, "/mylock/user123");
4. myReaper.addPath("/mylock/user123")
5. // Use the mutex...
However, when reading the implementation of ChildReaper (version 2.7.1) it
looks like the paths member field never removes "/mylock/user123" if it has
been reaped by the reaper for "/mylock". This would essentially become a
memory leak if I have a lot of users, right? Or have I misunderstood
something?
Generally this isn't a pretty abstraction since a ChildReaper user (me)
must essentially know how the internals of a recipe to know if it creates
additional hierarchies or not. A different approach would be to add a
`Reapable` interface to all recipe classes and have them implement how they
would reap themselves. Possibly they could simply delegate to a
RecursiveChildReaper that supports a configurable cleanup level (1 for
basic locks, 2 for InterProcessSemaphoreMutexes that has two level znode
hierarchies).
Thanks,
Jens
On Thu, Sep 17, 2015 at 3:38 PM, Jordan Zimmerman <
[email protected]> wrote:
>
> You’d need to add each level to the ChildReaper. You can also move to
ZooKeeper 3.5.1-alpha and Curator 2.9.0 and you’ll get automatic cleanup
(via Container nodes).
>
> -Jordan
>
>
>
> On September 17, 2015 at 8:11:49 AM, Jens Rantil ([email protected])
wrote:
>
> Hi again,
>
> A follow-up regarding this: I also have a couple of
InterProcessSemaphoreMutexes that I'd like to clean up and since they are
using using a nested hierachy (containing "locks" and "leases"), they
aren't being cleaned up my ChildReaper. Is there a way to clean these up
automatically?
>
> Thanks,
> Jens
>
> On Mon, Sep 14, 2015 at 3:47 PM, Jordan Zimmerman <
[email protected]> wrote:
>>
>> ChildReaper uses a Reaper internally. So, just use whichever one works
for you.
>>
>> -Jordan
>>
>>
>>
>> On September 14, 2015 at 8:30:19 AM, Jens Rantil ([email protected])
wrote:
>>
>> Hi again,
>>
>> A small follow-up question. What are the pros and cons of using Reaper
vs. ChildReaper? It seems to me that using ChildReaper will involve a lot
less intrusive code. Are there performance implications of one or the other?
>>
>> Thanks,
>> Jens
>>
>> On Sun, Sep 13, 2015 at 10:27 PM, Jordan Zimmerman <
[email protected]> wrote:
>>>
>>> This has long been an issue in ZooKeeper. The good news is that there
are solutions:
>>>
>>> * Current releases: Curator has utilities for cleaning up old parent
nodes. Look at Reaper and ChildReaper
>>> * New releases: ZooKeeper 3.5.1 + Curator 2.9.0 (just released) support
the new Container node feature. Curator 2.9.0 will automatically create
container nodes which are automatically cleaned up when empty.
>>>
>>> -Jordan
>>>
>>>
>>> On September 13, 2015 at 11:41:41 AM, Jens Rantil ([email protected])
wrote:
>>>
>>> Hi again,
>>>
>>> I have a distributed application which has InterProcessSemaphoreMutexes
which occasionally are taken per user. To do this, the mutex path is
appended with a user's unique ID (example: /mylock/{userid}). All is fine
and dandy and this generally works. However, I've noticed that the paths
given to the InterProcessSemaphoreMutexes aren't cleaned up after I'm done
with the locks.
>>>
>>> This means, my Zookeeper ensemble has every single one of my previous
mutexes held in memory. When I list my znodes I see:
>>> /mylock/1
>>> /mylock/2
>>> ...
>>> /mylock/XXX
>>>
>>> Given that I generally only take ~20 locks at a time it feels
unnecessary to keep all these in memory. We have had to increase initLimit
in Zookeeper configuration, most certainly due to all of these znodes.
>>>
>>> Two questions:
>>>
>>> Is there a reason for keeping these znodes hanging around?
>>> Given that I need to use mutexes, what are the best practise for taking
locks per user like this? Hash the user IDs and bucket them into a limitted
number of mutexes (enough to avoid contention)? Have a separate cleaning
thread that cleans up old znodes? How are you guys handling this?
>>>
>>> Thanks,
>>> Jens
>>>
>>> --
>>> Jens Rantil
>>> Backend engineer
>>> Tink AB
>>>
>>> Email: [email protected]
>>> Phone: +46 708 84 18 32
>>> Web: www.tink.se
>>>
>>> Facebook Linkedin Twitter
>>
>>
>>
>>
>> --
>> Jens Rantil
>> Backend engineer
>> Tink AB
>>
>> Email: [email protected]
>> Phone: +46 708 84 18 32
>> Web: www.tink.se
>>
>> Facebook Linkedin Twitter
>
>
>
>
> --
> Jens Rantil
> Backend engineer
> Tink AB
>
> Email: [email protected]
> Phone: +46 708 84 18 32
> Web: www.tink.se
>
> Facebook Linkedin Twitter
--
Jens Rantil
Backend engineer
Tink AB
Email: [email protected]
Phone: +46 708 84 18 32
Web: www.tink.se
Facebook Linkedin Twitter
