Curator puts nodes to be deleted in a background thread if there is a problem deleting the node. Also, the nodes are ephemeral so they will be automatically delete if the client has a connection issue. Is it possible that the client didn’t call release on the lock? I haven’t heard of this issue from anyone else. What does a thread dump of that instance show? Where is it blocking?
From: Bae, Jae Hyeon [email protected] Reply: Bae, Jae Hyeon [email protected] Date: May 27, 2014 at 11:41:42 AM To: Jordan Zimmerman [email protected] Cc: [email protected] [email protected] Subject: Re: Issue with mutexes not being released More information. - Curator version seems not changed and the version is 2.4.2. - The team is using InterProcessMutex recipe. - "not always deleted" means the znode isn't deleted Here is the info for the most recent lock not being released: Path /Jabba/agent/pop/-750322352/_c_07036068-c848-4720-b553-d7b446d413a3-lock-0000001573 Stat czxid: 730803957058, mzxid: 730803957058, ctime: 1401067656032, mtime: 1401067656032, version: 0, cversion: 0, aversion: 0, ephemeralOwner: 4487297975270022858, dataLength: 16, numChildren: 0, pzxid: 730803957058 Data Bytes 6a 61 62 62 61 2d 69 2d 32 38 65 39 38 38 37 38 Data as String jabba-i-<instanceID> On Tue, May 27, 2014 at 6:35 AM, Jordan Zimmerman <[email protected]> wrote: Somehow these locks are being created but when cleared out by the finally, they seem to not always be deleted. What do you mean by “not always be deleted”? Do you mean the lock isn’t released? How do you know? Also, Is this in 2.4.2? Which recipe? InterProcessMutex? Nothing has changed in that code that I know of (though it uses code that has changed for other reasons). There are numerous tests for locks and they all currently pass. Any more details on this? -JZ From: Bae, Jae Hyeon [email protected] Reply: [email protected] [email protected] Date: May 27, 2014 at 1:10:48 AM To: [email protected] [email protected] Subject: Issue with mutexes not being released Hi I got reported mutexes not being released. Somehow these locks are being created but when cleared out by the finally, they seem to not always be deleted. This just suddenly started happening since they upgraded curator version, despite there being no changes to this code at all for quite some time. Do you have any idea what have changed? Thank you Best, Jae
