[jira] Commented: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[
https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616603#action_12616603
]
james strachan commented on ZOOKEEPER-78:
-
{quote}
Let's stick with a process for now that all contributors can use, not just ASF
committers.
{quote}
Huh? Everyone has access to ASF svn? Only committers can commit using either
approach. I don't grok your point.
> added a high level protocol/feature - for easy Leader Election or exclusive
> Write Lock creation
> ---
>
> Key: ZOOKEEPER-78
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78
> Project: Zookeeper
> Issue Type: New Feature
> Components: java client
>Affects Versions: 3.0.0
>Reporter: james strachan
> Attachments: patch_with_including_Benjamin's_fix.patch,
> using_zookeeper_facade.patch
>
>
> Here's a patch which adds a little WriteLock helper class for performing
> leader elections or creating exclusive locks in some directory znode. Note
> its an early cut; am sure we can improve it over time. The aim is to avoid
> folks having to use the low level ZK stuff but provide a simpler high level
> abstraction.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-83) Switch to using maven to build ZooKeeper
[ https://issues.apache.org/jira/browse/ZOOKEEPER-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616580#action_12616580 ] james strachan commented on ZOOKEEPER-83: - Hudson does support Maven and Ant natively http://hudson.gotdns.com/wiki/display/HUDSON/Plugins#Plugins-Buildtools > Switch to using maven to build ZooKeeper > > > Key: ZOOKEEPER-83 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-83 > Project: Zookeeper > Issue Type: Improvement > Components: build >Reporter: Hiram Chirino > Attachments: zookeeper-mavened.tgz > > > Maven is a great too for building java projects at the ASF. It helps > standardize the build a bit since it's a convention oriented. > It's dependency auto downloading would remove the need to store the > dependencies in svn, and it will handle many of the suggested ASF policies > like gpg signing of the releases and such. > The ZooKeeper build is almost vanilla except for the jute compiler bits. > Things that would need to change are: > * re-organize the source tree a little so that it uses the maven directory > conventions > * seperate the jute bits out into seperate modules so that a maven plugin > can be with it > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616579#action_12616579 ] james strachan commented on ZOOKEEPER-78: - Wow I confess to be being kinda surprised by that response :) I didn't realise you guys were so attached to the exact svn command line used to apply a patch - I thought you'd welcome all contributions and that the svn commit history would be more useful to you, given the large amount of changes and complexity of the code and large number of comments already on this JIRA - rather than focus purely on a minor couple of keystrokes required apply the patch :) FWIW I've attached about 5 patch files so far, with none of them being committed anywhere - then made about 7 patches since then in svn with history and I'm sure there'll be another 5 or so changes to go before this patch is done. Never mind - I'll happily comply with your strict patch acceptance policy. Give me a few weeks or so to completely finish the code and documentation and I'll submit a single big patch for all the work to this JIRA. If you want you can get all the history too with a trivial alternative svn command - but if that offends you, please forget I'm using a sandbox svn area to work on this (pretend I'm just saving it on my hard drive and please disregard the links I've added to some JIRAs to refer to parts of this patch in a simple way) and just use the single patch file I'll attach in a few weeks or so. > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan > Attachments: patch_with_including_Benjamin's_fix.patch, > using_zookeeper_facade.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-58) Race condition on ClientCnxn.java
[ https://issues.apache.org/jira/browse/ZOOKEEPER-58?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-58: Status: Patch Available (was: Open) just marking this issue as having a patch - does a committer fancy applying it? :) > Race condition on ClientCnxn.java > - > > Key: ZOOKEEPER-58 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-58 > Project: Zookeeper > Issue Type: Bug > Components: java client >Reporter: Flavio Paiva Junqueira >Assignee: Flavio Paiva Junqueira > Attachments: patch-incoming-race.txt > > > There is a race condition involving the ByteByffer incomingBuffer, a field of > ClientCnxn.SendThread. SendThread reads a packet in two steps: first it reads > the length of the packet, followed by a read of the packet itself. Each of > these steps corresponds to a call to doIO() from the main loop of run(). If > there is an exception or the session times out, then it may leave > incomingBuffer in an inconsistent state. > The attached patch adds code to reset incomingBuffer upon a call to > SendThread.cleanup(). This method is called upon an exception on run(). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-74) Cleaning/restructuring up Zookeeper server code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616518#action_12616518 ] james strachan commented on ZOOKEEPER-74: - its a very minor thing, but the contributing guide says to use Sun's coding conventions.. http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute yet lots of the code tends to litter fields in classes in between methods; its sometimes a bit hard looking at the source to grok what fields are owned by what object. I prefer the Sun standards where all the fields are at the top as most java folks and apache java projects do. Though good IDE's can kinda work around this though and so I tend to rely on the outline view in my IDE rather than the source to grok what state a class has :) > Cleaning/restructuring up Zookeeper server code > --- > > Key: ZOOKEEPER-74 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-74 > Project: Zookeeper > Issue Type: Improvement > Components: server >Reporter: Mahadev konar >Assignee: Mahadev konar > Fix For: 3.0.0 > > > I have been thinking this for a while and find that the zookeeper server code > needs some cleaning up. The server code is a little tricky/confusing to read > sometimes gievn that there is no clearity on ownership of objects. I will put > down a proposal for restructuring/cleaning the code up with javadocs so that > the code is easier to understand and develop on. comments on what you find > confusing are welcome on this jira. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-88) implement java.util.concurrent.locks.Lock
[ https://issues.apache.org/jira/browse/ZOOKEEPER-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-88: Status: Patch Available (was: Open) I've just submitted an [initial patch at implementing this|http://svn.apache.org/viewvc?rev=679435&view=rev] which could use some more tests and code review > implement java.util.concurrent.locks.Lock > - > > Key: ZOOKEEPER-88 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-88 > Project: Zookeeper > Issue Type: Sub-task > Components: java client >Reporter: james strachan > > we should implement the > [java.util.concurrent.locks.Lock|http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/locks/Lock.html] > to make it easier for folks to reuse the Lock and to help make the API be > more natural to end users -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-90) invoke WhenOwnerListener.whenNotOwner() when the ZK session expires and the znode is the leader
[ https://issues.apache.org/jira/browse/ZOOKEEPER-90?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616456#action_12616456 ] james strachan commented on ZOOKEEPER-90: - I guess the earlier we can notify a leader/owner that they are no longer gonna be the leader/owner the better. e.g. its better to notify them up front than wait until the effect of a socket closing etc? > invoke WhenOwnerListener.whenNotOwner() when the ZK session expires and the > znode is the leader > --- > > Key: ZOOKEEPER-90 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-90 > Project: Zookeeper > Issue Type: Sub-task >Reporter: james strachan > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-90) invoke WhenOwnerListener.whenNotOwner() when the ZK session expires and the znode is the leader
[ https://issues.apache.org/jira/browse/ZOOKEEPER-90?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616455#action_12616455 ] james strachan commented on ZOOKEEPER-90: - FWIW I don't think the ZooKeeperFacade needs to have a WhenOwnerListener to be honest; as the underlying ZooKeeper should invoke the Watcher directly - when the connection is closed via the session expired reconnect or via close(). I guess it doesn't harm to have a few too many events firing in there just in case :) > invoke WhenOwnerListener.whenNotOwner() when the ZK session expires and the > znode is the leader > --- > > Key: ZOOKEEPER-90 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-90 > Project: Zookeeper > Issue Type: Sub-task >Reporter: james strachan > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-92) spring factory beans for ZooKeeper, ZooKeeperFacade and ZooKeeperServer
spring factory beans for ZooKeeper, ZooKeeperFacade and ZooKeeperServer --- Key: ZOOKEEPER-92 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-92 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: james strachan for folks who use Spring for Dependency Injection it might be handy to have a couple of factory beans to make it easier to create and configure the ZooKeeper, ZooKeeperFacade and ZooKeeperServer via the normal Spring dependency mechanism; via Java or XML code etc -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (ZOOKEEPER-89) invoke WhenOwnerListener.whenNotOwner() when the ZK connection fails
[ https://issues.apache.org/jira/browse/ZOOKEEPER-89?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan resolved ZOOKEEPER-89. - Resolution: Fixed fixed now with [this patch|http://svn.apache.org/viewvc?rev=679354&view=rev] from ZOOKEEPER-78 > invoke WhenOwnerListener.whenNotOwner() when the ZK connection fails > > > Key: ZOOKEEPER-89 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-89 > Project: Zookeeper > Issue Type: Sub-task >Reporter: james strachan > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616446#action_12616446 ] james strachan commented on ZOOKEEPER-78: - Benjamin I added ZOOKEEPER-89 and ZOOKEEPER-90 to track the dealing with loss of ownership/leader with connection reconnects and with session expiration. I've not been able to test out the latter yet; but I've tested the former and I think both are implemented now via the patch for [ZOOKEEPER-90|http://svn.apache.org/viewvc?rev=679354&view=r] and [ZOOKEEPER-89|http://svn.apache.org/viewvc?rev=679341&view=rev] > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan > Attachments: patch_with_including_Benjamin's_fix.patch, > using_zookeeper_facade.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-91) provide an option for the WriteLock to also watch the locks own znode, so that if someone else deletes it then it is equivalent to calling WriteLock.unlock()
provide an option for the WriteLock to also watch the locks own znode, so that if someone else deletes it then it is equivalent to calling WriteLock.unlock() - Key: ZOOKEEPER-91 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-91 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: james strachan Most clients probably wont need this, but it could be a handy system management feature to allow the WriteLock to watch its own znode so that if someone else deletes it, it then relinquishes the lock and tries to get it back again -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (ZOOKEEPER-90) invoke WhenOwnerListener.whenNotOwner() when the ZK session expires and the znode is the leader
[ https://issues.apache.org/jira/browse/ZOOKEEPER-90?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan resolved ZOOKEEPER-90. - Resolution: Fixed this is now fixed in [this patch|http://svn.apache.org/viewvc?rev=679341&view=rev] to ZOOKEEPER-78 > invoke WhenOwnerListener.whenNotOwner() when the ZK session expires and the > znode is the leader > --- > > Key: ZOOKEEPER-90 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-90 > Project: Zookeeper > Issue Type: Sub-task >Reporter: james strachan > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-63) Race condition in client close() operation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-63?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616426#action_12616426 ] james strachan commented on ZOOKEEPER-63: - So this patch does not attempt to fix the race condition problem, apologies if I gave that impression :) What it does do though is act as a workaround so that if a client is not able to properly send a disconnect packet to the server for *any reason at all* such as * a hung socket (which can be quite common) * no servers available * a race condition in the ZK client code of some kind (which we definitely have now) to not hang the client application forever - as its trying to close and shut down anyway :). So its a side benefit that it acts as a band aid until someone fixes all the possible race conditions and potential socket hangs. Let me put it another way. Given that the client is closing; is it really correct to leave it potentially hanging around forever just because it cannot be sure if the disconnect packet was received and properly processed by the server? If the socket is dead before the call to close(), is it really correct to block until a connection can be re-established, just so it can be properly closed - when the code will effectively close the hung socket without sending a disconnect packet anyway :) ? The server has to detect and timeout failed sessions; whether it receives an explicit disconnect packet or not (as a process could just hang). So do we really need to be super strict on the client side, forcing clients to block when trying to shut down if they can't do so cleanly within some time period? I totally agree that we should fix the race condition though :). I just wanted a work around to avoid my ZK test cases hanging forever due to the race condition :) > Race condition in client close() operation > -- > > Key: ZOOKEEPER-63 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-63 > Project: Zookeeper > Issue Type: Bug > Components: java client >Reporter: Patrick Hunt >Assignee: Benjamin Reed > Attachments: patch_ZOOKEEPER-63.patch > > > There is a race condition in the java close operation on ZooKeeper.java. > Client is sending a disconnect request to the server. Server will close any > open connections with the client when it receives this. If the client has not > yet shutdown it's subthreads (event/send threads for example) these threads > may consider the condition an error. We see this alot in the tests where the > clients output error logs because they are unaware that a disconnection has > been requested by the client. > Ben mentioned: perhaps we just have to change state to closed (on client) > before sending disconnect request. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-84) provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException
[ https://issues.apache.org/jira/browse/ZOOKEEPER-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616420#action_12616420 ] james strachan commented on ZOOKEEPER-84: - BTW here is the code for [ZooKeeperFacade|http://svn.apache.org/viewvc/activemq/sandbox/zookeeper/zookeeper-protocols/src/main/java/org/apache/zookeeper/protocols/ZooKeeperFacade.java?view=markup] as I've checked in the patch for ZOOKEEPER-78 into a temporary sandbox area, [details here|https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616391#action_12616391] > provide a mechanism to reconnect a ZooKeeper if a client receives a > SessionExpiredException > --- > > Key: ZOOKEEPER-84 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-84 > Project: Zookeeper > Issue Type: Improvement > Components: java client >Reporter: james strachan >Assignee: Benjamin Reed > Attachments: reconnect_patch.patch > > > am about to attach a patch which adds a reconnect() method to easily > re-establish a connection if a session expires - along with a toString() > implementation for easier debugging -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-90) invoke WhenOwnerListener.whenNotOwner() when the ZK session expires and the znode is the leader
invoke WhenOwnerListener.whenNotOwner() when the ZK session expires and the znode is the leader --- Key: ZOOKEEPER-90 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-90 Project: Zookeeper Issue Type: Sub-task Reporter: james strachan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-89) invoke WhenOwnerListener.whenNotOwner() when the ZK connection fails
invoke WhenOwnerListener.whenNotOwner() when the ZK connection fails Key: ZOOKEEPER-89 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-89 Project: Zookeeper Issue Type: Sub-task Reporter: james strachan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-22) Automatic request retries on connect failover
[ https://issues.apache.org/jira/browse/ZOOKEEPER-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616418#action_12616418 ] james strachan commented on ZOOKEEPER-22: - BTW you can see the code for [ProtocolSupport|http://svn.apache.org/viewvc/activemq/sandbox/zookeeper/zookeeper-protocols/src/main/java/org/apache/zookeeper/protocols/ProtocolSupport.java?view=markup] and [ZooKeeperFacade|http://svn.apache.org/viewvc/activemq/sandbox/zookeeper/zookeeper-protocols/src/main/java/org/apache/zookeeper/protocols/ZooKeeperFacade.java?view=markup] as I've checked in the patch for ZOOKEEPER-78 into a temporary sandbox area, [details here|https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616391#action_12616391] > Automatic request retries on connect failover > - > > Key: ZOOKEEPER-22 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-22 > Project: Zookeeper > Issue Type: New Feature > Components: c client, java client >Reporter: Patrick Hunt > > Moved from SourceForge to Apache. > http://sourceforge.net/tracker/index.php?func=detail&aid=1831412&group_id=209147&atid=1008547 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616416#action_12616416 ] james strachan commented on ZOOKEEPER-78: - Just added the WhenOwnerListener interface : http://svn.apache.org/viewvc?view=rev&revision=679325 I just need to figure out how to add notifications of loss of owner/leader status when the connection fails or the session expires etc. > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan > Attachments: patch_with_including_Benjamin's_fix.patch, > using_zookeeper_facade.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-88) implement java.util.concurrent.locks.Lock
implement java.util.concurrent.locks.Lock - Key: ZOOKEEPER-88 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-88 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: james strachan we should implement the [java.util.concurrent.locks.Lock|http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/locks/Lock.html] to make it easier for folks to reuse the Lock and to help make the API be more natural to end users -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[
https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616413#action_12616413
]
james strachan commented on ZOOKEEPER-78:
-
Thanks for the great comments Benjamin! Have already [added the constructor for
you|https://svn.apache.org/repos/asf/activemq/sandbox/zookeeper/zookeeper-protocols/src/main/java/org/apache/zookeeper/protocols/WriteLock.java]
:)
BTW I was pondering about switching the whenOwner from a Runnable to some kinda
interface that invokes you when you become the leader/owner - or when you stop
being the leader/owner. Something like
{code}
public interface WhenOwnerListener {
void whenOwner();
void whenNotOwner();
}
{code}
Where only znodes that are the owner would be notified with the whenOwner()
method; but then if a connection fails or session times out, they'd be notified
with a call to whenNotOwner();
Spookily - I'd set myself the target today to properly implement the watches so
that WriteLock gets a notification of it no longer being the leader/owner when
a connection fails (which normally auto-reconnects anyway right now in the base
ZooKeeper). Then I was gonna add a notification mechanism so we could notify
the leader/owner is no longer the leader/owner when the session expired
exception occurs.
So we're absolutely on the same page; once I'd grokked the proper watch code
for dealing with normal connection failures & reconnects I was hoping to add
something vaguely similar to the ZooKeeperFacade so that higher level protocols
can be aware of both when ZooKeeper reconnects and when ZooKeeperFacade creates
a whole new connection.
Does that make sense? I totally understand your concerns at making sure the
WriteLock and associated helper classes like ProtocolSupport/ZooKeeperFacade do
the right thing - I want exactly the same thing :) I'd just not yet had the
chance to go through all the different failure conditions and scenarios and
make sure they all work properly :)
> added a high level protocol/feature - for easy Leader Election or exclusive
> Write Lock creation
> ---
>
> Key: ZOOKEEPER-78
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78
> Project: Zookeeper
> Issue Type: New Feature
> Components: java client
>Affects Versions: 3.0.0
>Reporter: james strachan
> Attachments: patch_with_including_Benjamin's_fix.patch,
> using_zookeeper_facade.patch
>
>
> Here's a patch which adds a little WriteLock helper class for performing
> leader elections or creating exclusive locks in some directory znode. Note
> its an early cut; am sure we can improve it over time. The aim is to avoid
> folks having to use the low level ZK stuff but provide a simpler high level
> abstraction.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616391#action_12616391 ] james strachan commented on ZOOKEEPER-78: - h3. Moving patches for this issue to subversion for easier tracking http://svn.apache.org/repos/asf/activemq/sandbox/zookeeper/zookeeper-protocols/ I've already submitted about five patches for this issue so far and I'm sure there's gonna be loads more coming. Developing higher level protocols is a much bigger job than I previously thought ;-) particularly with having tests for all the various failure scenarios and adding support for the various other higher level protocols. Its kinda time consuming creating loads of patches & attaching them to the same issue and deleting the old ones so its easy for commmitters to review - but more importantly, all the history of the many patches gets totally lost using the attach-patch-to-jira model - which also makes it harder for committers to watch progress on this issue. I've never done this before on any other Apache project - and this approach is *temporary* and only reserved for the single ZOOKEEPER-78 issue; but I've checked in this patch into an svn sandbox area at Apache that I have commit karma on and will continue to work on it there; so that all the history is preserved. I can then do many more frequent & smaller commits; any ZK committer can review and easily apply my patches whenever they feel like - and its gonna be much easier for anyone in the ZK community to track progress on this issue and see how the implementation has changed over time as some things work or I find better ways of solving the issue. This approach is totally temporary; its not an attempt to move the code outside of the ZK community or anything like that. At any point feel free to commit (actually just copy in svn which will keep all the history & commit comments etc) to the ZK trunk. You could even mirror the code to the ZK tree in sandbox/contrib area if you like - just like Hiram did to mirror the ZK code to the maven-patch example in the activemq sandbox. I'm hoping in a few weeks my hacking on this issue will near completion and we can permanently move the code back into the ZK tree; but in the meantime its trivial to reuse it where it is or mirror it into the ZK tree as folks in the ZK community see fit. Also if I ever earn committer karma on ZK I can just move it into some ZK contrib area myself :) h3. Building the code In terms of sandbox - I ended up reusing Hiram's sandbox area that shows the maven build working on ZK; as I prefer to use maven and it was then super easy for me to create a new maven module, zookeeper-protocols that just includes the source and test cases for the high level protocols. If you're new to maven and want to build it, I've checked in instructions here... https://svn.apache.org/repos/asf/activemq/sandbox/zookeeper/BUILDING-maven.txt Whenever we move this code back into the ZK trunk am sure we can hack an Ant build for it. > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan > Attachments: patch_with_including_Benjamin's_fix.patch, > using_zookeeper_facade.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-22) Automatic request retries on connect failover
[ https://issues.apache.org/jira/browse/ZOOKEEPER-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616374#action_12616374 ] james strachan commented on ZOOKEEPER-22: - BTW this discussion came up recently on the dev lists too... http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-dev/200807.mbox/[EMAIL PROTECTED] To be able to retry operations on conection close (or due to session expiration) there is a patch in https://issues.apache.org/jira/browse/ZOOKEEPER-78 which adds a ZooKeeperFacade for dealing with reconnecting on session expiration and some helper methods in ProtocolSupport for retrying synchronous operations or blocks of code in light of connection failures > Automatic request retries on connect failover > - > > Key: ZOOKEEPER-22 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-22 > Project: Zookeeper > Issue Type: New Feature > Components: c client, java client >Reporter: Patrick Hunt > > Moved from SourceForge to Apache. > http://sourceforge.net/tracker/index.php?func=detail&aid=1831412&group_id=209147&atid=1008547 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-86) intermittent test failure of org.apache.zookeeper.test.AsyncTest
[ https://issues.apache.org/jira/browse/ZOOKEEPER-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616365#action_12616365 ] james strachan commented on ZOOKEEPER-86: - BTW I have sometimes still seen the AsyncHammerTest fail on OS X still; the basic issue is the restart of the quorum servers - its often the 3rd one - the server socket has not yet been released by the OS which tends to cause the failure. While things seem to work much better now, we might wanna add a bigger sleep in between restarts if it starts getting more common again > intermittent test failure of org.apache.zookeeper.test.AsyncTest > > > Key: ZOOKEEPER-86 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-86 > Project: Zookeeper > Issue Type: Bug > Components: tests > Environment: OS X and linux. It sometimes passes; but mostly seems to > fail on OS X each time >Reporter: james strachan > Attachments: patch_for_ZOOKEEPER-86.patch, > TEST-org.apache.zookeeper.test.AsyncTest.txt > > > Will attach the test output in an attachment... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-86) intermittent test failure of org.apache.zookeeper.test.AsyncTest
[ https://issues.apache.org/jira/browse/ZOOKEEPER-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-86: Attachment: patch_for_ZOOKEEPER-86.patch this patch seems to fix the test case on OS X at least; I've split the test case into 2 parts (so they are forked separately) and added more delays before trying to rebind to the server socket which seems to fix the error > intermittent test failure of org.apache.zookeeper.test.AsyncTest > > > Key: ZOOKEEPER-86 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-86 > Project: Zookeeper > Issue Type: Bug > Components: tests > Environment: OS X and linux. It sometimes passes; but mostly seems to > fail on OS X each time >Reporter: james strachan > Attachments: patch_for_ZOOKEEPER-86.patch, > TEST-org.apache.zookeeper.test.AsyncTest.txt > > > Will attach the test output in an attachment... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-86) intermittent test failure of org.apache.zookeeper.test.AsyncTest
[ https://issues.apache.org/jira/browse/ZOOKEEPER-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-86: Status: Patch Available (was: Open) about to attach > intermittent test failure of org.apache.zookeeper.test.AsyncTest > > > Key: ZOOKEEPER-86 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-86 > Project: Zookeeper > Issue Type: Bug > Components: tests > Environment: OS X and linux. It sometimes passes; but mostly seems to > fail on OS X each time >Reporter: james strachan > Attachments: TEST-org.apache.zookeeper.test.AsyncTest.txt > > > Will attach the test output in an attachment... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-86) intermittent test failure of org.apache.zookeeper.test.AsyncTest
[ https://issues.apache.org/jira/browse/ZOOKEEPER-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-86: Attachment: TEST-org.apache.zookeeper.test.AsyncTest.txt here's the output when ran on OS X (using Leopard) > intermittent test failure of org.apache.zookeeper.test.AsyncTest > > > Key: ZOOKEEPER-86 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-86 > Project: Zookeeper > Issue Type: Bug > Components: tests > Environment: OS X and linux. It sometimes passes; but mostly seems to > fail on OS X each time >Reporter: james strachan > Attachments: TEST-org.apache.zookeeper.test.AsyncTest.txt > > > Will attach the test output in an attachment... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-86) intermittent test failure of org.apache.zookeeper.test.AsyncTest
intermittent test failure of org.apache.zookeeper.test.AsyncTest Key: ZOOKEEPER-86 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-86 Project: Zookeeper Issue Type: Bug Components: tests Environment: OS X and linux. It sometimes passes; but mostly seems to fail on OS X each time Reporter: james strachan Will attach the test output in an attachment... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-63) Race condition in client close() operation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-63?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-63: Status: Patch Available (was: Open) about to attach a patch > Race condition in client close() operation > -- > > Key: ZOOKEEPER-63 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-63 > Project: Zookeeper > Issue Type: Bug > Components: java client >Reporter: Patrick Hunt >Assignee: Benjamin Reed > Attachments: patch_ZOOKEEPER-63.patch > > > There is a race condition in the java close operation on ZooKeeper.java. > Client is sending a disconnect request to the server. Server will close any > open connections with the client when it receives this. If the client has not > yet shutdown it's subthreads (event/send threads for example) these threads > may consider the condition an error. We see this alot in the tests where the > clients output error logs because they are unaware that a disconnection has > been requested by the client. > Ben mentioned: perhaps we just have to change state to closed (on client) > before sending disconnect request. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-63) Race condition in client close() operation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-63?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-63: Attachment: patch_ZOOKEEPER-63.patch This patch avoids the close() method blocking forever. It waits just once, up to the closeTimeout so if the socket is blocked or some other strangeness is going on, the calling thread will only wait up to the timeout (which defaults to 2 seconds). BTW this patch fixes the hang I was having in the test case to ZOOKEEPER-78 > Race condition in client close() operation > -- > > Key: ZOOKEEPER-63 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-63 > Project: Zookeeper > Issue Type: Bug > Components: java client >Reporter: Patrick Hunt >Assignee: Benjamin Reed > Attachments: patch_ZOOKEEPER-63.patch > > > There is a race condition in the java close operation on ZooKeeper.java. > Client is sending a disconnect request to the server. Server will close any > open connections with the client when it receives this. If the client has not > yet shutdown it's subthreads (event/send threads for example) these threads > may consider the condition an error. We see this alot in the tests where the > clients output error logs because they are unaware that a disconnection has > been requested by the client. > Ben mentioned: perhaps we just have to change state to closed (on client) > before sending disconnect request. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-63) Race condition in client close() operation
[
https://issues.apache.org/jira/browse/ZOOKEEPER-63?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616151#action_12616151
]
james strachan commented on ZOOKEEPER-63:
-
BTW here's the hang I seem to be able to get quite easily using the test case
WriteLockTest in the ZOOKEEPER-78 patch (you need to set
workAroundClosingLastZNodeFails to false to make it hang)
{code}
[junit] "main" prio=5 tid=0x01001710 nid=0xb0801000 in
Object.wait() [0xb07ff000..0xb0800148]
[junit] at java.lang.Object.wait(Native Method)
[junit] - waiting on <0x096105e0> (a
org.apache.zookeeper.ClientCnxn$Packet)
[junit] at java.lang.Object.wait(Object.java:474)
[junit] at
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:822)
[junit] - locked <0x096105e0> (a org.apache.zookeeper.ClientCnxn$Packet)
[junit] at org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:329)
[junit] - locked <0x0bd54108> (a org.apache.zookeeper.ZooKeeper)
[junit] at
org.apache.zookeeper.protocols.ZooKeeperFacade.close(ZooKeeperFacade.java:99)
[junit] at
org.apache.zookeeper.protocols.WriteLockTest.tearDown(WriteLockTest.java:146)
[junit] at junit.framework.TestCase.runBare(TestCase.java:140)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
[junit] at junit.framework.TestResult.run(TestResult.java:113)
[junit] at junit.framework.TestCase.run(TestCase.java:124)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:232)
[junit] at junit.framework.TestSuite.run(TestSuite.java:227)
[junit] at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81)
[junit] at
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:36)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:421)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:912)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:766)
{code}
> Race condition in client close() operation
> --
>
> Key: ZOOKEEPER-63
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-63
> Project: Zookeeper
> Issue Type: Bug
> Components: java client
>Reporter: Patrick Hunt
>Assignee: Benjamin Reed
>
> There is a race condition in the java close operation on ZooKeeper.java.
> Client is sending a disconnect request to the server. Server will close any
> open connections with the client when it receives this. If the client has not
> yet shutdown it's subthreads (event/send threads for example) these threads
> may consider the condition an error. We see this alot in the tests where the
> clients output error logs because they are unaware that a disconnection has
> been requested by the client.
> Ben mentioned: perhaps we just have to change state to closed (on client)
> before sending disconnect request.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-85) register the ZooKeeper mailing lists with nabble.com
register the ZooKeeper mailing lists with nabble.com Key: ZOOKEEPER-85 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-85 Project: Zookeeper Issue Type: Task Reporter: james strachan Assignee: Patrick Hunt Many apache projects including Hadoop register with nabble to host online forums & great online archives of the mailing lists... http://www.nabble.com/Hadoop-f17066.html Currently there's hadoop-core, hbase and lucene on there. I often refer to mailing list posts by nabble link; they're really handy. Plus end users often prefer the forum style nabble approach to getting every single email sent to a mailing list. Does a committer on zookeeper fancy registering the ZK mailing lists too (as a child of the Hadoop list)? I'd do it myself but then I'd be the owner of the forum which doesn't feel right - a committer should probably do it. Its a pretty quick process, click here http://n2.nabble.com/more/MailingListRequest.jtp and fill out the details. The hardest part is knowing the mailing list software which AFAIK is ezmlm :) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-83) Switch to using maven to build ZooKeeper
[ https://issues.apache.org/jira/browse/ZOOKEEPER-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616149#action_12616149 ] james strachan commented on ZOOKEEPER-83: - So long as dependencies don't change much and the build process doesn't undergo any major radical changes, its not gonna require much in the way of maintenance keeping the maven build going. (BTW Maven can auto-generate an Ant build too which uses the Maven ant tasks to work with dependencies and repos). Am sure Hiram or myself would happily keep the maven build ticking along. So its zero overhead to the ZK team. In terms of Ivy; yeah I've heard it can work with maven repos; but I've never worked on a project that really uses it - it just seams much easier if you wanna work with maven metadata, dependencies, hierarchical projects and repositories to just use Maven :). There's a zillion issues with keeping your metadata up to date on compile/optional/test dependencies and dealing with hierarchial projects and transitive dependency issues - so I've no idea what issues you'd run into using Ivy - over the years we've bumped into a zillion issues with Maven :( so I certainly understand the maven resistence. Everyone has a love/hate relationship with it - until you try to re-implement what it does in Ant and you end up with some kinda grudging respect for it :). (Especially if you ever have to do any OSGi work!) But sure if you guys wanna figure out the Ivy route so it'd be easy to build, test and install the ZK jars into a local or remote Maven repo, I'd be really happy! Given both Hiram and myself have been really heavy Maven users for many years (after being heavy Ant users before then) I doubt you'll be getting any Ant+Ivy patches for the build system from either of us though :). Heck if the Maven build only lives around until someone figures out how to replace it with Ant+Ivy I think we'd all be happy; so starting with a maven build then replacing it later if someone can figure out how to do it with Ant+Ivy sounds a reasonable approach. (And we could always keep the maven build ticking along until the Ant+Ivy approach really does work). > Switch to using maven to build ZooKeeper > > > Key: ZOOKEEPER-83 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-83 > Project: Zookeeper > Issue Type: Improvement > Components: build >Reporter: Hiram Chirino >Assignee: Hiram Chirino > Attachments: zookeeper-mavened.tgz > > > Maven is a great too for building java projects at the ASF. It helps > standardize the build a bit since it's a convention oriented. > It's dependency auto downloading would remove the need to store the > dependencies in svn, and it will handle many of the suggested ASF policies > like gpg signing of the releases and such. > The ZooKeeper build is almost vanilla except for the jute compiler bits. > Things that would need to change are: > * re-organize the source tree a little so that it uses the maven directory > conventions > * seperate the jute bits out into seperate modules so that a maven plugin > can be with it > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-84) provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException
[ https://issues.apache.org/jira/browse/ZOOKEEPER-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616144#action_12616144 ] james strachan commented on ZOOKEEPER-84: - If ZOOKEEPER-78 ever gets committed (hint, hint :) we can just refer folks to the ZooKeeperFacade if ever folks hit the SessionExpiredException > provide a mechanism to reconnect a ZooKeeper if a client receives a > SessionExpiredException > --- > > Key: ZOOKEEPER-84 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-84 > Project: Zookeeper > Issue Type: Improvement > Components: java client >Reporter: james strachan >Assignee: Benjamin Reed > Attachments: reconnect_patch.patch > > > am about to attach a patch which adds a reconnect() method to easily > re-establish a connection if a session expires - along with a toString() > implementation for easier debugging -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-63) Race condition in client close() operation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-63?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616136#action_12616136 ] james strachan commented on ZOOKEEPER-63: - I wonder if I've seen this too - I can reliably get a hung test when trying to close a client (though the server is still up at the point if the hang). I'm thinking the close() method should not wait() forever on the disconnect packet, just a closeTimeout length - say a few seconds. Afterall blocking and forcing a reconnect just to redeliver the disconnect packet seems a bit silly - when the server has to deal with clients which just have their sockets fail anyway :) > Race condition in client close() operation > -- > > Key: ZOOKEEPER-63 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-63 > Project: Zookeeper > Issue Type: Bug > Components: java client >Reporter: Patrick Hunt >Assignee: Benjamin Reed > > There is a race condition in the java close operation on ZooKeeper.java. > Client is sending a disconnect request to the server. Server will close any > open connections with the client when it receives this. If the client has not > yet shutdown it's subthreads (event/send threads for example) these threads > may consider the condition an error. We see this alot in the tests where the > clients output error logs because they are unaware that a disconnection has > been requested by the client. > Ben mentioned: perhaps we just have to change state to closed (on client) > before sending disconnect request. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-78: Assignee: (was: james strachan) Status: Patch Available (was: Open) Patch is now attached > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan > Attachments: patch_with_including_Benjamin's_fix.patch, > using_zookeeper_facade.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[
https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
james strachan updated ZOOKEEPER-78:
Attachment: using_zookeeper_facade.patch
This patch no longer requires ZOOKEEPER-84, we now use a ZooKeeperFacade which
wraps up the creation of the ZooKeeper instance and allows it to be replaced if
a SessionExpiredException occurs.
The test case works in the current patch. To get the test case to hang closing
the 3rd client, just edit WriteLockTest and set the
workAroundClosingLastZNodeFails field to a value of false. You will then get
this stack dump when the test hangs (on OS X at least :)...
{code}
[junit] "main" prio=5 tid=0x01001710 nid=0xb0801000 in Object.wait()
[0xb07ff000..0xb0800148]
[junit] at java.lang.Object.wait(Native Method)
[junit] - waiting on <0x096105e0> (a
org.apache.zookeeper.ClientCnxn$Packet)
[junit] at java.lang.Object.wait(Object.java:474)
[junit] at
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:822)
[junit] - locked <0x096105e0> (a org.apache.zookeeper.ClientCnxn$Packet)
[junit] at org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:329)
[junit] - locked <0x0bd54108> (a org.apache.zookeeper.ZooKeeper)
[junit] at
org.apache.zookeeper.protocols.ZooKeeperFacade.close(ZooKeeperFacade.java:99)
[junit] at
org.apache.zookeeper.protocols.WriteLockTest.tearDown(WriteLockTest.java:146)
[junit] at junit.framework.TestCase.runBare(TestCase.java:140)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
[junit] at junit.framework.TestResult.run(TestResult.java:113)
[junit] at junit.framework.TestCase.run(TestCase.java:124)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:232)
[junit] at junit.framework.TestSuite.run(TestSuite.java:227)
[junit] at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81)
[junit] at
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:36)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:421)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:912)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:766)
{code}
This could maybe just be an OS X based timing issue
> added a high level protocol/feature - for easy Leader Election or exclusive
> Write Lock creation
> ---
>
> Key: ZOOKEEPER-78
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78
> Project: Zookeeper
> Issue Type: New Feature
> Components: java client
>Affects Versions: 3.0.0
>Reporter: james strachan
>Assignee: james strachan
> Attachments: patch_with_including_Benjamin's_fix.patch,
> using_zookeeper_facade.patch
>
>
> Here's a patch which adds a little WriteLock helper class for performing
> leader elections or creating exclusive locks in some directory znode. Note
> its an early cut; am sure we can improve it over time. The aim is to avoid
> folks having to use the low level ZK stuff but provide a simpler high level
> abstraction.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-84) provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException
[ https://issues.apache.org/jira/browse/ZOOKEEPER-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616114#action_12616114 ] james strachan commented on ZOOKEEPER-84: - You can mark this issue as RESOLVED/WILL_NOT_FIX if you like now - I've implemented a ZooKeeperFacade to wrap up the reconnectWithNewSession() logic for ZOOKEEPER-78 > provide a mechanism to reconnect a ZooKeeper if a client receives a > SessionExpiredException > --- > > Key: ZOOKEEPER-84 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-84 > Project: Zookeeper > Issue Type: Improvement > Components: java client >Reporter: james strachan >Assignee: james strachan > Attachments: reconnect_patch.patch > > > am about to attach a patch which adds a reconnect() method to easily > re-establish a connection if a session expires - along with a toString() > implementation for easier debugging -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-84) provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException
[ https://issues.apache.org/jira/browse/ZOOKEEPER-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616065#action_12616065 ] james strachan commented on ZOOKEEPER-84: - I hear you :) So an Elect Leader or Write Lock protocol has to deal with expired sessions and create new sessions; at some point someone has to recreate something. You can pass the buck and say we're not gonna allow the ZooKeeper to reconnect. Then say we're not allowed to have the WriteLock reconnect, then the next and next layer of the onion. But eventually there's gonna be something somewhere that recreates a session :) For now I'll work on the assumption we're gonna have to have an object which is a wrapper around a ZooKeeper so that it can handle reconnections by just discarding one ZooKeeper instance and creating another. This object could be shared across Protocols (we might wanna reuse one connection with ZK to make multiple locks for example). > provide a mechanism to reconnect a ZooKeeper if a client receives a > SessionExpiredException > --- > > Key: ZOOKEEPER-84 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-84 > Project: Zookeeper > Issue Type: Improvement > Components: java client >Reporter: james strachan >Assignee: james strachan > Attachments: reconnect_patch.patch > > > am about to attach a patch which adds a reconnect() method to easily > re-establish a connection if a session expires - along with a toString() > implementation for easier debugging -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-84) provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException
[ https://issues.apache.org/jira/browse/ZOOKEEPER-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616012#action_12616012 ] james strachan commented on ZOOKEEPER-84: - the ZooKeeper will reconnect by default if the socket fails right? So I'm only really talking about reconectingWithNewSession() in those rare cases that the ZK server times out the session > provide a mechanism to reconnect a ZooKeeper if a client receives a > SessionExpiredException > --- > > Key: ZOOKEEPER-84 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-84 > Project: Zookeeper > Issue Type: Improvement > Components: java client >Reporter: james strachan >Assignee: james strachan > Attachments: reconnect_patch.patch > > > am about to attach a patch which adds a reconnect() method to easily > re-establish a connection if a session expires - along with a toString() > implementation for easier debugging -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-84) provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException
[ https://issues.apache.org/jira/browse/ZOOKEEPER-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615957#action_12615957 ] james strachan commented on ZOOKEEPER-84: - I'm kinda confused by that :) So what am I meant to do if I get a SessionExpiredException? I want to reconnect with a new session; as the current connection is totally useless and invalid. Right now the only option I can see is to recreate from scratch the entire ZooKeeper object? If so that means we've gotta introduce a ZooKeeperProxy thingy which wraps the ZooKeeper object and just lets it be recreated if a SessionExpiredException occurs. Seems kinda unnecessary when really a ZooKeeper instance is capable of easily recreating the connection to the ZK server when the session is expired. Maybe the issue is the phrase reconnect() - maybe a method called recreateNewSession() is better? We could also document it that this is only to be called if your client becomes invalid because the ZK server has expired the session? > provide a mechanism to reconnect a ZooKeeper if a client receives a > SessionExpiredException > --- > > Key: ZOOKEEPER-84 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-84 > Project: Zookeeper > Issue Type: Improvement > Components: java client >Reporter: james strachan >Assignee: james strachan > Attachments: reconnect_patch.patch > > > am about to attach a patch which adds a reconnect() method to easily > re-establish a connection if a session expires - along with a toString() > implementation for easier debugging -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-83) Switch to using maven to build ZooKeeper
[ https://issues.apache.org/jira/browse/ZOOKEEPER-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615954#action_12615954 ] james strachan commented on ZOOKEEPER-83: - Note its pretty trivial to maintain an Ant build as well as a Maven build if folks really have an aversion to Maven. There's really no reason at all to disallow a maven build from being created; they can both happily coexist - and its also a pretty trivial bit of work - not some huge bit of R&D thats gonna slow down development on other things. Also note that a non committer has already contributed the patch already for you - so no more work is required other than committing it :) Its worth remembering that pretty much all popular Java software at Apache is now released into a Maven repository... http://people.apache.org/repo/m2-ibiblio-rsync-repository/org/apache/ so if folks stick with Ant and refuse to allow a maven build to coexist with the Ant build, someone should volunteer to figure out how to hack the Ant build to release ZooKeeper into the Apache maven repository - otherwise its pretty hard for folks who do use maven to reuse your code (and see from the repo how many Apache projects we're talking about not being able to easily reuse ZooKeeper). i.e. as part of the move to the ASF and being a good Apache citizen, I'd recommend hugely that as a minimum ZooKeeper releases its jars into the Apache Maven repository like most other projects do. The easiest way to do this is just to reuse a Maven build to do releases (there doesn't yet seem to be anything in the Ant build to do actual signed releases or deploy builds anywhere) - and let folks who prefer Ant to stick to that for day to day development. The easier it is to reuse a project, the more likely it'll get used and the more likely you'll get contributions; at least thats my experience at Apache. That doesn't mean you have to force your Ant-loving developers to switch build tools! > Switch to using maven to build ZooKeeper > > > Key: ZOOKEEPER-83 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-83 > Project: Zookeeper > Issue Type: Improvement > Components: build >Reporter: Hiram Chirino >Assignee: Hiram Chirino > Attachments: zookeeper-mavened.tgz > > > Maven is a great too for building java projects at the ASF. It helps > standardize the build a bit since it's a convention oriented. > It's dependency auto downloading would remove the need to store the > dependencies in svn, and it will handle many of the suggested ASF policies > like gpg signing of the releases and such. > The ZooKeeper build is almost vanilla except for the jute compiler bits. > Things that would need to change are: > * re-organize the source tree a little so that it uses the maven directory > conventions > * seperate the jute bits out into seperate modules so that a maven plugin > can be with it > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-78: Attachment: patch_with_including_Benjamin's_fix.patch this modified patch includes an implementation of Benjamin's algorithm, using w-sessionId-sequenceNumber as a naming convention so that we can reuse files created for the same session if we get a connection failure. this patch also includes a unit test which tests out the WriteLock still working if we stop and start the server. also the code is refactored so that all the retry logic is in ProtocolSupport > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan >Assignee: james strachan > Attachments: patch_with_including_Benjamin's_fix.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-78: Attachment: (was: writeLock_protocol_version3.patch) > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan >Assignee: james strachan > Attachments: patch_with_including_Benjamin's_fix.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-84) provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException
[ https://issues.apache.org/jira/browse/ZOOKEEPER-84?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-84: Attachment: reconnect_patch.patch sorry I forgot to add the patch before :) here it is now, hopefully this will make more sense now :) > provide a mechanism to reconnect a ZooKeeper if a client receives a > SessionExpiredException > --- > > Key: ZOOKEEPER-84 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-84 > Project: Zookeeper > Issue Type: Improvement > Components: java client >Reporter: james strachan > Attachments: reconnect_patch.patch > > > am about to attach a patch which adds a reconnect() method to easily > re-establish a connection if a session expires - along with a toString() > implementation for easier debugging -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-84) provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException
[ https://issues.apache.org/jira/browse/ZOOKEEPER-84?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-84: Status: Patch Available (was: Open) about to submit a match - whoops forgot to add it :) > provide a mechanism to reconnect a ZooKeeper if a client receives a > SessionExpiredException > --- > > Key: ZOOKEEPER-84 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-84 > Project: Zookeeper > Issue Type: Improvement > Components: java client >Reporter: james strachan > > am about to attach a patch which adds a reconnect() method to easily > re-establish a connection if a session expires - along with a toString() > implementation for easier debugging -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-84) provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException
provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException --- Key: ZOOKEEPER-84 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-84 Project: Zookeeper Issue Type: Improvement Components: java client Reporter: james strachan am about to attach a patch which adds a reconnect() method to easily re-establish a connection if a session expires - along with a toString() implementation for easier debugging -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615618#action_12615618 ] james strachan commented on ZOOKEEPER-78: - Great catch Benjamin! I've a working patch using your algorithm; am using x-sessionId-sequenceNumber and its working a treat (though its a tad hard to force ZK to fail mid-create :). Am working on some unit tests to try out the server stopping/starting which I'll attach shortly once they're working a bit better... > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan >Assignee: james strachan > Attachments: writeLock_protocol_version3.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-83) Switch to using maven to build ZooKeeper
[
https://issues.apache.org/jira/browse/ZOOKEEPER-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615546#action_12615546
]
james strachan commented on ZOOKEEPER-83:
-
I just took a look at the patch; basically the maven conventions are to put
source in ${module-name}/src/main/java and tests in
${module-name}/src/test/java and resources in src/main/resources etc.
Plus it looks like hiram's split the project into multiple maven modules. (e.g.
so that the Java 6 JMX code is a separate module so that the core of zookeeper
can be used on Java 5 - which is a good thing IMHO - plus separating the jute
stuff so it can be used in development time to generate code etc). Its also
easy to generate an uber-jar if folks want later on.
This patch looks good to me - assuming folks are happy to go the maven route
(which many other apache projects do btw - it certainly makes it much easier
for zookeeper to be reused by other maven projects).
If this patch gets applied I'll happily volunteer to refactor my
recipes/protocols patch to create a zookeeper-protocols module to create a
separate jar for higher level stuff
> Switch to using maven to build ZooKeeper
>
>
> Key: ZOOKEEPER-83
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-83
> Project: Zookeeper
> Issue Type: Improvement
> Components: build
>Reporter: Hiram Chirino
>Assignee: Hiram Chirino
> Attachments: zookeeper-mavened.tgz
>
>
> Maven is a great too for building java projects at the ASF. It helps
> standardize the build a bit since it's a convention oriented.
> It's dependency auto downloading would remove the need to store the
> dependencies in svn, and it will handle many of the suggested ASF policies
> like gpg signing of the releases and such.
> The ZooKeeper build is almost vanilla except for the jute compiler bits.
> Things that would need to change are:
> * re-organize the source tree a little so that it uses the maven directory
> conventions
> * seperate the jute bits out into seperate modules so that a maven plugin
> can be with it
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614748#action_12614748 ] james strachan commented on ZOOKEEPER-78: - BTW I just deleted the other 2 patches to avoid confusion; the latest patch includes the previous changes etc > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan >Assignee: james strachan > Attachments: writeLock_protocol_version3.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-78: Attachment: writeLock_protocol_version3.patch Here is an improved version. * we use more optimal comparison by using a ZNodeName object which caches the prefix & sequence number for ordering node names. We can also use this to order node names using different prefixes - maybe useful for read/write locks * fixed a bug and enhanced the test case so that we now test that a leader is established; then when that leader fails another leader/owner is created > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan >Assignee: james strachan > Attachments: writeLock_protocol_version3.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-78: Attachment: (was: writeLock_protocol_with_documentation-version2.patch) > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan >Assignee: james strachan > Attachments: writeLock_protocol_version3.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-78: Attachment: (was: writeLock_protocol.patch) > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan >Assignee: james strachan > Attachments: writeLock_protocol_version3.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-78: Attachment: writeLock_protocol_with_documentation-version2.patch here's an updated patch which has better documentation and includes the recipe documentation linked to from the javadoc - but which could be used stand alone as well if required. I've also included the description from ZOOKEEPER-79 as well > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan >Assignee: james strachan > Attachments: writeLock_protocol.patch, > writeLock_protocol_with_documentation-version2.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614703#action_12614703 ] james strachan commented on ZOOKEEPER-78: - Thanks Flavio! Totally agreed on 1. Strictly speaking we should catch all exceptions and handle them properly (which may mean throwing some, or responding properly to others or whatever). One of the main reasons for the retry logic was to avoid errors like trying to create a znode that already exists or loosing connection to the ZK server etc - but we should go through all possible exceptions and handle them much cleaner. In particular we really need test cases that show the server closing and restarting during the process of acquiring the lock or after a lock owner has the lock etc. I figured I'd send a patch first and see if anyone else had a better implementation lying around - or knew a neater way to solve this - before I spent too much time getting it totally correct etc. For 2) I just added that so that when running the unit tests you could see INFO or DEBUG level logging etc (particularly when running in your IDE) > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan >Assignee: james strachan > Attachments: writeLock_protocol.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-79) Document jacob's leader election on the wiki recipes page
[
https://issues.apache.org/jira/browse/ZOOKEEPER-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614448#action_12614448
]
james strachan commented on ZOOKEEPER-79:
-
Ah cool :) Was just checking we were not about to do the same thing separate :).
I've basically followed the same algorithm from the wiki recipe - and the same
one described in the ZooKeeper tutorial...
http://developer.yahoo.com/blogs/hadoop/2008/03/intro-to-zookeeper-video.html
So AFAIK yes its the same
> Document jacob's leader election on the wiki recipes page
> -
>
> Key: ZOOKEEPER-79
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-79
> Project: Zookeeper
> Issue Type: New Feature
> Components: documentation
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
>
> The following discussion occurred on the zookeeper-user list. We need to
> formalize this recipe and document on the wiki recipes page:
> -from jacob
> Avinash
>
> The following protocol will help you fix the observed misbehavior. As Flavio
> points out, you cannot rely on the order of nodes in getChildren, you must
> use an intrinsic property of each node to determine who is the leader. The
> protocol devised by Runping Qi and described here will do that.
>
> First of all, when you create child nodes of the node that holds the
> leadership bids, you must create them with the EPHEMERAL and SEQUENCE flag.
> ZooKeeper guarantees to give you an ephemeral node named uniquely and with a
> sequence number larger by at least one than any previously created node in
> the sequence. You provide a prefix, like "L_" or your own choice, and
> ZooKeeper creates nodes named "L_23", "L_24", etc. The sequence number starts
> at 0 and increases monotonously.
>
> Once you've placed your leadership bid, you search backwards from the
> sequence number of *your* node to see if there are any preceding (in terms of
> the sequence number) nodes. When you find one, you place a watch on it and
> wait for it to disappear. When you get the watch notification, you search
> again, until you do not find a preceding node, then you know you're the
> leader. This protocol guarantees that there is at any time only one node that
> thinks it is the leader. But it does not disseminate information about who is
> the leader. If you want everyone to know who is the leader, you can have an
> additional Znode whose value is the name of the current leader (or some
> identifying information on how to contact the leader, etc.). Note that this
> cannot be done atomically, so by the time other nodes find out who the leader
> is, the leadership may already have passed on to a different node.
>
> Flavio
>
> Might it make sense to provide a standardized implementation of leader
> election in the library code in Java?
>
> --Jacob
>
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Flavio
> Junqueira
> Sent: Friday, July 11, 2008 1:02 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: [Zookeeper-user] Leader election
>
> Hi Avinash, getChildren returns a list in lexicographic order, so if you are
> updating the children of the election node concurrently, then you may get a
> different first node with different clients. If you are using the sequence
> flag to create nodes, then you may consider stripping the prefix of the node
> name and using the sufix value to determine order.
> Hope it helps.
> -Flavio
>
> - Original Message
> From: Avinash Lakshman <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Friday, July 11, 2008 7:20:06 AM
> Subject: [Zookeeper-user] Leader election
> Hi
> I am trying to elect leader among 50 nodes. There is always one odd guy who
> seems to think that someone else distinct from what some other nodes see as
> leader. Could someone please tell me what is wrong with the following code
> for leader election:
> public void electLeader()
> {
> ZooKeeper zk = StorageService.instance().getZooKeeperHandle();
> String path = "/Leader";
> try
> {
> String createPath = path + "/L-";
>
> LeaderElector.createLock_.lock();
> while( true )
> {
> /* Get all znodes under the Leader znode */
> List values = zk.getChildren(path, false);
> /*
> * Get the first znode and if it is the
> * pathCreated created above then the data
> * in that znode is the leader's identity.
> */
> if ( leader_ == null )
> {
>
[jira] Commented: (ZOOKEEPER-79) Document jacob's leader election on the wiki recipes page
[
https://issues.apache.org/jira/browse/ZOOKEEPER-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614431#action_12614431
]
james strachan commented on ZOOKEEPER-79:
-
BTW https://issues.apache.org/jira/browse/ZOOKEEPER-78 contains a patch of just
that :)
> Document jacob's leader election on the wiki recipes page
> -
>
> Key: ZOOKEEPER-79
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-79
> Project: Zookeeper
> Issue Type: New Feature
> Components: documentation
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
>
> The following discussion occurred on the zookeeper-user list. We need to
> formalize this recipe and document on the wiki recipes page:
> -from jacob
> Avinash
>
> The following protocol will help you fix the observed misbehavior. As Flavio
> points out, you cannot rely on the order of nodes in getChildren, you must
> use an intrinsic property of each node to determine who is the leader. The
> protocol devised by Runping Qi and described here will do that.
>
> First of all, when you create child nodes of the node that holds the
> leadership bids, you must create them with the EPHEMERAL and SEQUENCE flag.
> ZooKeeper guarantees to give you an ephemeral node named uniquely and with a
> sequence number larger by at least one than any previously created node in
> the sequence. You provide a prefix, like "L_" or your own choice, and
> ZooKeeper creates nodes named "L_23", "L_24", etc. The sequence number starts
> at 0 and increases monotonously.
>
> Once you've placed your leadership bid, you search backwards from the
> sequence number of *your* node to see if there are any preceding (in terms of
> the sequence number) nodes. When you find one, you place a watch on it and
> wait for it to disappear. When you get the watch notification, you search
> again, until you do not find a preceding node, then you know you're the
> leader. This protocol guarantees that there is at any time only one node that
> thinks it is the leader. But it does not disseminate information about who is
> the leader. If you want everyone to know who is the leader, you can have an
> additional Znode whose value is the name of the current leader (or some
> identifying information on how to contact the leader, etc.). Note that this
> cannot be done atomically, so by the time other nodes find out who the leader
> is, the leadership may already have passed on to a different node.
>
> Flavio
>
> Might it make sense to provide a standardized implementation of leader
> election in the library code in Java?
>
> --Jacob
>
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Flavio
> Junqueira
> Sent: Friday, July 11, 2008 1:02 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: [Zookeeper-user] Leader election
>
> Hi Avinash, getChildren returns a list in lexicographic order, so if you are
> updating the children of the election node concurrently, then you may get a
> different first node with different clients. If you are using the sequence
> flag to create nodes, then you may consider stripping the prefix of the node
> name and using the sufix value to determine order.
> Hope it helps.
> -Flavio
>
> - Original Message
> From: Avinash Lakshman <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Friday, July 11, 2008 7:20:06 AM
> Subject: [Zookeeper-user] Leader election
> Hi
> I am trying to elect leader among 50 nodes. There is always one odd guy who
> seems to think that someone else distinct from what some other nodes see as
> leader. Could someone please tell me what is wrong with the following code
> for leader election:
> public void electLeader()
> {
> ZooKeeper zk = StorageService.instance().getZooKeeperHandle();
> String path = "/Leader";
> try
> {
> String createPath = path + "/L-";
>
> LeaderElector.createLock_.lock();
> while( true )
> {
> /* Get all znodes under the Leader znode */
> List values = zk.getChildren(path, false);
> /*
> * Get the first znode and if it is the
> * pathCreated created above then the data
> * in that znode is the leader's identity.
> */
> if ( leader_ == null )
> {
> leader_ = new AtomicReference(
> EndPoint.fromBytes( zk.getData(path + "/" + values.get(0), false, null) ) );
> }
> else
> {
> lead
[jira] Created: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation --- Key: ZOOKEEPER-78 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 Project: Zookeeper Issue Type: New Feature Components: java client Affects Versions: 3.0.0 Reporter: james strachan Attachments: writeLock_protocol.patch Here's a patch which adds a little WriteLock helper class for performing leader elections or creating exclusive locks in some directory znode. Note its an early cut; am sure we can improve it over time. The aim is to avoid folks having to use the low level ZK stuff but provide a simpler high level abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] james strachan updated ZOOKEEPER-78: Attachment: writeLock_protocol.patch > added a high level protocol/feature - for easy Leader Election or exclusive > Write Lock creation > --- > > Key: ZOOKEEPER-78 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78 > Project: Zookeeper > Issue Type: New Feature > Components: java client >Affects Versions: 3.0.0 >Reporter: james strachan > Attachments: writeLock_protocol.patch > > > Here's a patch which adds a little WriteLock helper class for performing > leader elections or creating exclusive locks in some directory znode. Note > its an early cut; am sure we can improve it over time. The aim is to avoid > folks having to use the low level ZK stuff but provide a simpler high level > abstraction. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
