[jira] Commented: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12934870#action_12934870 ] Thomas Koch commented on ZOOKEEPER-836: --- Hi Benjamin, we can not call it HostSet, because the new class I added isn't a HostSet, it's a parser for a connectString. I don't know, what the HostSet you suggest should look like. What's the difference between a List or Set of InetSocketAddress and a HostSet? - Should there be any methods on HostSet appart from the normal List/Set methods? - Should there be a reference to HostSet outside of ClientCnxn to manipulate HostSet after it has been given to ClientCnxn? - Should the elements of HostSet be instances of InetSocketAddress or just pairs of hostname and port? In that case, do we need to prepare for anything special in regard to IPv6 support? This issue had to purposes for me: - Get rid of connectionString parsing in ClientCnxn. - Separate the HostSet from the chrootPath. IMHO these two things don't belong together and it was maybe a mistake from the beginning to let the user give the chrootPath inside of the connectString. After this patch, I'd like to prepare the next three commits: - ZOOKEEPER-849: Provide a Path class - This will also affect the ConnectStringParser, because I'd like it to return a Path instance instead of a String - ZOOKEEPER-838: Chroot is an attribute of ClientCnxn - It is possible, that only the ZooKeeper object knows about the chroot and ClientCnxn doesn't need to have any knowledge about it. Thus it can become possible in the long run to have multiple ZooKeeper objects instantiated using the same ClientCnxn but with different Chroots. - ZOOKEEPER-837: cyclic dependency ClientCnxn, ZooKeeper - Of course to have multiple ZooKeeper objects we have to get rid of the ClientCnxn-ZooKeeper dependency. hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Attachments: ZOOKEEPER-836.patch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-836: -- Attachment: ZOOKEEPER-836.patch hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Attachments: ZOOKEEPER-836.patch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch reassigned ZOOKEEPER-836: - Assignee: Thomas Koch hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Attachments: ZOOKEEPER-836.patch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-836: -- Status: Patch Available (was: Open) hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Attachments: ZOOKEEPER-836.patch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (ZOOKEEPER-895) ClientCnxn.authInfo must be thread safe
[ https://issues.apache.org/jira/browse/ZOOKEEPER-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch resolved ZOOKEEPER-895. --- Resolution: Fixed Has been solved as part of ZOOKEEPER-909 by using CopyOnWriteArraySet. ClientCnxn.authInfo must be thread safe --- Key: ZOOKEEPER-895 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-895 Project: Zookeeper Issue Type: Bug Reporter: Thomas Koch authInfo can be accessed concurrently by different Threads, as exercised in org.apache.zookeeper.test.ACLTest The two concurrent access points in this case were (presumably): org.apache.zookeeper.ClientCnxn$SendThread.primeConnection(ClientCnxn.java:805) and org.apache.zookeeper.ClientCnxn.addAuthInfo(ClientCnxn.java:1121) The line numbers refer to the latest patch in ZOOKEEPER-823. The exception that pointed to this issue: [junit] 2010-10-13 09:35:55,113 [myid:] - WARN [main-SendThread(localhost:11221):clientcnxn$sendthr...@713] - Session 0x0 for server localhost/127.0.0.1:11221, unexpected error, closing socket connection and attempting reconnect [junit] java.util.ConcurrentModificationException [junit] at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) [junit] at java.util.AbstractList$Itr.next(AbstractList.java:343) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.primeConnection(ClientCnxn.java:805) [junit] at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:247) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:694) Proposed solution: Use a thread save list for authInfo -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-836: -- Attachment: (was: ZOOKEEPER-836.patch) hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-836: -- Status: Open (was: Patch Available) hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-836: -- Attachment: ZOOKEEPER-836.patch forgot to add a file in the last patch hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Attachments: ZOOKEEPER-836.patch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-836: -- Status: Patch Available (was: Open) hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Attachments: ZOOKEEPER-836.patch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-836: -- Attachment: (was: ZOOKEEPER-836.patch) hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Attachments: ZOOKEEPER-836.patch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-836: -- Attachment: ZOOKEEPER-836.patch now also added apache license headers ... hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Attachments: ZOOKEEPER-836.patch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-836: -- Status: Patch Available (was: Open) hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Attachments: ZOOKEEPER-836.patch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-836) hostlist as string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-836: -- Status: Open (was: Patch Available) hostlist as string -- Key: ZOOKEEPER-836 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-836 Project: Zookeeper Issue Type: Sub-task Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Attachments: ZOOKEEPER-836.patch The hostlist is parsed in the ctor of ClientCnxn. This violates the rule of not doing (too much) work in a ctor. Instead the ClientCnxn should receive an object of class HostSet. HostSet could then be instantiated e.g. with a comma separated string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: FYI: Netty is forking
Patrick Hunt: http://www.jboss.org/netty/community.html#nabble-td5730963 Patrick Hi Patrick, you did shock me. :-) So the news is not, that the netty community would split, but just that netty moves from SVN to GIT. Cheers, Thomas Koch, http://www.koch.ro
[jira] Updated: (ZOOKEEPER-908) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-908: -- Attachment: (was: ZOOKEEPER-908.patch) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation - Key: ZOOKEEPER-908 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-908 Project: Zookeeper Issue Type: Sub-task Components: server Reporter: Thomas Koch Assignee: Thomas Koch Priority: Minor Fix For: 3.4.0 rename record - request (since their is a counterpart record named response) rename header - requestHeader (to distinguish from responseHeader) remove ByteBuffer creation code from primeConnection() method and use the duplicate code in the Packet constructor. Therefor the Bytebuffer bb parameter could also be removed from the constructor's parameters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-908) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-908: -- Attachment: ZOOKEEPER-908.patch new patch against the current Client after ZOOKEEPER-909 There are not new tests in this patch. Please commit it anyway. I'm planning to move the Packet class out to a separate file as part of ZOOKEEPER-894. Then I'd like to do ZOOKEEPER-878. After this, it would be the right time to see if the Packet class needs any tests. I'd like to collect these low hanging fruits and make the Client java stuff more comprehensible before continuing with the Netty stuff in ZOOKEEPER-823 Remove code duplication and inconsistent naming in ClientCnxn.Packet creation - Key: ZOOKEEPER-908 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-908 Project: Zookeeper Issue Type: Sub-task Components: server Reporter: Thomas Koch Assignee: Thomas Koch Priority: Minor Fix For: 3.4.0 Attachments: ZOOKEEPER-908.patch rename record - request (since their is a counterpart record named response) rename header - requestHeader (to distinguish from responseHeader) remove ByteBuffer creation code from primeConnection() method and use the duplicate code in the Packet constructor. Therefor the Bytebuffer bb parameter could also be removed from the constructor's parameters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-908) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-908: -- Status: Patch Available (was: Open) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation - Key: ZOOKEEPER-908 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-908 Project: Zookeeper Issue Type: Sub-task Components: server Reporter: Thomas Koch Assignee: Thomas Koch Priority: Minor Fix For: 3.4.0 Attachments: ZOOKEEPER-908.patch rename record - request (since their is a counterpart record named response) rename header - requestHeader (to distinguish from responseHeader) remove ByteBuffer creation code from primeConnection() method and use the duplicate code in the Packet constructor. Therefor the Bytebuffer bb parameter could also be removed from the constructor's parameters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12930544#action_12930544 ] Thomas Koch commented on ZOOKEEPER-909: --- Dear Hudson, you're terribly annoying me! Looking at your web interface I can't find any failed unit test for this build. And the javadoc warning is due to the acquisition of SUN by Oracle (and the following unavailability of the old sun javadoc websites). - Don't blame me for that! Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-925) Consider maven site generation to replace our forrest site and documentation generation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12930547#action_12930547 ] Thomas Koch commented on ZOOKEEPER-925: --- Please don't check in _any_ generated files in version control. When I package zookeeper for Debian (same applies to any other free software distro) I may not include generated files. Debian needs to make sure that it has all source files for everything that's delivered and that it attributes the copyright correctly. Not having any generated files but to generate everything at Debian package build time makes this a lot easier. Just a note: The same applies for the released tarballs, but that's another topic. Consider maven site generation to replace our forrest site and documentation generation --- Key: ZOOKEEPER-925 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-925 Project: Zookeeper Issue Type: Wish Components: documentation Reporter: Patrick Hunt Assignee: Patrick Hunt Attachments: ZOOKEEPER-925.patch See WHIRR-19 for some background. In whirr we looked at a number of site/doc generation facilities. In the end Maven site generation plugin turned out to be by far the best option. You can see our nascent site here (no attempt at styling,etc so far): http://incubator.apache.org/whirr/ In particular take a look at the quick start: http://incubator.apache.org/whirr/quick-start-guide.html which was generated from http://svn.apache.org/repos/asf/incubator/whirr/trunk/src/site/confluence/quick-start-guide.confluence notice this was standard wiki markup (confluence wiki markup, same as available from apache) You can read more about mvn site plugin here: http://maven.apache.org/guides/mini/guide-site.html Notice that other formats are available, not just confluence markup, also note that you can use different markup formats if you like in the same site (although probably not a great idea, but in some cases might be handy, for example whirr uses the confluence wiki, so we can pretty much copy/paste source docs from wiki to our site (svn) if we like) Re maven vs our current ant based build. It's probably a good idea for us to move the build to maven at some point. We could initially move just the doc generation, and then incrementally move functionality from build.xml to mvn over a longer time period. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Status: Patch Available (was: Open) trying to trigger the hudson build once again to see whether the javadoc issue is still there. Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12930558#action_12930558 ] Thomas Koch commented on ZOOKEEPER-909: --- In our build.xml we still have the old Sun link for javadoc ( http://java.sun.com/javase/6/docs/api/ ). This link redirects to http://download.oracle.com/javase/6/docs/api/ However when trying the old Sun link in my browser I first got a timeout while the second attempt worked. So one possibility would be that I just had bad luck? Or is there any cache between the javadoc task and the Sun website? I've introduced a new class ( java.util.concurrent.CopyOnWriteArraySet ). Maybe this class has not yet been used elsewhere and therefor I need to hit the Sun website while other builds are served from some cache? Anyhow. It would be best, if the build would not need to access the internet at all. I don't now javadoc good enough, but the ant task documentation gives me the impression that it should be possible to build the docs also offline: http://ant.apache.org/manual/Tasks/javadoc.html ( see documentation for the link element ) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Status: Open (was: Patch Available) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Status: Open (was: Patch Available) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Attachment: (was: ZOOKEEPER-909.patch) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Attachment: ZOOKEEPER-909.patch Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Status: Patch Available (was: Open) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12930074#action_12930074 ] Thomas Koch commented on ZOOKEEPER-909: --- 3 out of 3 test runs did pass (as expected :-). Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Attachment: ZOOKEEPER-909.patch resubmitting the last patch without changes just to make hudson happy. Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Attachment: ClientCnxnSocketNetty.java just FYI: This is the ClientCnxnSocketNetty class that should be added after this jira has been applied and should bring Netty support for the java client. Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ClientCnxnSocketNetty.java, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12929395#action_12929395 ] Thomas Koch commented on ZOOKEEPER-909: --- I've added some javadoc and renamed socket to clientCnxnSocket everywhere. I'll upload the patch when the tests completed. Moving ZooKeeper.state to ClientCnxn.state came in handy at some point to avoid unnecessary redirection and to clean the object dependency graph ( ZOOKEEPER-837 ). The state variable is only once accessed from ZooKeeper but several times from ClientCnxn so it makes a lot of sense to move it where it is needed. SessionExpiredException can actually remain private, but EndOfStreamException is used in ClientCnxnSocketNIO. Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Attachment: ZOOKEEPER-909.patch I couldn't wait for the tests to finish now, but it compiles and I did only renames, comments, visibility change and removal of dead code since the last patch. Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Status: Patch Available (was: Open) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Assignee: Patrick Hunt (was: Thomas Koch) Status: Patch Available (was: Open) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Attachment: ZOOKEEPER-909.patch add copyright blocks, replace deprecated ChannelPipelineCoverage annotation Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Status: Open (was: Patch Available) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Heisenbugs, Bohrbugs, Mandelbugs?
Mahadev Konar: Hi Thomas, Could you verify this by just testing the trunk without your patch? You might very well be right that those tests are a little flaky. As for the hudson builds, Nigel is working on getting the patch builds for zookeeper running. As soon as that gets fixed this flaky tests would show up more often. Thanks mahadev On 10/20/10 11:48 PM, Thomas Koch tho...@koch.ro wrote: Hi, last night I let my hudson server do 42 (sic) builds of ZooKeeper trunk. One of this builds failed: junit.framework.AssertionFailedError: Leader hasn't joined: 5 at org.apache.zookeeper.test.FLETest.testLE(FLETest.java:312) I did this many builds of trunk, because in my quest to redo the client netty integration step by step I made one step which resulted in 2 failed builds out of 8. The two failures were both: Hi Mahadev, as I've written, I did 42 builds of trunk over the night from which 2 failed and 8 builds of my patch during work time with 2 failures. I also did another round of builds of my patch during last night and got only 1 failure out of ~40 succesful builds. So I believe that the high failure rate of 2/8 from the initial round of patch builds is because I did this builds over the day while other developers also used other virtual machines on the same host. Have a nice weekend, Thomas Koch, http://www.koch.ro
[jira] Assigned: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch reassigned ZOOKEEPER-909: - Assignee: Thomas Koch (was: Patrick Hunt) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923986#action_12923986 ] Thomas Koch commented on ZOOKEEPER-909: --- Hi Benjamin, you mean, I should add a class javadoc for ClientCnxnSocket and a javadoc for the socket property in ClientCnxn.SendThread? You're right. However I did not yet come to an end with thinking about an elegant structure for the Classes ClientCnxn, SendThread and ClientCnxnSocket. I believe that the ClientCnxnSocket class won't remain for long as it is in this patch. For example SendThread and ClientCnxn have a circular dependency which I really don't like. Also both classes work on the common properties incomingBuffer and outgoingBuffer which is suboptimal. So I'd like to ask for forgiveness for sparse (or inexistent) documentation until we settle on a final design. I also want to start to learn the server code now to see, whether it makes sense to generalize certain things. Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Heisenbugs, Bohrbugs, Mandelbugs?
Hi, last night I let my hudson server do 42 (sic) builds of ZooKeeper trunk. One of this builds failed: junit.framework.AssertionFailedError: Leader hasn't joined: 5 at org.apache.zookeeper.test.FLETest.testLE(FLETest.java:312) I did this many builds of trunk, because in my quest to redo the client netty integration step by step I made one step which resulted in 2 failed builds out of 8. The two failures were both: junit.framework.AssertionFailedError: Threads didn't join at org.apache.zookeeper.test.FLERestartTest.testLERestart(FLERestartTest.java:198) I can't find any relationship between the above test and my changes. The test does not use the ZooKeeper client code at all. So I begin to believe that there are some Heisenbugs, Bohrbugs or Mandelbugs[1] in ZooKeeper that just happen to show up from time to time without any relationship to the current changes. I'll try to investigate the cause further, maybe there is some relationship I've not yet found. But if my assumption should apply, then these kind of bugs would be a strong argument in favor of refactoring. These bugs are best found by cleaning the code, most important implementing strict separation of concerns. Wouldn't you like to setup Hudson to build ZooKeeper trunk every half an hour? [1] http://en.wikipedia.org/wiki/Unusual_software_bug Best regards, Thomas Koch, http://www.koch.ro
[jira] Commented: (ZOOKEEPER-102) Need to replace Jute with supported code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923380#action_12923380 ] Thomas Koch commented on ZOOKEEPER-102: --- Another option would be avro. Does anybody know, whether avro is wire compatible with jute? We would need to support communication between older jute-only Clients/Servers and newer avro(+jute?) Clients/Servers. Would this be a good Google Summer of Code task? Need to replace Jute with supported code Key: ZOOKEEPER-102 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102 Project: Zookeeper Issue Type: Improvement Reporter: Benjamin Reed ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers. Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-908) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation
Remove code duplication and inconsistent naming in ClientCnxn.Packet creation - Key: ZOOKEEPER-908 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-908 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch rename record - request (since their is a counterpart record named response) rename header - requestHeader (to distinguish from responseHeader) remove ByteBuffer creation code from primeConnection() method and use the duplicate code in the Packet constructor. Therefor the Bytebuffer bb parameter could also be removed from the constructor's parameters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-908) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-908: -- Attachment: ZOOKEEPER-908.patch ant test-core did work without failures. Remove code duplication and inconsistent naming in ClientCnxn.Packet creation - Key: ZOOKEEPER-908 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-908 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch Attachments: ZOOKEEPER-908.patch rename record - request (since their is a counterpart record named response) rename header - requestHeader (to distinguish from responseHeader) remove ByteBuffer creation code from primeConnection() method and use the duplicate code in the Packet constructor. Therefor the Bytebuffer bb parameter could also be removed from the constructor's parameters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Patrick Hunt Fix For: 3.4.0 This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Attachment: ZOOKEEPER-909.patch Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Status: Patch Available (was: Open) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-910) Use SelectionKey.isXYZ() methods instead of complicated binary logic
Use SelectionKey.isXYZ() methods instead of complicated binary logic Key: ZOOKEEPER-910 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-910 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch Fix For: 3.4.0 The SelectionKey class provides methods to replace something like this (k.readyOps() (SelectionKey.OP_READ | SelectionKey.OP_WRITE)) != 0 with selectionKey.isReadable() || selectionKey.isWritable() It may be possible, that the first version saves a CPU cycle or two, but the later version saves developer brain cycles which are much more expensive. I suppose that there are many more places in the server code where this replacement could be done. I propose that whoever touches a code line like this should make the replacement. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
What's the magic behind lenBuffer and incomingBuffer?
Hi,
inside ClientCnxn.SendThread we have
final ByteBuffer lenBuffer = ByteBuffer.allocateDirect(4);
ByteBuffer incomingBuffer = lenBuffer;
So incomingBuffer and lenBuffer do refer to the same object. There are several
other places where lenBuffer is again assigned to incomingBuffer.
Now inside the doIO() method we got
if (incomingBuffer == lenBuffer) {
recvCount++;
readLength();
} else if (!initialized) {
incomingBuffer is never assigned anything else then lenBuffer, lenBuffer stays
the same all the time. So as far as my knowledge of java reaches (which may
not be too far) incomingBuffer == lenBuffer _always_ evaluates to true. Isn't
that true?
So effectively we've got dead code in the elseif and else branches, didn't we?
Best regards,
Thomas Koch, http://www.koch.ro
[jira] Created: (ZOOKEEPER-911) move operations from methods to individual classes
move operations from methods to individual classes
--
Key: ZOOKEEPER-911
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-911
Project: Zookeeper
Issue Type: New Feature
Components: java client
Reporter: Thomas Koch
Copied from my email to the ZK dev list from 2010/05/26:
For my current code I'm using zkclient[1] and have also looked at cages[2] for
some ZK usage examples. I observed, that there's a common pattern to wrap ZK
operations in callables and feed them to a retryUntilConnected executor.
Now my idea is, that ZK should already come with operations in classes, e.g.:
o.a.z.operation.Create extends Operation implements callable{
private path, data[], acl, createMode
public Create( .. all kind of ctors .. )
public call(){
.. move code from Zookeeper.create() here
}
}
Similiar classes should be provided for getChildren, delete, exists, getData,
getACL, setACL and setData.
One could then feed such operations to an ZkExecutor, which has the necessary
knowledge about the ZkConnection and can execute a command either
synchronously or asynchronously.
One could also wrap operations in an ExceptionCatcher to ignore certain
Exceptions or in a RetryPolicy.
This is only an idea so far, but I wanted to share my thoughts before starting
to try it out. (BTW: You can meet me at BerlinBuzzwords.de)
[1] http://github.com/sgroschupf/zkclient
[2] http://code.google.com/p/cages/
And a reply from Patrick Hunt at my mail:
Hi Thomas, you might take a look at this JIRA
https://issues.apache.org/jira/browse/ZOOKEEPER-679
there's definitely been interest in this area, however there are some
real challenges as well. Most users do end up wrapping the basic api
with some code, esp the retry metaphor is a common case, so I think it
would be valuable. At the same time getting the semantics right is hard
(and covering all the corner cases). Perhaps you could sync up with
Aaron/Chris, I'd personally like to see this go into contrib, but I
understand the extra burden the patch process presents -- it may make
more sense to rapidly iterate on something like github and then move to
contrib once you have something less frequently changing, where the
patch issue would be less of a problem (see 679, there's discussion on
this there). Regardless which way you take it we'd be happy to work with
you.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-835) Refactoring Zookeeper Client Code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923476#action_12923476 ] Thomas Koch commented on ZOOKEEPER-835: --- Hi Benjamin, I'm sorry, I should have filled ZOOKEEPER-911 sooner. Actually ZOOKEEPER-911 would make it easier to implement any retry logic and ZOOKEEPER-911 depends on some of the sub-issues of this issue. Refactoring Zookeeper Client Code - Key: ZOOKEEPER-835 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-835 Project: Zookeeper Issue Type: Improvement Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Thomas Koch asked me to fill individual issues for the points raised in his mail to zookeeper-dev: [Mail of Thomas Koch| http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-dev/201008.mbox/%3c20100845.17507.tho...@koch.ro%3e ] He published several issues, which are present in the current zookeeper client, so a refactoring of the code would be an facility for other developers working with zookeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Description: This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! was: This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Attachment: ZOOKEEPER-909.patch netty2 could not be found by ivy --- a/ivysettings.xml +++ b/ivysettings.xml @@ -20,7 +20,7 @@ property name=repo.maven.org value=http://repo1.maven.org/maven2/; override=false/ property name=repo.jboss.org -value=http://repository.jboss.com/maven2/; override=false/ +value=http://repository.jboss.org/nexus/content/groups/public/; override=false/ Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
New strategy for Netty (ZOOKEEPER-823) was: What's the QA strategy of ZooKeeper?
Benjamin Reed: actually, the other way of doing the netty patch (since i'm scared of merges) would be to do a refactor cleanup patch with an eye toward netty, and then another patch to actually add netty. [...] Hi Benjamin, I've had exactly the same thought last evening. Instead of trying to find the bug(s) in the current patch, I'd like to start it over again and do small incremental changes from the current trunk towards the current ZOOKEEPER-823 patch. Maybe I could do this in ZOOKEEPER-823 patch, this would mean to revert the already applied ZOOKEEPER-823 patch. Then I want to test each incremental step at least 5 times to find the step(s) that breaks ZK. This approach should take me another two weeks, I believe, mostly because each Test run takes ~15-25 minutes. Cheers, Thomas Koch, http://www.koch.ro
Re: Branch hudson job created for ZOOKEEPER-823
Patrick Hunt: I created a branch in SVN and a new Hudson job to track the ZOOKEEPER-823 patch. Getting this one done has been tough, Thomas has been working on it but we are seeing intermittent failures. lmk if you have any questions. Hi Patrick, thank you for the branch. Is there a way, how I could commit to this branch? Or if sbd else commits on this branch, how should I send patches? If I can commit, is there a way, how I could use GIT to work on this branch? I've not yet used git-svn to do commits. Have a nice weekend, Thomas Koch, http://www.koch.ro
Re: What's the QA strategy of ZooKeeper?
Hi Benjamin, thank you for your response. Please find some comments inline. Benjamin Reed: code quality is important, and there are things we should keep in mind, but in general i really don't like the idea of risking code breakage because of a gratuitous code cleanup. we should be watching out for these things when patches get submitted or when new things go in. I didn't want to say it that clear, but especially the new Netty code, both on client and server side is IMHO an example of new code in very bad shape. The client code patch even changes the FindBugs configuration to exclude the new code from the FindBugs checks. i think this is inline with what pat was saying. just to expand a bit. in my opinion clean up refactorings have the following problems: 1) you risk breaking things in production for a potential future maintenance advantage. If your code is already in such a bad shape, that every change includes considerable risk to break something, then you already are in trouble. With every new feature (or bugfix!) you also risk to break something. If you don't have the attitude of permanent refactoring to improve the code quality, you will inevitably lower the maintainability of your code with every new feature. New features will build on the dirty concepts already in the code and therfor make it more expensive to ever clean things up. 2) there is always subjectivity: quality code for one code quality zealot is often seen as a bad design by another code quality zealot. unless there is an objective reason to do it, don't. I don't agree. IMHO, the area of subjectivism in code quality is actually very small compared to hard established standards of quality metrics and best practices. I believe that my list given in the first mail of this thread gives examples of rather objective guidelines. 3) you may cleanup the wrong way. you may restructure to make the current code clean and then end up rewriting and refactoring again to change the logic. Yes. Refactoring isn't easy, but necessary. Only over time you better understand your domain and find better structures. Over time you introduce features that let code grow so that it should better be split up in smaller units that the human brain can still handle. i think we can mitigate 1) by only doing it when necessary. as a corollary we can mitigate 2) and 3) by only doing refactoring/cleanups when motivated by some new change: fix a bug, increased performance, new feature, etc. I agree that refactoring should be carefully planned and done in small steps. Therefor I collected each refactoring item for the java client in a small separate bug in https://issues.apache.org/jira/browse/ZOOKEEPER-835 that can individually be discussed, reviewed and tested. Have a nice weekend after Hadoop World! Thomas Koch, http://www.koch.ro
[jira] Created: (ZOOKEEPER-895) ClientCnxn.authInfo must be thread safe
ClientCnxn.authInfo must be thread safe --- Key: ZOOKEEPER-895 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-895 Project: Zookeeper Issue Type: Bug Reporter: Thomas Koch authInfo can be accessed concurrently by different Threads, as exercised in org.apache.zookeeper.test.ACLTest The two concurrent access points in this case were (presumably): org.apache.zookeeper.ClientCnxn$SendThread.primeConnection(ClientCnxn.java:805) and org.apache.zookeeper.ClientCnxn.addAuthInfo(ClientCnxn.java:1121) The line numbers refer to the latest patch in ZOOKEEPER-823. The exception that pointed to this issue: [junit] 2010-10-13 09:35:55,113 [myid:] - WARN [main-SendThread(localhost:11221):clientcnxn$sendthr...@713] - Session 0x0 for server localhost/127.0.0.1:11221, unexpected error, closing socket connection and attempting reconnect [junit] java.util.ConcurrentModificationException [junit] at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) [junit] at java.util.AbstractList$Itr.next(AbstractList.java:343) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.primeConnection(ClientCnxn.java:805) [junit] at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:247) [junit] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:694) Proposed solution: Use a thread save list for authInfo -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Status: Patch Available (was: Open) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: ZOOKEEPER-823.patch wrapped access to ClientCnxn.authInfo in synchonized(authInfo). Did 13 builds with 6 failed builds. Failed Tests: org.apache.zookeeper.test.ACLTest.testAcls org.apache.zookeeper.test.ChrootClientTest.testClientCleanup org.apache.zookeeper.test.WatcherTest.testWatchAutoResetWithPending org.apache.zookeeper.test.FLERestartTest.testLERestart org.apache.zookeeper.test.WatcherTest.testWatchAutoResetWithPending org.apache.zookeeper.test.QuorumTest.testFollowersStartAfterLeader org.apache.zookeeper.test.QuorumTest.testNoLogBeforeLeaderEstablishment org.apache.zookeeper.test.QuorumTest.testNull org.apache.zookeeper.test.QuorumZxidSyncTest.testLateLogs What puzzles me, is that also tests totally unrelated to the java client failed. My test machine is also a development server for the whole team, so its load varies heavily. update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: testWatchAutoResetWithPending_FAILURE I did 10 builds now in total with the last patch. 8 builds were succesful, 2 failed with different test cases, see the *_FAILURE files. Both failures happened in the NettyNettySuiteTest. It's hard to debug this kind of Heisenbugs that happen only very seldom. It would make things easier, if I could start cleaning up the code to better identify spots where things could go wrong. update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-894) add Package o.a.Zookeeper.Client
add Package o.a.Zookeeper.Client Key: ZOOKEEPER-894 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-894 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-894) add Package o.a.Zookeeper.Client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-894: -- Description: I'd like to move classes that are not part of the API but belong to the ZK Client into a separate Client package. These classes are: - Inner classes that should become normal classes: Zookeeper.ZkWatchManager Zookeeper.WatchRegistration ClientCnxn.SendThread (should become a Runnable anyhow) ClientCnxn.EventThread ClientCnxn.Package - Classes now in the Zookeeper package: ClientCnxn - Client.Cnxn ClientCnxnSocket* - Client.CnxnSocket* ... Maybe some others that can be moved without breaking the API - Classes yet to be written: PendingQueue (?) OutgoingQueue (?) add Package o.a.Zookeeper.Client Key: ZOOKEEPER-894 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-894 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch I'd like to move classes that are not part of the API but belong to the ZK Client into a separate Client package. These classes are: - Inner classes that should become normal classes: Zookeeper.ZkWatchManager Zookeeper.WatchRegistration ClientCnxn.SendThread (should become a Runnable anyhow) ClientCnxn.EventThread ClientCnxn.Package - Classes now in the Zookeeper package: ClientCnxn - Client.Cnxn ClientCnxnSocket* - Client.CnxnSocket* ... Maybe some others that can be moved without breaking the API - Classes yet to be written: PendingQueue (?) OutgoingQueue (?) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
What's the QA strategy of ZooKeeper?
Hi, after filling 13 refactoring issues against the Java Client code[1], I started to dig into the server site code to understand the last issues with the Netty stuff. I feel bad. It's this feeling of I don't wanna hurt you, but ZooKeeper is quite an important piece of the Hadoop ecosystem containing some of the most complicated pieces of code. And it'll only get more complex with more features. I'd propose to have a word about quality assurance. Is there already a strategy to ensure the ongoing maintainability of ZK? Is there a code style guide, a list of Dos-And-Donts (where I'd like to add some points)? Should PMD be added to Hudson? What is the level of FindBugs? Should it be raised? Some of the points I'd like to add to a style guide: - Don't write methods longer then 20-40 lines of code - Are you sure you want to use inner classes? - If there is a new operator in a method? Could the method maybe already receive the object as a parameter? - Are you sure you want to use system properties? They are like global variables and the IDE does not know about them - Are you sure you want to extend a class? Often an aggregation is more elegant. - Don't nest ifs and loops deeper then 2 or 3 levels. If you do so, you should better break your code into more methods. - Use Enums or constants instead of plain status integers - please document your intentions in code comments. You don't need to comment the what? but the why?. Do you agree with me, that there is a need for better code quality in ZooKeeper? If so, it's not really scalable if a manic like me fights like Don Quichotte to clean up the code. All developers would need to establish a sense for clean code and constantly improve the code. [1] https://issues.apache.org/jira/browse/ZOOKEEPER-835 Best regards, Thomas Koch, http://www.koch.ro
[jira] Updated: (ZOOKEEPER-894) add Package o.a.zookeeper.client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-894: -- Description: I'd like to move classes that are not part of the API but belong to the ZK Client into a separate Client package. These classes are: - Inner classes that should become normal classes: Zookeeper.ZkWatchManager Zookeeper.WatchRegistration ClientCnxn.SendThread (should become a Runnable anyhow) ClientCnxn.EventThread ClientCnxn.Package ClientCnxn.AuthData ? - Classes now in the zookeeper package: ClientCnxn - Client.Cnxn ClientCnxnSocket* - Client.CnxnSocket* ... Maybe some others that can be moved without breaking the API - Classes yet to be written: PendingQueue ? OutgoingQueue ? was: I'd like to move classes that are not part of the API but belong to the ZK Client into a separate Client package. These classes are: - Inner classes that should become normal classes: Zookeeper.ZkWatchManager Zookeeper.WatchRegistration ClientCnxn.SendThread (should become a Runnable anyhow) ClientCnxn.EventThread ClientCnxn.Package - Classes now in the Zookeeper package: ClientCnxn - Client.Cnxn ClientCnxnSocket* - Client.CnxnSocket* ... Maybe some others that can be moved without breaking the API - Classes yet to be written: PendingQueue (?) OutgoingQueue (?) Summary: add Package o.a.zookeeper.client (was: add Package o.a.Zookeeper.Client) add Package o.a.zookeeper.client Key: ZOOKEEPER-894 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-894 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch I'd like to move classes that are not part of the API but belong to the ZK Client into a separate Client package. These classes are: - Inner classes that should become normal classes: Zookeeper.ZkWatchManager Zookeeper.WatchRegistration ClientCnxn.SendThread (should become a Runnable anyhow) ClientCnxn.EventThread ClientCnxn.Package ClientCnxn.AuthData ? - Classes now in the zookeeper package: ClientCnxn - Client.Cnxn ClientCnxnSocket* - Client.CnxnSocket* ... Maybe some others that can be moved without breaking the API - Classes yet to be written: PendingQueue ? OutgoingQueue ? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12920715#action_12920715 ] Thomas Koch commented on ZOOKEEPER-823: --- Hi Patrick, I saw the same exceptions as you saw. They are not _thrown_ inside the netty code, but they occurred in my cases every time during the NettyNettySuiteTest. It isn't very obvious from the junit log, whether a test was run as part of the NettyNettySuiteTest or not. I do a forward search for FAILED and from this position a backwards search for running to find the name of the actual test suite. Are you perfecty sure, the failed tests were not run as part of the NettyNettySuiteTest? update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12920750#action_12920750 ] Thomas Koch commented on ZOOKEEPER-823: --- The problem with the failing testWatchAutoResetWithPending might be the following. The test uses testableZooKeeper.pauseCnxn($milliSeconds) PauseCnxn tries to block the ClientCnxn by holding a lock on the ClientCnxn instance, interrupting the connection and not releasing the lock for $milliSeconds. Actually I wonder how this could have ever worked reliable in the first place. The only thing I see that also synchronizes on ClientCnxn is the getXid() method. The solution might be to find a better implementation for testableZooKeeper.pauseCnxn update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Status: Open (was: Patch Available) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: ZOOKEEPER-823.patch Corrected another synchronization issue. - write.awaitUninterruptibly() may not be in any synchronization block because otherwise we get timeouts. - Writing to the netty queue and adding the package to the pendingQueue must be in a sync(pendingQueue), otherwise we can receive the answer to the package, try to work with it in readResponse() but don't find the package in the pendingQueue. update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: testDisconnectedAddAuth_FAILURE I've built this patch 4 times now and had only ones a Failure in testDisconnectedAddAuth instead of multiple failures on every build. - Please commit to trunk and we'll also catch these last few issues. Anyway, all observed failures were only seen with Netty and we could mark them as Experimental as long as we don't fully trust this code. update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, testDisconnectedAddAuth_FAILURE, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12919293#action_12919293 ] Thomas Koch commented on ZOOKEEPER-823: --- How to continue with this? I know have a couple of Heisenbugs that do not appear with every test run. Most often * org.apache.zookeeper.test.ACLTest.testAcls * org.apache.zookeeper.test.AsyncHammerTest.testObserversHammer * org.apache.zookeeper.test.AsyncHammerTest.testHammer I have the following believing: - there are some issues with missing / misplaced synchronization blocks in the Netty java client code - there may also be some bugs in the server netty code. Sometimes I get bugs like this, which could also be caused on the server side, but I have not yet learned the ZK server code: Exception caught [id: 0x19509443, /127.0.0.1:53300 = localhost/127.0.0.1:11270] EXCEPTION: java.io.IOException: Xid out of order. Got Xid 32 with err 0 expected Xid 33 for a packet with details: I'd rather like to commit the current state to trunk and to continue the bug hunting in trunk. This would allow me to do some cleanups that could help to understand the correct synchronizations. This cleanups should be ZOOKEEPER-878 and ZOOKEEPER-879 This Exception by example could be better understood by solving the above mentioned issues: java.lang.NullPointerException at org.apache.zookeeper.ClientCnxn.conLossPacket(ClientCnxn.java:574) at org.apache.zookeeper.ClientCnxn.access$2300(ClientCnxn.java:76) at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:869) at org.apache.zookeeper.ClientCnxnSocketNetty.cleanup(ClientCnxnSocketNetty.java:147) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:741) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: ZOOKEEPER-823.patch I may have fixed another issue: I wrapped sendThread.readResponse(incomingBuffer) into a synchronization on the OutgoingQueue, because it might happen otherwise, that a package is send over netty and processed by the server, but not yet added to the pendingQueue. This fix solved all the Heisenbugs I saw. However there's still a bug with ASyncHammer and that the wait to join threads times out. I added more Debugging information. The Thread that times out hangs on ClientCnxnSocketNetty.wakeupCnxn where it waits for the synchronized(outgoingQueue). It seems that the outgoingQueue is already owned and blocked in the doWrites method, hanging on write.awaitUninterruptibly(). doWrites is called by doTransport where the synchronized(outgoingQueue) happens. update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Status: Patch Available (was: Open) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Status: Open (was: Patch Available) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: ZOOKEEPER-823.patch I did another version of the patch with an example how I'd solve the deadlock mentioned in my last comment. I made the synchronized blocks in doTransport and doWrites smaller. update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: (was: ZOOKEEPER-823.patch) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: ZOOKEEPER-823.patch update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-762) Allow dynamic addition/removal of server nodes in the client API
[ https://issues.apache.org/jira/browse/ZOOKEEPER-762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12916168#action_12916168 ] Thomas Koch commented on ZOOKEEPER-762: --- There are many changes in the java client on the horizont ( ZOOKEEPER-835 ) so if you implement this now, your patch will most likely not apply afterwards. Allow dynamic addition/removal of server nodes in the client API Key: ZOOKEEPER-762 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-762 Project: Zookeeper Issue Type: Sub-task Components: c client, java client Reporter: Dave Wright Priority: Minor Attachments: 762.diff Currently the list of zookeeper servers needs to be provided to the client APIs at construction time, and cannot be changed without a complete shutdown/restart of the client API. However, there are scenarios that require the server list to be updated, such as removal or addition of a ZK cluster node, and it would be nice if the list could be updated via a simple API call. The general approach (in the Java client) would be to RemoveServer()/AddServer() functions for Zookeeper that calls down to ClientCnxn, where they are just maintained in a list. Of course if the server being removed is the one currently connected, we'd need to disconnect, but a simple call to disconnect() seems like it would resolve that and trigger the automatic re-connection logic. An equivalent change could be made in the C code. This change would also make dynamic cluster membership in ZOOKEEPER-107 easier to implement. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-878) finishPacket and conLossPacket should be methods of Packet
finishPacket and conLossPacket should be methods of Packet -- Key: ZOOKEEPER-878 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-878 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch Those methods change the inner state of Packet, work on Packet so they should better be methods of class Packet. This may help to clarify synchronization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12913954#action_12913954 ] Thomas Koch commented on ZOOKEEPER-823: --- I can not reproduce any test failure on my system. So the reported failures from phunt and Junqueira are: - phunt: NettyNettySuiteTest - ACLTest.testDisconnectedAddAuth(ACLTest.java:67): org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for / at org.apache.zookeeper.KeeperException.create(KeeperException.java:90) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.setACL(ZooKeeper.java:1259) I don't understand, what exactly this test should test. A code comment would help. However I can imagine some race condition because the test does not wait for a client connection to be established before continuing. - phunt: NettyNettySuiteTest - AsyncHammerTest.testObserversHammer(AsyncHammerTest.java:213) I can't confirm this neither. However the test has some non determinism because it waits only 60 seconds for a thread to join. - Junqueira: NettyNettySuiteTest - ClientTest.testMutipleWatcherObjs(ClientTest.java:241) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /foo-89 I can't really help without access to machines where the tests fail. @Flavio What are the specs of your test machine? While investigating these failures I already felt the urge to continue my quest for clean code and refactor: ZOOKEEPER-878, ZOOKEEPER-879 However these two issues would blow up the patch even more. update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: NettyNettySuiteTest.rtf, TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-879) outgoingQueue should be a class
outgoingQueue should be a class --- Key: ZOOKEEPER-879 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-879 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch I'm not yet 100% sure about this yet, but it seems reasonable to me. Currently outgoingQueue is a simple list. Whether additional items can be added to the queue and the logic to add sth to the queue is handled by ClientCnxn. class OutgoingQueue - isOpen + add(Packet) / offer(Packet) + poll() / take() OutgoingQueue must have knowledge about the state of SendThreat and may only accept additional Packets if SendThread has not yet terminated. OutgoingQueue knows, when it must call ConnectionLoss on the remaining Packets in its queue. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12913446#action_12913446 ] Thomas Koch commented on ZOOKEEPER-823: --- With the latest trunk merged in, works on my machine. :-) BUILD SUCCESSFUL update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
REMINDER: development on ClientCnxn is blocked for 1.5 months
Hi, I don't know what is blocking progress right now and whether somebody is working on it, so I just cry for help: Since 08/11 I'm waiting to submit some improvements to the java client code (ZOOKEEPER-835), which has been blocked by ZOOKEEPER-823. Since 02/09 there's a patch for ZOOKEEPER-823 waiting to be tested by hudson, updated for newer trunk on 15/09. Today is 21/09. Could anybody please give a status update? I'll have my last exam at university this saturday, so I'd have some time to contribute afterwards, if you could help me. happy hacking, Thomas Koch, http://www.koch.ro
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Status: Patch Available (was: Open) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Status: Open (was: Patch Available) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Status: Open (was: Patch Available) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: ZOOKEEPER-823.patch updated patch to newest trunk state update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Status: Patch Available (was: Open) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12907270#action_12907270 ] Thomas Koch commented on ZOOKEEPER-823: --- Hi Ivan, thank you for looking over the patch. I agree with all your comments but the last one on src/java/main/org/apache/zookeeper/ClientCnxnSocketNIO:L110. Yes, the separate try-catches seem necessary to me. I've added a link to issue ZOOKEEPER-702 about the incompatibility. In general, it's easier to review patches that do small incremental changes. So I'd propose that you open separate jira issues for your observations so that we can work on them after this patch has been committed. I already opened ZOOKEEPER-835 as an umbrella issue for several clean ups in the ZK java client code. Your issues would fit in there. update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Review Request: ZOOKEEPER-823
Hi Ben, Mahadev, Patrick suggested I might ask you for review on ZOOKEEPER-823. It does some refactoring on ClientCnxn and blocks other issues which also want to edit ClientCnxn. Thanks, Thomas Koch, http://www.koch.ro
[jira] Updated: (ZOOKEEPER-849) Provide Path class
[ https://issues.apache.org/jira/browse/ZOOKEEPER-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-849: -- Status: Open (was: Patch Available) The ctor currently throws an unchecked exception, should be checked. Compare: http://stackoverflow.com/questions/1371369/about-constructors-in-java The path class should maybe also be made final. Provide Path class -- Key: ZOOKEEPER-849 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-849 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-849.patch, ZOOKEEPER-849.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-866) Adding no disk persistence option in zookeeper.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12906503#action_12906503 ] Thomas Koch commented on ZOOKEEPER-866: --- Thinking more about this, I'd consider this requirement a premature optimization. Wouldn't it be much more important to make ZK rock solid stable, well documented (also the code) and polish the usability of ZK before worrying about speed? Are you sure, that there's no other way to solve the issue at hand? Instead of fine grained locking, maybe it'd be possible to lock packages at once. Or some kind of constant hashing: By placing a node on a constant hashing ring, I lock everything until the next node. It'd be interesting to see the use case that require such fine grained locking, if that'd be possible. Adding no disk persistence option in zookeeper. --- Key: ZOOKEEPER-866 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-866 Project: Zookeeper Issue Type: New Feature Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 3.4.0 Attachments: ZOOKEEPER-nodisk.patch Its been seen that some folks would like to use zookeeper for very fine grained locking. Also, in there use case they are fine with loosing all old zookeeper state if they reboot zookeeper or zookeeper goes down. The use case is more of a runtime locking wherein forgetting the state of locks is acceptable in case of a zookeeper reboot. Not logging to disk allows high throughput on and low latency on the writes to zookeeper. This would be a configuration option to set (ofcourse the default would be logging to disk). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-866) Adding no disk persistence option in zookeeper.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12906328#action_12906328 ] Thomas Koch commented on ZOOKEEPER-866: --- Just 2c: I'm going to use ZK too for some middle grained locking, locking domains a crawler is working on. - I'd be very fine, if I could specify durability for every single create. This way I can use the same ZK instance for fine grained locking as well as for system configuration. Maybe another flag besides PERSISTENT | EPHEMERAL: DURABLE? - Actually there are more shades of grey then just durable and non durable: It'd be possible, that not so important locks will only be flushed every few minutes. But maybe this would make things too complicate? Adding no disk persistence option in zookeeper. --- Key: ZOOKEEPER-866 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-866 Project: Zookeeper Issue Type: New Feature Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 3.4.0 Attachments: ZOOKEEPER-nodisk.patch Its been seen that some folks would like to use zookeeper for very fine grained locking. Also, in there use case they are fine with loosing all old zookeeper state if they reboot zookeeper or zookeeper goes down. The use case is more of a runtime locking wherein forgetting the state of locks is acceptable in case of a zookeeper reboot. Not logging to disk allows high throughput on and low latency on the writes to zookeeper. This would be a configuration option to set (ofcourse the default would be logging to disk). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: ZOOKEEPER-823.patch changes: - call ClientCnxn.cleanup() from ClientCnxnSocketNIO.cleanup(), was lost during the refactoring - cleaned the formatting changes to make the patch smaller Now there are only three failures left: NettyNettySuiteTest - ACLTest.testAcls KeeperErrorCode = ConnectionLoss for /0 org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /0 at org.apache.zookeeper.KeeperException.create(KeeperException.java:90) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:640) at org.apache.zookeeper.test.ACLTest.testAcls(ACLTest.java:104) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:51) When I run the whole suite in eclipse as JUnit test, it does not fail. NettyNettySuiteHammerTest - The log doesn't tell me anything, I assume it's just the same as in NettyNettySuiteTest NioNettySuiteTest - ClientTest.testClientCleanup open fds after test are not significantly higher than before junit.framework.AssertionFailedError: open fds after test are not significantly higher than before at org.apache.zookeeper.test.ClientTest.testClientCleanup(ClientTest.java:731) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:51) When I run the whole suite in eclipse, the test still fails, however when I run only ClientTest.testClientCleanup alone, it does not fail anymore. I would really appreciate, if you could help me from now on. I double, partly triple checked the refactoring. update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Status: Patch Available (was: Open) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
race condition in InvalidSnapShotTest on client close
Hi, I believe, that I've found a race condition in org.apache.zookeeper.server.InvalidSnapshotTest In this test the server is closed before the client. The client, on close(), submits as last package with type ZooDefs.OpCode.closeSession and waits for this package to be finished. However, nobody is there to awake the thread from packet.wait(). The sendThread will on cleanup call packet.notifyAll() in finishpackage. The race condition is: If an exception occurs in the sendThread, closing is already true, so the sendThread breaks out of it's loop, calls cleanup and finishes. If this happens, before the main thread calls packet.wait() then there's nobody left to awake the main thread. Regards, Thomas Koch, http://www.koch.ro
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[
https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Koch updated ZOOKEEPER-823:
--
Attachment: ZOOKEEPER-823.patch
I changed the following things:
- call the correct cleanup() method. I f..ed up to refactor the call hierarchy
- combine doReads() and doWrites() again into doIO() in NIO as it was before
- all access to selector is synchronized on ClientCnxnSocket
I'm having problems to run the testsuite and believe that I'm somehow bitten by
ZOOKEEPER-126. Some Tests just block and when I debug them, they always wait
indefinitely on packet.wait().
Something like the following helped, however with every different attempt to
solve ZOOKEEPER-126 other tests failed respectively blocked.
src/java/main/org/apache/zookeeper/ClientCnxn.java
@@ -1088,6 +1088,10 @@ public class ClientCnxn {
ReplyHeader r = new ReplyHeader();
Packet packet = queuePacket(h, r, request, response, null, null, null,
null, watchRegistration);
+if(h.getType() == ZooDefs.OpCode.closeSession){
+Thread.sleep((long) (sessionTimeout*1.1));
+return r;
+}
synchronized (packet) {
while (!packet.finished) {
packet.wait();
My tests still run now with the above code. Some take 10 minutes and more.
I also had to reduce the number of hammerthreads in
org.apache.zookeeper.test.AsyncHammerTest to 10 to make it work.
I'm running openjdk
java version 1.6.0_18
OpenJDK Runtime Environment (IcedTea6 1.8.1) (6b18-1.8.1-1)
OpenJDK 64-Bit Server VM (build 16.0-b13, mixed mode)
update ZooKeeper java client to optionally use Netty for connections
Key: ZOOKEEPER-823
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823
Project: Zookeeper
Issue Type: New Feature
Components: java client
Reporter: Patrick Hunt
Assignee: Patrick Hunt
Fix For: 3.4.0
Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch,
ZOOKEEPER-823.patch, ZOOKEEPER-823.patch
This jira will port the client side connection code to use netty rather than
direct nio.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Status: Patch Available (was: Open) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: ZOOKEEPER-823.patch This is a refactoring of phunt's patch to separate the socket specific code into subclasses of ClientCnxnSocket and make ClientCnxn concrete again. IMHO this makes the code much more comprehendable. I'm sorry, that eclipse made the patch a little larger then necessary. I've turned the auto-reformatting off now, but I don't know any easy way to undo the formatting changes. update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: ZOOKEEPER-823.patch update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections
[ https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-823: -- Attachment: (was: ZOOKEEPER-823.patch) update ZooKeeper java client to optionally use Netty for connections Key: ZOOKEEPER-823 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823 Project: Zookeeper Issue Type: New Feature Components: java client Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-823.patch, ZOOKEEPER-823.patch This jira will port the client side connection code to use netty rather than direct nio. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
apache commons configuration
Hi, just out of curiosity: Is there any particular reason, why Hadoop projects or ZooKeeper do not use apache commons configuration[1]? I'm just evaluating it for an internal project. Would you recommend it or not? I imagine, that it could help to make projects configurable either via XML or INI files. Some people may prefer one or the other. [1] http://commons.apache.org/configuration/ Best regards, Thomas Koch, http://www.koch.ro
WIP: introduce ClientCnxnSocket classes for nicer netty integration
Hi, I got permission from Patrick to try to make the java client netty introduction a little bit nicer. I could need a little help now at one point. Please have a look at http://github.com/thkoch2001/zookeeper/tree/use_netty2/src/java/main/org/apache/zookeeper/ The classes ClientCnxnSocketNIO and ClientCnxnSocketNetty are used by ClientCnxn.SendThread. Both have a method xyz() now, where I've moved some lines of code that were different in the run() loop of both implementations. If you could help me to move the logic from xyz() back in the run() method with fitting calls to the socket() class, then I might be done. I'm a bit puzzled because in NIO there's first a selector.select(to) and then doWrites(), while in Netty there's first doWrites() and then outgoingQueue.wait(to). Best regards, Thomas Koch, http://www.koch.ro
