Heisenbugs, Bohrbugs, Mandelbugs?
Hi, last night I let my hudson server do 42 (sic) builds of ZooKeeper trunk. One of this builds failed: junit.framework.AssertionFailedError: Leader hasn't joined: 5 at org.apache.zookeeper.test.FLETest.testLE(FLETest.java:312) I did this many builds of trunk, because in my quest to redo the client netty integration step by step I made one step which resulted in 2 failed builds out of 8. The two failures were both: junit.framework.AssertionFailedError: Threads didn't join at org.apache.zookeeper.test.FLERestartTest.testLERestart(FLERestartTest.java:198) I can't find any relationship between the above test and my changes. The test does not use the ZooKeeper client code at all. So I begin to believe that there are some Heisenbugs, Bohrbugs or Mandelbugs[1] in ZooKeeper that just happen to show up from time to time without any relationship to the current changes. I'll try to investigate the cause further, maybe there is some relationship I've not yet found. But if my assumption should apply, then these kind of bugs would be a strong argument in favor of refactoring. These bugs are best found by cleaning the code, most important implementing strict separation of concerns. Wouldn't you like to setup Hudson to build ZooKeeper trunk every half an hour? [1] http://en.wikipedia.org/wiki/Unusual_software_bug Best regards, Thomas Koch, http://www.koch.ro
Re: Restarting discussion on ZooKeeper as a TLP
+1 for moving forward, and I was wondering if you have an idea of when you'd have a draft of the proposal. It would be good to iterate over it perhaps.-FlavioOn Oct 20, 2010, at 7:50 PM, Patrick Hunt wrote:It's been a few days, any thoughts? Acceptable? I'd like to keep moving theball forward. Thanks.PatrickOn Sun, Oct 17, 2010 at 8:43 PM, 明珠刘 redis...@gmail.com wrote:+12010/10/14 Patrick Hunt ph...@apache.orgIn March of this year we discussed a request from the Apache Board, andHadoop PMC, that we become a TLP rather than a subproject of Hadoop:Original discussionhttp://markmail.org/thread/42cobkpzlgotcbinI originally voted against this move, my primary concern being that wewerenot "ready" to move to tlp status given our small contributor base andlimited contributor diversity. However I'd now like to revisit thatdiscussion/decision. Since that time the team has been working hard toattract new contributors, and we've seen significant new contributionscomein. There has also been feedback from board/pmc addressing many of theseconcerns (both on the list and in private). I am now less concerned aboutthis issue and don't see it as a blocker for us to move to TLP status.A second concern was that by becoming a TLP the project would lose it'sconnection with Hadoop, a big source of new users for us. I've beenassured(and you can see with the other projects that have moved to tlp status;pig/hive/hbase/etc...) that this connection will be maintained. TheHadoopZooKeeper tab for example will redirect to our new homepage.Other Apache members also pointed out to me that we are essentiallyoperating as a TLP within the Hadoop PMC. Most of the other PMC membershavelittle or no experience with ZooKeeper and this makes it difficult forthemto monitor and advise us. By moving to TLP status we'll be able to governourselves and better set our direction.I believe we are ready to become a TLP. Please respond to this email withyour thoughts and any issues. I will call a vote in a few days, oncediscussion settles.Regards,Patrick flaviojunqueiraresearch scientistf...@yahoo-inc.comdirect +34 93-183-8828avinguda diagonal 177, 8th floor, barcelona, 08018, esphone (408) 349 3300fax (408) 349 3301
[jira] Commented: (ZOOKEEPER-102) Need to replace Jute with supported code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923380#action_12923380 ] Thomas Koch commented on ZOOKEEPER-102: --- Another option would be avro. Does anybody know, whether avro is wire compatible with jute? We would need to support communication between older jute-only Clients/Servers and newer avro(+jute?) Clients/Servers. Would this be a good Google Summer of Code task? Need to replace Jute with supported code Key: ZOOKEEPER-102 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102 Project: Zookeeper Issue Type: Improvement Reporter: Benjamin Reed ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers. Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-908) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation
Remove code duplication and inconsistent naming in ClientCnxn.Packet creation - Key: ZOOKEEPER-908 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-908 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch rename record - request (since their is a counterpart record named response) rename header - requestHeader (to distinguish from responseHeader) remove ByteBuffer creation code from primeConnection() method and use the duplicate code in the Packet constructor. Therefor the Bytebuffer bb parameter could also be removed from the constructor's parameters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-794) Callbacks are not invoked when the client is closed
[ https://issues.apache.org/jira/browse/ZOOKEEPER-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923396#action_12923396 ] Hudson commented on ZOOKEEPER-794: -- Integrated in ZooKeeper-trunk #975 (See [https://hudson.apache.org/hudson/job/ZooKeeper-trunk/975/]) Callbacks are not invoked when the client is closed --- Key: ZOOKEEPER-794 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-794 Project: Zookeeper Issue Type: Bug Components: java client Affects Versions: 3.3.1 Reporter: Alexis Midon Assignee: Alexis Midon Priority: Blocker Fix For: 3.3.2, 3.4.0 Attachments: ZOOKEEPER-794.patch.txt, ZOOKEEPER-794.txt, ZOOKEEPER-794_2.patch, ZOOKEEPER-794_3.patch, ZOOKEEPER-794_4.patch.txt, ZOOKEEPER-794_5.patch.txt, ZOOKEEPER-794_5_br33.patch I noticed that ZooKeeper has different behaviors when calling synchronous or asynchronous actions on a closed ZooKeeper client. Actually a synchronous call will throw a session expired exception while an asynchronous call will do nothing. No exception, no callback invocation. Actually, even if the EventThread receives the Packet with the session expired err code, the packet is never processed since the thread has been killed by the ventOfDeath. So the call back is not invoked. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-908) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-908: -- Attachment: ZOOKEEPER-908.patch ant test-core did work without failures. Remove code duplication and inconsistent naming in ClientCnxn.Packet creation - Key: ZOOKEEPER-908 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-908 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch Attachments: ZOOKEEPER-908.patch rename record - request (since their is a counterpart record named response) rename header - requestHeader (to distinguish from responseHeader) remove ByteBuffer creation code from primeConnection() method and use the duplicate code in the Packet constructor. Therefor the Bytebuffer bb parameter could also be removed from the constructor's parameters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (ZOOKEEPER-815) fill in TBDs in overview doc
[ https://issues.apache.org/jira/browse/ZOOKEEPER-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrei Savu reassigned ZOOKEEPER-815: - Assignee: (was: Andrei Savu) fill in TBDs in overview doc -- Key: ZOOKEEPER-815 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-815 Project: Zookeeper Issue Type: Bug Components: documentation Affects Versions: 3.3.1 Reporter: Patrick Hunt Priority: Minor Fix For: 3.3.2, 3.4.0 Funny: Ephemeral nodes are useful when you want to implement [tbd]. there are a few others in that doc that are should really be fixed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-906) Improve C client connection reliability by making it sleep between reconnect attempts as in Java Client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923431#action_12923431 ] Jared Cantwell commented on ZOOKEEPER-906: -- Some other small comments based on what I've seen so far with this project: 1. Are the changes to mt_adaptor still needed? 2. I think it can be restructured to remove the must_sleep bool and be more obvious whats going on. The check that resets the index to 0 can happen all the time. And then the two sleep conditions can be an if/else if, with a final else that does what the !must_sleep branch does now. 3. Changing 'zh-connect_index == zh-addrs_count' to 'zh-connect_index = zh-addrs_count' may seem subtle, but there is a whole bug devoted to it, so this patch probably shouldn't change that. See ZOOKEEPER-458. 4. Before this gets committed, i imagine its going to need a unittest or two. 5. Indentation should be 4 spaces. Improve C client connection reliability by making it sleep between reconnect attempts as in Java Client --- Key: ZOOKEEPER-906 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-906 Project: Zookeeper Issue Type: Improvement Components: c client Affects Versions: 3.3.1 Reporter: Radu Marin Assignee: Radu Marin Fix For: 3.4.0 Attachments: ZOOKEEPER-906.patch Original Estimate: 24h Remaining Estimate: 24h Currently, when a C client get disconnected, it retries a couple of hosts (not all) with no delay between attempts and then if it doesn't succeed it sleeps for 1/3 session expiration timeout period before trying again. In the worst case the disconnect event can occur after 2/3 of session expiration timeout has past, and sleeping for even more 1/3 session timeout will cause a session loss in most of the times. A better approach is to check all hosts but with random delay between reconnect attempts. Also the delay must be independent of session timeout so if we increase the session timeout we also increase the number of available attempts. This improvement covers the case when the C client experiences network problems for a short period of time and is not able to reach any zookeeper hosts. Java client already uses this logic and works very good. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Patrick Hunt Fix For: 3.4.0 This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Attachment: ZOOKEEPER-909.patch Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Status: Patch Available (was: Open) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-910) Use SelectionKey.isXYZ() methods instead of complicated binary logic
Use SelectionKey.isXYZ() methods instead of complicated binary logic Key: ZOOKEEPER-910 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-910 Project: Zookeeper Issue Type: Sub-task Reporter: Thomas Koch Fix For: 3.4.0 The SelectionKey class provides methods to replace something like this (k.readyOps() (SelectionKey.OP_READ | SelectionKey.OP_WRITE)) != 0 with selectionKey.isReadable() || selectionKey.isWritable() It may be possible, that the first version saves a CPU cycle or two, but the later version saves developer brain cycles which are much more expensive. I suppose that there are many more places in the server code where this replacement could be done. I propose that whoever touches a code line like this should make the replacement. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
What's the magic behind lenBuffer and incomingBuffer?
Hi,
inside ClientCnxn.SendThread we have
final ByteBuffer lenBuffer = ByteBuffer.allocateDirect(4);
ByteBuffer incomingBuffer = lenBuffer;
So incomingBuffer and lenBuffer do refer to the same object. There are several
other places where lenBuffer is again assigned to incomingBuffer.
Now inside the doIO() method we got
if (incomingBuffer == lenBuffer) {
recvCount++;
readLength();
} else if (!initialized) {
incomingBuffer is never assigned anything else then lenBuffer, lenBuffer stays
the same all the time. So as far as my knowledge of java reaches (which may
not be too far) incomingBuffer == lenBuffer _always_ evaluates to true. Isn't
that true?
So effectively we've got dead code in the elseif and else branches, didn't we?
Best regards,
Thomas Koch, http://www.koch.ro
[jira] Created: (ZOOKEEPER-911) move operations from methods to individual classes
move operations from methods to individual classes
--
Key: ZOOKEEPER-911
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-911
Project: Zookeeper
Issue Type: New Feature
Components: java client
Reporter: Thomas Koch
Copied from my email to the ZK dev list from 2010/05/26:
For my current code I'm using zkclient[1] and have also looked at cages[2] for
some ZK usage examples. I observed, that there's a common pattern to wrap ZK
operations in callables and feed them to a retryUntilConnected executor.
Now my idea is, that ZK should already come with operations in classes, e.g.:
o.a.z.operation.Create extends Operation implements callable{
private path, data[], acl, createMode
public Create( .. all kind of ctors .. )
public call(){
.. move code from Zookeeper.create() here
}
}
Similiar classes should be provided for getChildren, delete, exists, getData,
getACL, setACL and setData.
One could then feed such operations to an ZkExecutor, which has the necessary
knowledge about the ZkConnection and can execute a command either
synchronously or asynchronously.
One could also wrap operations in an ExceptionCatcher to ignore certain
Exceptions or in a RetryPolicy.
This is only an idea so far, but I wanted to share my thoughts before starting
to try it out. (BTW: You can meet me at BerlinBuzzwords.de)
[1] http://github.com/sgroschupf/zkclient
[2] http://code.google.com/p/cages/
And a reply from Patrick Hunt at my mail:
Hi Thomas, you might take a look at this JIRA
https://issues.apache.org/jira/browse/ZOOKEEPER-679
there's definitely been interest in this area, however there are some
real challenges as well. Most users do end up wrapping the basic api
with some code, esp the retry metaphor is a common case, so I think it
would be valuable. At the same time getting the semantics right is hard
(and covering all the corner cases). Perhaps you could sync up with
Aaron/Chris, I'd personally like to see this go into contrib, but I
understand the extra burden the patch process presents -- it may make
more sense to rapidly iterate on something like github and then move to
contrib once you have something less frequently changing, where the
patch issue would be less of a problem (see 679, there's discussion on
this there). Regardless which way you take it we'd be happy to work with
you.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-835) Refactoring Zookeeper Client Code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923476#action_12923476 ] Thomas Koch commented on ZOOKEEPER-835: --- Hi Benjamin, I'm sorry, I should have filled ZOOKEEPER-911 sooner. Actually ZOOKEEPER-911 would make it easier to implement any retry logic and ZOOKEEPER-911 depends on some of the sub-issues of this issue. Refactoring Zookeeper Client Code - Key: ZOOKEEPER-835 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-835 Project: Zookeeper Issue Type: Improvement Components: java client Affects Versions: 3.3.1 Reporter: Patrick Datko Assignee: Thomas Koch Thomas Koch asked me to fill individual issues for the points raised in his mail to zookeeper-dev: [Mail of Thomas Koch| http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-dev/201008.mbox/%3c20100845.17507.tho...@koch.ro%3e ] He published several issues, which are present in the current zookeeper client, so a refactoring of the code would be an facility for other developers working with zookeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Description: This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! was: This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Patrick Hunt Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Heisenbugs, Bohrbugs, Mandelbugs?
On Wed, Oct 20, 2010 at 11:48 PM, Thomas Koch tho...@koch.ro wrote: Hi, last night I let my hudson server do 42 (sic) builds of ZooKeeper trunk. One of this builds failed: junit.framework.AssertionFailedError: Leader hasn't joined: 5 at org.apache.zookeeper.test.FLETest.testLE(FLETest.java:312) I did this many builds of trunk, because in my quest to redo the client netty integration step by step I made one step which resulted in 2 failed builds out of 8. The two failures were both: junit.framework.AssertionFailedError: Threads didn't join at org.apache.zookeeper.test.FLERestartTest.testLERestart(FLERestartTest.java:198) Hi Thomas, there's an open jira for this: https://issues.apache.org/jira/browse/ZOOKEEPER-653 great if you'd like to address it. I can't find any relationship between the above test and my changes. The test does not use the ZooKeeper client code at all. So I begin to believe that there are some Heisenbugs, Bohrbugs or Mandelbugs[1] in ZooKeeper that just happen to show up from time to time without any relationship to the current changes. I'll try to investigate the cause further, maybe there is some relationship I've not yet found. But if my assumption should apply, then these kind of bugs would be a strong argument in favor of refactoring. These bugs are best found by cleaning the code, most important implementing strict separation of concerns. I believe the bug is in the test, rather than in the code. Forming a quorum is non-deterministic, the test assumes that it's allowing enough time for everyone to join, this may not be the case. The opposite may be true as well however, it might be the case that something is really failing, however my understanding from Flavio is that it's the former. The unfortunate thing is that since we don't really know which it is, we sort of ignore these failures. Really we should fix this issue for reals. Whatever that means... Flavio perhaps you could give Thomas some insight, if you have ideas he is motivated to help resolve. Also notice that we are currently @Ignore ing a handful of tests. These are also broken tests, tests which we really need to fix and bring back online. The session moved in particular needs to be fixed (again, non-deterministic test, probably could benefit from some refactoring, however I think it's more a design for test issue). Take a look at the clover output for some insight on areas that need more testing and refactoring (coverage/complexity): https://hudson.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/clover/ Wouldn't you like to setup Hudson to build ZooKeeper trunk every half an hour? I wouldn't mind, but we'd probably get yelled at by the apache hudson admins. :-) Hudson is a shared resource and we typically need to play nice. Also there's been problems with hadoop on hudson for the past few months, Nigel is working on that, might be a good thing to bring up again once that's addressed (patch queue primarily). Patrick
[jira] Commented: (ZOOKEEPER-102) Need to replace Jute with supported code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923516#action_12923516 ] Doug Cutting commented on ZOOKEEPER-102: Avro is not wire-compatible with Jute. Need to replace Jute with supported code Key: ZOOKEEPER-102 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102 Project: Zookeeper Issue Type: Improvement Reporter: Benjamin Reed ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers. Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-102) Need to replace Jute with supported code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923519#action_12923519 ] Patrick Hunt commented on ZOOKEEPER-102: bq. avro is wire compatible with jute? in general no, however Doug was wondering if there was some way to define the avro spec in such a way that it would work. bq. Would this be a good Google Summer of Code task? it would be an interesting project regardless. We've discussed it a few times, the general idea being that we should add the ability to configure a new client port using Avro, maintain b/w compatibility by leaving the old jute based port alone (although allow it to be turned off through config). Once all clients were migrated the zk ensemble admin could turn off the legacy jute port. Btw, while fixing this we should also split the client and admin (4letterword) ports. This causes us all kinds of headache (today shared) and really the 4letterword feature should be expanded to allow for more sophisticated usage (for example long lived connection). Also I've heard complaints from admins that they don't want to expose the 4letter words to all users (it's r/o but they want to lock down that data). Again, having a separate admin more might allow for more features. One other issue comes to mind. Last time I benchmarked our message definitions on Jute vs Avro the avro C client performance was terrible. We wouldn't want to switch over from jute to avro (default I'm saying) until this is addressed. I'm also not sure if the avro c api is locked down. Need to replace Jute with supported code Key: ZOOKEEPER-102 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102 Project: Zookeeper Issue Type: Improvement Reporter: Benjamin Reed ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers. Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-102) Need to replace Jute with supported code
[ https://issues.apache.org/jira/browse/ZOOKEEPER-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923521#action_12923521 ] Patrick Hunt commented on ZOOKEEPER-102: haha, doug and I crossed beams on comments. :-) Need to replace Jute with supported code Key: ZOOKEEPER-102 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-102 Project: Zookeeper Issue Type: Improvement Reporter: Benjamin Reed ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers. Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-908) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923526#action_12923526 ] Patrick Hunt commented on ZOOKEEPER-908: Thanks Thomas. As we are refactoring this code anyway, and our raised consciousness on best practices, how about implementing some tests for the effected class (packet?). Any low hanging fruit that you see (perhaps a bit more refactoring may be needed) wrt test coverage? If you think it's ready, or if after you add tests, click on the submit patch link to transition this to patch available (at which point we'll review for commit, and typically (if it wasn't currently broken) the automated patch tests would run.) Remove code duplication and inconsistent naming in ClientCnxn.Packet creation - Key: ZOOKEEPER-908 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-908 Project: Zookeeper Issue Type: Sub-task Components: server Reporter: Thomas Koch Assignee: Thomas Koch Priority: Minor Fix For: 3.4.0 Attachments: ZOOKEEPER-908.patch rename record - request (since their is a counterpart record named response) rename header - requestHeader (to distinguish from responseHeader) remove ByteBuffer creation code from primeConnection() method and use the duplicate code in the Packet constructor. Therefor the Bytebuffer bb parameter could also be removed from the constructor's parameters. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Hunt updated ZOOKEEPER-909: --- Assignee: Thomas Koch (was: Patrick Hunt) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-910) Use SelectionKey.isXYZ() methods instead of complicated binary logic
[ https://issues.apache.org/jira/browse/ZOOKEEPER-910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Hunt updated ZOOKEEPER-910: --- Component/s: server Assignee: Thomas Koch Thomas, feel free to make this change and submit as a patch. Much easier to review/resolve that way. Use SelectionKey.isXYZ() methods instead of complicated binary logic Key: ZOOKEEPER-910 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-910 Project: Zookeeper Issue Type: Sub-task Components: server Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 The SelectionKey class provides methods to replace something like this (k.readyOps() (SelectionKey.OP_READ | SelectionKey.OP_WRITE)) != 0 with selectionKey.isReadable() || selectionKey.isWritable() It may be possible, that the first version saves a CPU cycle or two, but the later version saves developer brain cycles which are much more expensive. I suppose that there are many more places in the server code where this replacement could be done. I propose that whoever touches a code line like this should make the replacement. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Koch updated ZOOKEEPER-909: -- Attachment: ZOOKEEPER-909.patch netty2 could not be found by ivy --- a/ivysettings.xml +++ b/ivysettings.xml @@ -20,7 +20,7 @@ property name=repo.maven.org value=http://repo1.maven.org/maven2/; override=false/ property name=repo.jboss.org -value=http://repository.jboss.com/maven2/; override=false/ +value=http://repository.jboss.org/nexus/content/groups/public/; override=false/ Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Restarting discussion on ZooKeeper as a TLP
Looks good, please do call a vote. On 21 October 2010 09:29, Patrick Hunt ph...@apache.org wrote: Here's a draft board resolution (not a vote, just discussion). It lists all current committers (except as noted in the next paragraph) as the initial members of the project management committee (PMC) and myself as the initial chair. Notice that I have left Andrew off the PMC as he has not been active with the project for over two years. I believe we should continue to include him on the committer roles subsequent to moving to tlp, however as he has not been an active member of the community for such a long period we would not include him on the PMC at this time. If others feel differently let me know, I'm willing to include him if the people feel differently. LMK if this looks good to you and I'll call for an official vote on this list (then we'll be ready to call a vote on the hadoop pmc). Regards, Patrick X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is responsible for the creation and maintenance of software related to data serialization; and be it further RESOLVED, that the office of Vice President, Apache ZooKeeper be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache ZooKeeper Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache ZooKeeper Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache ZooKeeper Project: * Patrick Hunt ph...@apache.org * Flavio Junqueira f...@apache.org * Mahadev Konarmaha...@apache.org * Benjamin Reedbr...@apache.org NOW, THEREFORE, BE IT FURTHER RESOLVED, that Patrick Hunt be appointed to the office of Vice President, Apache ZooKeeper, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the initial Apache ZooKeeper PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache ZooKeeper Project; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is tasked with the migration and rationalization of the Apache Hadoop ZooKeeper sub-project; and be it further RESOLVED, that all responsibilities pertaining to the Apache Hadoop ZooKeeper sub-project encumbered upon the Apache Hadoop Project are hereafter discharged. On Thu, Oct 21, 2010 at 1:00 AM, Flavio Junqueira f...@yahoo-inc.comwrote: +1 for moving forward, and I was wondering if you have an idea of when you'd have a draft of the proposal. It would be good to iterate over it perhaps. -Flavio On Oct 20, 2010, at 7:50 PM, Patrick Hunt wrote: It's been a few days, any thoughts? Acceptable? I'd like to keep moving the ball forward. Thanks. Patrick On Sun, Oct 17, 2010 at 8:43 PM, 明珠刘 redis...@gmail.com wrote: +1 2010/10/14 Patrick Hunt ph...@apache.org In March of this year we discussed a request from the Apache Board, and Hadoop PMC, that we become a TLP rather than a subproject of Hadoop: Original discussion http://markmail.org/thread/42cobkpzlgotcbin I originally voted against this move, my primary concern being that we were not ready to move to tlp status given our small contributor base and limited contributor diversity. However I'd now like to revisit that discussion/decision. Since that time the team has been working hard to attract new contributors, and we've seen significant new contributions come in. There has also been feedback from board/pmc addressing many of these concerns (both on the list and in private). I am now less concerned about this issue and don't see it as a blocker for us to move to TLP status. A second concern was that by
[jira] Assigned: (ZOOKEEPER-904) super digest is not actually acting as a full superuser
[ https://issues.apache.org/jira/browse/ZOOKEEPER-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Camille Fournier reassigned ZOOKEEPER-904: -- Assignee: Camille Fournier super digest is not actually acting as a full superuser --- Key: ZOOKEEPER-904 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-904 Project: Zookeeper Issue Type: Bug Components: server Affects Versions: 3.3.1 Reporter: Camille Fournier Assignee: Camille Fournier The documentation states: New in 3.2: Enables a ZooKeeper ensemble administrator to access the znode hierarchy as a super user. In particular no ACL checking occurs for a user authenticated as super. However, if a super user does something like: zk.setACL(/, Ids.READ_ACL_UNSAFE, -1); the super user is now bound by read-only ACL. This is not what I would expect to see given the documentation. It can be fixed by moving the chec for the super authId in PrepRequestProcessor.checkACL to before the for(ACL a : acl) loop. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-906) Improve C client connection reliability by making it sleep between reconnect attempts as in Java Client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radu Marin updated ZOOKEEPER-906: - Attachment: ZOOKEEPER-906.patch Removed mt_adaptor.c changes. Refactored the code based on Jared's review. Improve C client connection reliability by making it sleep between reconnect attempts as in Java Client --- Key: ZOOKEEPER-906 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-906 Project: Zookeeper Issue Type: Improvement Components: c client Affects Versions: 3.3.1 Reporter: Radu Marin Assignee: Radu Marin Fix For: 3.4.0 Attachments: ZOOKEEPER-906.patch Original Estimate: 24h Remaining Estimate: 24h Currently, when a C client get disconnected, it retries a couple of hosts (not all) with no delay between attempts and then if it doesn't succeed it sleeps for 1/3 session expiration timeout period before trying again. In the worst case the disconnect event can occur after 2/3 of session expiration timeout has past, and sleeping for even more 1/3 session timeout will cause a session loss in most of the times. A better approach is to check all hosts but with random delay between reconnect attempts. Also the delay must be independent of session timeout so if we increase the session timeout we also increase the number of available attempts. This improvement covers the case when the C client experiences network problems for a short period of time and is not able to reach any zookeeper hosts. Java client already uses this logic and works very good. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-906) Improve C client connection reliability by making it sleep between reconnect attempts as in Java Client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radu Marin updated ZOOKEEPER-906: - Attachment: (was: ZOOKEEPER-906.patch) Improve C client connection reliability by making it sleep between reconnect attempts as in Java Client --- Key: ZOOKEEPER-906 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-906 Project: Zookeeper Issue Type: Improvement Components: c client Affects Versions: 3.3.1 Reporter: Radu Marin Assignee: Radu Marin Fix For: 3.4.0 Attachments: ZOOKEEPER-906.patch Original Estimate: 24h Remaining Estimate: 24h Currently, when a C client get disconnected, it retries a couple of hosts (not all) with no delay between attempts and then if it doesn't succeed it sleeps for 1/3 session expiration timeout period before trying again. In the worst case the disconnect event can occur after 2/3 of session expiration timeout has past, and sleeping for even more 1/3 session timeout will cause a session loss in most of the times. A better approach is to check all hosts but with random delay between reconnect attempts. Also the delay must be independent of session timeout so if we increase the session timeout we also increase the number of available attempts. This improvement covers the case when the C client experiences network problems for a short period of time and is not able to reach any zookeeper hosts. Java client already uses this logic and works very good. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Restarting discussion on ZooKeeper as a TLP
Ack, I missed Henry in the list, sorry! In my defense I copied this: http://hadoop.apache.org/zookeeper/credits.html one more try (same as before except for adding henry to the pmc): X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is responsible for the creation and maintenance of software related to data serialization; and be it further RESOLVED, that the office of Vice President, Apache ZooKeeper be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache ZooKeeper Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache ZooKeeper Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache ZooKeeper Project: * Patrick Hunt ph...@apache.org * Flavio Junqueira f...@apache.org * Mahadev Konarmaha...@apache.org * Benjamin Reedbr...@apache.org * Henry Robinson he...@apache.org NOW, THEREFORE, BE IT FURTHER RESOLVED, that Matt Massie be appointed to the office of Vice President, Apache ZooKeeper, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the initial Apache ZooKeeper PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache ZooKeeper Project; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is tasked with the migration and rationalization of the Apache Hadoop ZooKeeper sub-project; and be it further RESOLVED, that all responsibilities pertaining to the Apache Hadoop ZooKeeper sub-project encumbered upon the Apache Hadoop Project are hereafter discharged. On Thu, Oct 21, 2010 at 10:44 AM, Henry Robinson he...@cloudera.com wrote: Looks good, please do call a vote. On 21 October 2010 09:29, Patrick Hunt ph...@apache.org wrote: Here's a draft board resolution (not a vote, just discussion). It lists all current committers (except as noted in the next paragraph) as the initial members of the project management committee (PMC) and myself as the initial chair. Notice that I have left Andrew off the PMC as he has not been active with the project for over two years. I believe we should continue to include him on the committer roles subsequent to moving to tlp, however as he has not been an active member of the community for such a long period we would not include him on the PMC at this time. If others feel differently let me know, I'm willing to include him if the people feel differently. LMK if this looks good to you and I'll call for an official vote on this list (then we'll be ready to call a vote on the hadoop pmc). Regards, Patrick X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is responsible for the creation and maintenance of software related to data serialization; and be it further RESOLVED, that the office of Vice President, Apache ZooKeeper be and hereby is created, the person holding such office to serve at the direction of the Board of Directors
Re: Restarting discussion on ZooKeeper as a TLP
Ha, I may just have excluded myself from eligibility due to my inability to read :) On 21 October 2010 13:28, Patrick Hunt ph...@apache.org wrote: Ack, I missed Henry in the list, sorry! In my defense I copied this: http://hadoop.apache.org/zookeeper/credits.html one more try (same as before except for adding henry to the pmc): X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is responsible for the creation and maintenance of software related to data serialization; and be it further RESOLVED, that the office of Vice President, Apache ZooKeeper be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache ZooKeeper Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache ZooKeeper Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache ZooKeeper Project: * Patrick Hunt ph...@apache.org * Flavio Junqueira f...@apache.org * Mahadev Konarmaha...@apache.org * Benjamin Reedbr...@apache.org * Henry Robinson he...@apache.org NOW, THEREFORE, BE IT FURTHER RESOLVED, that Matt Massie be appointed to the office of Vice President, Apache ZooKeeper, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the initial Apache ZooKeeper PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache ZooKeeper Project; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is tasked with the migration and rationalization of the Apache Hadoop ZooKeeper sub-project; and be it further RESOLVED, that all responsibilities pertaining to the Apache Hadoop ZooKeeper sub-project encumbered upon the Apache Hadoop Project are hereafter discharged. On Thu, Oct 21, 2010 at 10:44 AM, Henry Robinson he...@cloudera.com wrote: Looks good, please do call a vote. On 21 October 2010 09:29, Patrick Hunt ph...@apache.org wrote: Here's a draft board resolution (not a vote, just discussion). It lists all current committers (except as noted in the next paragraph) as the initial members of the project management committee (PMC) and myself as the initial chair. Notice that I have left Andrew off the PMC as he has not been active with the project for over two years. I believe we should continue to include him on the committer roles subsequent to moving to tlp, however as he has not been an active member of the community for such a long period we would not include him on the PMC at this time. If others feel differently let me know, I'm willing to include him if the people feel differently. LMK if this looks good to you and I'll call for an official vote on this list (then we'll be ready to call a vote on the hadoop pmc). Regards, Patrick X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is responsible for the creation and maintenance of software related to data serialization; and be it
Re: Restarting discussion on ZooKeeper as a TLP
NOW, THEREFORE, BE IT FURTHER RESOLVED, that Matt Massie be appointed to the office of Vice President, Apache ZooKeeper, to I think you meant Patrick Hunt ? :) Other than that it looks good. Thanks mahadev On 10/21/10 1:28 PM, Patrick Hunt ph...@apache.org wrote: Ack, I missed Henry in the list, sorry! In my defense I copied this: http://hadoop.apache.org/zookeeper/credits.html one more try (same as before except for adding henry to the pmc): X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is responsible for the creation and maintenance of software related to data serialization; and be it further RESOLVED, that the office of Vice President, Apache ZooKeeper be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache ZooKeeper Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache ZooKeeper Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache ZooKeeper Project: * Patrick Hunt ph...@apache.org * Flavio Junqueira f...@apache.org * Mahadev Konarmaha...@apache.org * Benjamin Reedbr...@apache.org * Henry Robinson he...@apache.org NOW, THEREFORE, BE IT FURTHER RESOLVED, that Matt Massie be appointed to the office of Vice President, Apache ZooKeeper, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the initial Apache ZooKeeper PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache ZooKeeper Project; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is tasked with the migration and rationalization of the Apache Hadoop ZooKeeper sub-project; and be it further RESOLVED, that all responsibilities pertaining to the Apache Hadoop ZooKeeper sub-project encumbered upon the Apache Hadoop Project are hereafter discharged. On Thu, Oct 21, 2010 at 10:44 AM, Henry Robinson he...@cloudera.com wrote: Looks good, please do call a vote. On 21 October 2010 09:29, Patrick Hunt ph...@apache.org wrote: Here's a draft board resolution (not a vote, just discussion). It lists all current committers (except as noted in the next paragraph) as the initial members of the project management committee (PMC) and myself as the initial chair. Notice that I have left Andrew off the PMC as he has not been active with the project for over two years. I believe we should continue to include him on the committer roles subsequent to moving to tlp, however as he has not been an active member of the community for such a long period we would not include him on the PMC at this time. If others feel differently let me know, I'm willing to include him if the people feel differently. LMK if this looks good to you and I'll call for an official vote on this list (then we'll be ready to call a vote on the hadoop pmc). Regards, Patrick X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is
[jira] Commented: (ZOOKEEPER-907) Spurious KeeperErrorCode = Session moved messages
[ https://issues.apache.org/jira/browse/ZOOKEEPER-907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12923627#action_12923627 ] Vishal K commented on ZOOKEEPER-907: Ben, What do you mean by partial sync? Will the follower still sync with the leader? Can you explain how sync works? I was going throught the code flow. I traced the sync call, but I could find the place that performs the data transfer between the leader and follwer. I will cleanup the patch. As per the test goes, I am not sure what is the best way to check if a server has received a request with null owner. I was planning to add a test that will initiate all zk client requests. In SessionTrackerImpl.checkesession(), set a flag if it sees a null owner. At the end of the test, fail the test if the flag is set. Any suggestions? -Vishal Spurious KeeperErrorCode = Session moved messages --- Key: ZOOKEEPER-907 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-907 Project: Zookeeper Issue Type: Bug Affects Versions: 3.3.1 Reporter: Vishal K Assignee: Vishal K Priority: Blocker Fix For: 3.3.2, 3.4.0 Attachments: ZOOKEEPER-907.patch The sync request does not set the session owner in Request. As a result, the leader keeps printing: 2010-07-01 10:55:36,733 - INFO [ProcessThread:-1:preprequestproces...@405] - Got user-level KeeperException when processing sessionid:0x298d3b1fa9 type:sync: cxid:0x6 zxid:0xfffe txntype:unknown reqpath:/ Error Path:null Error:KeeperErrorCode = Session moved -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-800) zoo_add_auth returns ZOK if zookeeper handle is in ZOO_CLOSED_STATE
[ https://issues.apache.org/jira/browse/ZOOKEEPER-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated ZOOKEEPER-800: Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) +1 I just committed this. thanks michi. zoo_add_auth returns ZOK if zookeeper handle is in ZOO_CLOSED_STATE --- Key: ZOOKEEPER-800 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-800 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.3.1 Reporter: Michi Mutsuzaki Assignee: Michi Mutsuzaki Priority: Minor Fix For: 3.3.2, 3.4.0 Attachments: ZOOKEEPER-800.patch This happened when I called zoo_add_auth() immediately after zookeeper_init(). It took me a while to figure out that authentication actually failed since zoo_add_auth() returned ZOK. It should return ZINVALIDSTATE instead. --Michi -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Restarting discussion on ZooKeeper as a TLP
Damn, not sure how that happened. Had it right originally, one undo too many? weird. Give this one a try (henry added, chair back as expected): X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is responsible for the creation and maintenance of software related to data serialization; and be it further RESOLVED, that the office of Vice President, Apache ZooKeeper be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache ZooKeeper Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache ZooKeeper Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache ZooKeeper Project: * Patrick Hunt ph...@apache.org * Flavio Junqueira f...@apache.org * Mahadev Konarmaha...@apache.org * Benjamin Reedbr...@apache.org * Henry Robinson he...@apache.org NOW, THEREFORE, BE IT FURTHER RESOLVED, that Patrick Hunt be appointed to the office of Vice President, Apache ZooKeeper, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the initial Apache ZooKeeper PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache ZooKeeper Project; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is tasked with the migration and rationalization of the Apache Hadoop ZooKeeper sub-project; and be it further RESOLVED, that all responsibilities pertaining to the Apache Hadoop ZooKeeper sub-project encumbered upon the Apache Hadoop Project are hereafter discharged. 2010/10/21 Mahadev Konar maha...@yahoo-inc.com NOW, THEREFORE, BE IT FURTHER RESOLVED, that Matt Massie be appointed to the office of Vice President, Apache ZooKeeper, to I think you meant Patrick Hunt ? :) Other than that it looks good. Thanks mahadev On 10/21/10 1:28 PM, Patrick Hunt ph...@apache.org wrote: Ack, I missed Henry in the list, sorry! In my defense I copied this: http://hadoop.apache.org/zookeeper/credits.html one more try (same as before except for adding henry to the pmc): X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is responsible for the creation and maintenance of software related to data serialization; and be it further RESOLVED, that the office of Vice President, Apache ZooKeeper be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache ZooKeeper Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache ZooKeeper Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache ZooKeeper Project: * Patrick Hunt ph...@apache.org * Flavio Junqueira f...@apache.org * Mahadev Konar
Re: Restarting discussion on ZooKeeper as a TLP
i think we want to be responsible for the creation and maintenance of software related to distributed system coordination. ben On 10/21/2010 01:43 PM, Mahadev Konar wrote: NOW, THEREFORE, BE IT FURTHER RESOLVED, that Matt Massie be appointed to the office of Vice President, Apache ZooKeeper, to I think you meant Patrick Hunt ? :) Other than that it looks good. Thanks mahadev On 10/21/10 1:28 PM, Patrick Hunt ph...@apache.org wrote: Ack, I missed Henry in the list, sorry! In my defense I copied this: http://hadoop.apache.org/zookeeper/credits.html one more try (same as before except for adding henry to the pmc): X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is responsible for the creation and maintenance of software related to data serialization; and be it further RESOLVED, that the office of Vice President, Apache ZooKeeper be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache ZooKeeper Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache ZooKeeper Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache ZooKeeper Project: * Patrick Hunt ph...@apache.org * Flavio Junqueira f...@apache.org * Mahadev Konarmaha...@apache.org * Benjamin Reedbr...@apache.org * Henry Robinson he...@apache.org NOW, THEREFORE, BE IT FURTHER RESOLVED, that Matt Massie be appointed to the office of Vice President, Apache ZooKeeper, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the initial Apache ZooKeeper PMC be and hereby is tasked with the creation of a set of bylaws intended to encourage open development and increased participation in the Apache ZooKeeper Project; and be it further RESOLVED, that the Apache ZooKeeper Project be and hereby is tasked with the migration and rationalization of the Apache Hadoop ZooKeeper sub-project; and be it further RESOLVED, that all responsibilities pertaining to the Apache Hadoop ZooKeeper sub-project encumbered upon the Apache Hadoop Project are hereafter discharged. On Thu, Oct 21, 2010 at 10:44 AM, Henry Robinson he...@cloudera.com wrote: Looks good, please do call a vote. On 21 October 2010 09:29, Patrick Hunt ph...@apache.org wrote: Here's a draft board resolution (not a vote, just discussion). It lists all current committers (except as noted in the next paragraph) as the initial members of the project management committee (PMC) and myself as the initial chair. Notice that I have left Andrew off the PMC as he has not been active with the project for over two years. I believe we should continue to include him on the committer roles subsequent to moving to tlp, however as he has not been an active member of the community for such a long period we would not include him on the PMC at this time. If others feel differently let me know, I'm willing to include him if the people feel differently. LMK if this looks good to you and I'll call for an official vote on this list (then we'll be ready to call a vote on the hadoop pmc). Regards, Patrick X. Establish the Apache ZooKeeper Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to data serialization for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the Apache ZooKeeper Project, be and
Re: What's the magic behind lenBuffer and incomingBuffer?
look in readLength(). incomingBuffer is set to a newly allocated
ByteBuffer.
ben
On 10/21/2010 07:52 AM, Thomas Koch wrote:
Hi,
inside ClientCnxn.SendThread we have
final ByteBuffer lenBuffer = ByteBuffer.allocateDirect(4);
ByteBuffer incomingBuffer = lenBuffer;
So incomingBuffer and lenBuffer do refer to the same object. There are several
other places where lenBuffer is again assigned to incomingBuffer.
Now inside the doIO() method we got
if (incomingBuffer == lenBuffer) {
recvCount++;
readLength();
} else if (!initialized) {
incomingBuffer is never assigned anything else then lenBuffer, lenBuffer stays
the same all the time. So as far as my knowledge of java reaches (which may
not be too far) incomingBuffer == lenBuffer _always_ evaluates to true. Isn't
that true?
So effectively we've got dead code in the elseif and else branches, didn't we?
Best regards,
Thomas Koch, http://www.koch.ro
[jira] Updated: (ZOOKEEPER-909) Extract NIO specific code from ClientCnxn
[ https://issues.apache.org/jira/browse/ZOOKEEPER-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Hunt updated ZOOKEEPER-909: --- Status: Patch Available (was: Open) Extract NIO specific code from ClientCnxn - Key: ZOOKEEPER-909 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-909 Project: Zookeeper Issue Type: Sub-task Components: java client Reporter: Thomas Koch Assignee: Thomas Koch Fix For: 3.4.0 Attachments: ZOOKEEPER-909.patch, ZOOKEEPER-909.patch This patch is mostly the same patch as my last one for ZOOKEEPER-823 minus everything Netty related. This means this patch only extract all NIO specific code in the class ClientCnxnSocketNIO which extends ClientCnxnSocket. I've redone this patch from current trunk step by step now and couldn't find any logical error. I've already done a couple of successful test runs and will continue to do so this night. It would be nice, if we could apply this patch as soon as possible to trunk. This allows us to continue to work on the netty integration without blocking the ClientCnxn class. Adding Netty after this patch should be only a matter of adding the ClientCnxnSocketNetty class with the appropriate test cases. You could help me by reviewing the patch and by running it on whatever test server you have available. Please send me any complete failure log you should encounter to thomas at koch point ro. Thx! Update: Until now, I've collected 8 successful builds in a row! -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
