Re: Restarting discussion on ZooKeeper as a TLP

2010-10-13 Thread Jeff Hammerbacher
Excited to see ZooKeeper graduate to TLP.

On Wed, Oct 13, 2010 at 2:10 PM, Patrick Hunt  wrote:

> In March of this year we discussed a request from the Apache Board, and
> Hadoop PMC, that we become a TLP rather than a subproject of Hadoop:
>
> Original discussion
> http://markmail.org/thread/42cobkpzlgotcbin
>
> I originally voted against this move, my primary concern being that we were
> not "ready" to move to tlp status given our small contributor base and
> limited contributor diversity. However I'd now like to revisit that
> discussion/decision. Since that time the team has been working hard to
> attract new contributors, and we've seen significant new contributions come
> in. There has also been feedback from board/pmc addressing many of these
> concerns (both on the list and in private). I am now less concerned about
> this issue and don't see it as a blocker for us to move to TLP status.
>
> A second concern was that by becoming a TLP the project would lose it's
> connection with Hadoop, a big source of new users for us. I've been assured
> (and you can see with the other projects that have moved to tlp status;
> pig/hive/hbase/etc...) that this connection will be maintained. The Hadoop
> ZooKeeper tab for example will redirect to our new homepage.
>
> Other Apache members also pointed out to me that we are essentially
> operating as a TLP within the Hadoop PMC. Most of the other PMC members
> have
> little or no experience with ZooKeeper and this makes it difficult for them
> to monitor and advise us. By moving to TLP status we'll be able to govern
> ourselves and better set our direction.
>
> I believe we are ready to become a TLP. Please respond to this email with
> your thoughts and any issues. I will call a vote in a few days, once
> discussion settles.
>
> Regards,
>
> Patrick
>


Restarting discussion on ZooKeeper as a TLP

2010-10-13 Thread Patrick Hunt
In March of this year we discussed a request from the Apache Board, and
Hadoop PMC, that we become a TLP rather than a subproject of Hadoop:

Original discussion
http://markmail.org/thread/42cobkpzlgotcbin

I originally voted against this move, my primary concern being that we were
not "ready" to move to tlp status given our small contributor base and
limited contributor diversity. However I'd now like to revisit that
discussion/decision. Since that time the team has been working hard to
attract new contributors, and we've seen significant new contributions come
in. There has also been feedback from board/pmc addressing many of these
concerns (both on the list and in private). I am now less concerned about
this issue and don't see it as a blocker for us to move to TLP status.

A second concern was that by becoming a TLP the project would lose it's
connection with Hadoop, a big source of new users for us. I've been assured
(and you can see with the other projects that have moved to tlp status;
pig/hive/hbase/etc...) that this connection will be maintained. The Hadoop
ZooKeeper tab for example will redirect to our new homepage.

Other Apache members also pointed out to me that we are essentially
operating as a TLP within the Hadoop PMC. Most of the other PMC members have
little or no experience with ZooKeeper and this makes it difficult for them
to monitor and advise us. By moving to TLP status we'll be able to govern
ourselves and better set our direction.

I believe we are ready to become a TLP. Please respond to this email with
your thoughts and any issues. I will call a vote in a few days, once
discussion settles.

Regards,

Patrick


[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections

2010-10-13 Thread Thomas Koch (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920750#action_12920750
 ] 

Thomas Koch commented on ZOOKEEPER-823:
---

The problem with the failing testWatchAutoResetWithPending might be the 
following. The test uses testableZooKeeper.pauseCnxn($milliSeconds) 
PauseCnxn tries to block the ClientCnxn by holding a lock on the ClientCnxn 
instance, interrupting the connection and not releasing the lock for 
$milliSeconds.
Actually I wonder how this could have ever worked reliable in the first place. 
The only thing I see that also synchronizes on ClientCnxn is the getXid() 
method.
The solution might be to find a better implementation for 
testableZooKeeper.pauseCnxn

> update ZooKeeper java client to optionally use Netty for connections
> 
>
> Key: ZOOKEEPER-823
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823
> Project: Zookeeper
>  Issue Type: New Feature
>  Components: java client
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
> Fix For: 3.4.0
>
> Attachments: NettyNettySuiteTest.rtf, 
> TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, 
> testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch
>
>
> This jira will port the client side connection code to use netty rather than 
> direct nio.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections

2010-10-13 Thread Patrick Hunt (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920737#action_12920737
 ] 

Patrick Hunt commented on ZOOKEEPER-823:


Thomas (also Mahadev and other committers)

Here's an idea. What if we setup a branch in SVN with these changes (netty). It 
would be a short lived branch. Then we can setup a new job on apache hudson, 
and just keep hammering on that thing until we get all the issues (intermittent 
bugs) resolved. 

How's that sound?

An alternative would be to use Git but apache hudson doesn't allow that. I'd 
create/maintain the svn branch (you need to be a committer).

> update ZooKeeper java client to optionally use Netty for connections
> 
>
> Key: ZOOKEEPER-823
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823
> Project: Zookeeper
>  Issue Type: New Feature
>  Components: java client
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
> Fix For: 3.4.0
>
> Attachments: NettyNettySuiteTest.rtf, 
> TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, 
> testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch
>
>
> This jira will port the client side connection code to use netty rather than 
> direct nio.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections

2010-10-13 Thread Patrick Hunt (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920734#action_12920734
 ] 

Patrick Hunt commented on ZOOKEEPER-823:


Sorry man, but I checked that, a few things indicate that it's nio and not 
netty:

1) The log would have something like this in it:
 [junit] Running org.apache.zookeeper.test.NettyNettySuiteHammerTest

which I don't see prior to this failure (running ... *Netty*

2) I can see log messages from NIO and not from netty in the log (NIO class 
loggers I mean)

3) I'm pretty sure (don't have one on hand) that if the *Netty*Test fails you 
see that in the stack trace.

> update ZooKeeper java client to optionally use Netty for connections
> 
>
> Key: ZOOKEEPER-823
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823
> Project: Zookeeper
>  Issue Type: New Feature
>  Components: java client
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
> Fix For: 3.4.0
>
> Attachments: NettyNettySuiteTest.rtf, 
> TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, 
> testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch
>
>
> This jira will port the client side connection code to use netty rather than 
> direct nio.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



Re: What's the QA strategy of ZooKeeper?

2010-10-13 Thread Patrick Hunt
On Wed, Oct 13, 2010 at 6:18 AM, Thomas Koch  wrote:

> I'd propose to have a word about quality assurance. Is there already a
> strategy to ensure the ongoing maintainability of ZK? Is there a code style
> guide, a list of Dos-And-Donts (where I'd like to add some points)?
>
>
This wiki page has some of the detail you're looking for (code style,
testing and such)
http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute


> Should PMD be added to Hudson? What is the level of FindBugs? Should it be
> raised?
>

It could be, but the issue I've seen with "add tool X" is that it really
needs to be "add tool X, configure it reasonably, then hunt down all the
issues you just found and require that ppl not introduce any going fwd,
setup automated tests to enforce this going fwd". Other wise it becomes just
another report that ppl ignore. We had similar issues with findbugs, until
recently hadoop decided to make it a priority, everyone went "pencils down"
for a few weeks and resolved all the issues. Also put guidelines in place
(and automated testing) to ensure no new issues come in. It's a good idea.
but not a panacea.


> Some of the points I'd like to add to a style guide:
>
> - Don't write methods longer then 20-40 lines of code
>
> - Are you sure you want to use inner classes?
>
> - If there is a new operator in a method? Could the method maybe already
> receive the object as a parameter?
>
> - Are you sure you want to use system properties? They are like global
> variables and the IDE does not know about them
>
> - Are you sure you want to extend a class? Often an aggregation is more
> elegant.
>
> - Don't nest ifs and loops deeper then 2 or 3 levels. If you do so, you
> should
> better break your code into more methods.
>
> - Use Enums or constants instead of plain status integers
>
> - please document your intentions in code comments. You don't need to
> comment
> the what? but the why?.
>
> Do you agree with me, that there is a need for better code quality in
> ZooKeeper? If so, it's not really scalable if a manic like me fights like
> Don
> Quichotte to clean up the code. All developers would need to establish a
> sense
> for clean code and constantly improve the code.
>

I suspect we are in general agreement, however there are realities to be
faced:
1) small team
2) semi-large existing codebase, lots of legacy in there.
3) existing user base

Are we open to fixing things, yes.. Do we want to improve things, many in
line with what you suggest, yes (ask mahadev, he/I talk about it frequently)
But at the same time we can't break existing functionality, we don't like to
destabilize the code (and what you are suggesting will cause instability)
because then users really get upset.

This netty set of changes is not helping things either. I suspect if we can
get that out of the way it would help alot, allowing other refactorings to
be done much more smoothly.

Patrick


>
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-835
>
> Best regards,
>
> Thomas Koch, http://www.koch.ro
>


[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections

2010-10-13 Thread Thomas Koch (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920715#action_12920715
 ] 

Thomas Koch commented on ZOOKEEPER-823:
---

Hi Patrick,

I saw the same exceptions as you saw. They are not _thrown_ inside the netty 
code, but they occurred in my cases every time during the NettyNettySuiteTest. 
It isn't very obvious from the junit log, whether a test was run as part of the 
NettyNettySuiteTest or not. I do a forward search for "FAILED" and from this 
position a backwards search for "running" to find the name of the actual test 
suite.
Are you perfecty sure, the failed tests were not run as part of the 
NettyNettySuiteTest?

> update ZooKeeper java client to optionally use Netty for connections
> 
>
> Key: ZOOKEEPER-823
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823
> Project: Zookeeper
>  Issue Type: New Feature
>  Components: java client
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
> Fix For: 3.4.0
>
> Attachments: NettyNettySuiteTest.rtf, 
> TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, 
> testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch
>
>
> This jira will port the client side connection code to use netty rather than 
> direct nio.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (ZOOKEEPER-885) Zookeeper drops connections under moderate IO load

2010-10-13 Thread Flavio Junqueira (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920713#action_12920713
 ] 

Flavio Junqueira commented on ZOOKEEPER-885:


I remember a while back fixing an issue with CommitProcessor, which was being 
killed by a runtime exception. As Pat pointed out, it does look like the 
pipeline is stalling, but it is still unclear why and I couldn't find anything 
that can indicate the cause. 

Let me try to reproduce it.

> Zookeeper drops connections under moderate IO load
> --
>
> Key: ZOOKEEPER-885
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-885
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.2.2, 3.3.1
> Environment: Debian (Lenny)
> 1Gb RAM
> swap disabled
> 100Mb heap for zookeeper
>Reporter: Alexandre Hardy
>Priority: Critical
> Attachments: tracezklogs.tar.gz, tracezklogs.tar.gz, 
> WatcherTest.java, zklogs.tar.gz
>
>
> A zookeeper server under minimum load, with a number of clients watching 
> exactly one node will fail to maintain the connection when the machine is 
> subjected to moderate IO load.
> In a specific test example we had three zookeeper servers running on 
> dedicated machines with 45 clients connected, watching exactly one node. The 
> clients would disconnect after moderate load was added to each of the 
> zookeeper servers with the command:
> {noformat}
> dd if=/dev/urandom of=/dev/mapper/nimbula-test
> {noformat}
> The {{dd}} command transferred data at a rate of about 4Mb/s.
> The same thing happens with
> {noformat}
> dd if=/dev/zero of=/dev/mapper/nimbula-test
> {noformat}
> It seems strange that such a moderate load should cause instability in the 
> connection.
> Very few other processes were running, the machines were setup to test the 
> connection instability we have experienced. Clients performed no other read 
> or mutation operations.
> Although the documents state that minimal competing IO load should present on 
> the zookeeper server, it seems reasonable that moderate IO should not cause 
> problems in this case.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (ZOOKEEPER-804) c unit tests failing due to "assertion cptr failed"

2010-10-13 Thread Jared Cantwell (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920689#action_12920689
 ] 

Jared Cantwell commented on ZOOKEEPER-804:
--

It seems like zookeeper_process unnecessarily calls api_prolog() and 
api_epilog() to begin with.  But given that api_prolog() is called at the 
beginning, if this new code path executes while a close is requested, then the 
threads will successfully exit, but the final part of zookeeper_close that 
releases the memory on the last reference will not execute (since the last 
reference will never be reached).  Unless I'm missing something, this will leak 
the zkhandle.

> c unit tests failing due to "assertion cptr failed"
> ---
>
> Key: ZOOKEEPER-804
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-804
> Project: Zookeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.0
> Environment: gcc 4.4.3, ubuntu lucid lynx, dual core laptop (intel)
>Reporter: Patrick Hunt
>Assignee: Michi Mutsuzaki
>Priority: Critical
> Fix For: 3.3.2, 3.4.0
>
> Attachments: ZOOKEEPER-804.patch
>
>
> I'm seeing this frequently:
>  [exec] Zookeeper_simpleSystem::testPing : elapsed 18006 : OK
>  [exec] Zookeeper_simpleSystem::testAcl : elapsed 1022 : OK
>  [exec] Zookeeper_simpleSystem::testChroot : elapsed 3145 : OK
>  [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started : 
> elapsed 25687 : OK
>  [exec] zktest-mt: 
> /home/phunt/dev/workspace/gitzk/src/c/src/zookeeper.c:1952: 
> zookeeper_process: Assertion `cptr' failed.
>  [exec] make: *** [run-check] Aborted
>  [exec] Zookeeper_simpleSystem::testHangingClient
> Mahadev can you take a look?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections

2010-10-13 Thread Patrick Hunt (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920686#action_12920686
 ] 

Patrick Hunt commented on ZOOKEEPER-823:


I ran the latest patch 4 times on my hudson setup and saw 3 failures (1 each):

java.util.concurrent.TimeoutException: Did not connect
at 
org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(ClientBase.java:124)
at 
org.apache.zookeeper.test.WatcherTest.testWatchAutoResetWithPending(WatcherTest.java:199)
at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:51)

org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = 
ConnectionLoss for /
at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.setACL(ZooKeeper.java:1259)
at 
org.apache.zookeeper.test.ACLTest.testDisconnectedAddAuth(ACLTest.java:67)
at 
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:51)


[junit] 2010-10-13 09:16:42,245 [myid:] - INFO  
[main:junit4zktestrunner$loggedinvokemet...@53] - TEST METHOD FAILED testAcls
[junit] org.apache.zookeeper.KeeperException$ConnectionLossException: 
KeeperErrorCode = ConnectionLoss for /0
[junit] at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
[junit] at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
[junit] at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:640)
[junit] at org.apache.zookeeper.test.ACLTest.testAcls(ACLTest.java:104)


If this was failing in the Netty code I'd be inclined to push the code to trunk 
and let things settle. However this is
in the mainline NIO code and I'm reticent to break existing functionality.


Perhaps a good idea would be to have Ben/Mahadev/Flavio review the patch and 
see if they notice any issues.

I'd really like to get this in (esp as it's blocking other changes), so if we 
could review this and resolve the issues asap that would be great.



> update ZooKeeper java client to optionally use Netty for connections
> 
>
> Key: ZOOKEEPER-823
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823
> Project: Zookeeper
>  Issue Type: New Feature
>  Components: java client
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
> Fix For: 3.4.0
>
> Attachments: NettyNettySuiteTest.rtf, 
> TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, 
> testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch
>
>
> This jira will port the client side connection code to use netty rather than 
> direct nio.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (ZOOKEEPER-894) add Package o.a.zookeeper.client

2010-10-13 Thread Thomas Koch (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Koch updated ZOOKEEPER-894:
--

Description: 
I'd like to move classes that are not part of the API but belong to the ZK 
Client into a separate Client package. These classes are:

- Inner classes that should become normal classes:
  Zookeeper.ZkWatchManager
  Zookeeper.WatchRegistration
  ClientCnxn.SendThread (should become a Runnable anyhow)
  ClientCnxn.EventThread
  ClientCnxn.Package
  ClientCnxn.AuthData ?

- Classes now in the zookeeper package:
  ClientCnxn -> Client.Cnxn
  ClientCnxnSocket* -> Client.CnxnSocket*
  ... Maybe some others that can be moved without breaking the API

- Classes yet to be written:
  PendingQueue ?
  OutgoingQueue ?

  was:
I'd like to move classes that are not part of the API but belong to the ZK 
Client into a separate Client package. These classes are:

- Inner classes that should become normal classes:
  Zookeeper.ZkWatchManager
  Zookeeper.WatchRegistration
  ClientCnxn.SendThread (should become a Runnable anyhow)
  ClientCnxn.EventThread
  ClientCnxn.Package

- Classes now in the Zookeeper package:
  ClientCnxn -> Client.Cnxn
  ClientCnxnSocket* -> Client.CnxnSocket*
  ... Maybe some others that can be moved without breaking the API

- Classes yet to be written:
  PendingQueue (?)
  OutgoingQueue (?)

Summary: add Package o.a.zookeeper.client  (was: add Package 
o.a.Zookeeper.Client)

> add Package o.a.zookeeper.client
> 
>
> Key: ZOOKEEPER-894
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-894
> Project: Zookeeper
>  Issue Type: Sub-task
>Reporter: Thomas Koch
>
> I'd like to move classes that are not part of the API but belong to the ZK 
> Client into a separate Client package. These classes are:
> - Inner classes that should become normal classes:
>   Zookeeper.ZkWatchManager
>   Zookeeper.WatchRegistration
>   ClientCnxn.SendThread (should become a Runnable anyhow)
>   ClientCnxn.EventThread
>   ClientCnxn.Package
>   ClientCnxn.AuthData ?
> - Classes now in the zookeeper package:
>   ClientCnxn -> Client.Cnxn
>   ClientCnxnSocket* -> Client.CnxnSocket*
>   ... Maybe some others that can be moved without breaking the API
> - Classes yet to be written:
>   PendingQueue ?
>   OutgoingQueue ?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Resolved: (ZOOKEEPER-890) C client invokes watcher callbacks multiple times

2010-10-13 Thread Austin Shoemaker (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Austin Shoemaker resolved ZOOKEEPER-890.


Resolution: Not A Problem

Closing, C client works as intended. Submitted a patch in ZOOKEEPER-888 to 
handle this properly in zkpython.

> C client invokes watcher callbacks multiple times
> -
>
> Key: ZOOKEEPER-890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-890
> Project: Zookeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.3.1
> Environment: Mac OS X 10.6.5
>Reporter: Austin Shoemaker
>Priority: Critical
> Attachments: watcher_twice.c, ZOOKEEPER-890.patch
>
>
> Code using the C client assumes that watcher callbacks are called exactly 
> once. If the watcher is called more than once, the process will likely 
> overwrite freed memory and/or crash.
> collect_session_watchers (zk_hashtable.c) gathers watchers from 
> active_node_watchers, active_exist_watchers, and active_child_watchers 
> without removing them. This results in watchers being invoked more than once.
> Test code is attached that reproduces the bug, along with a proposed patch.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (ZOOKEEPER-885) Zookeeper drops connections under moderate IO load

2010-10-13 Thread Patrick Hunt (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920655#action_12920655
 ] 

Patrick Hunt commented on ZOOKEEPER-885:


I notice this in the server log (continuous log):

2010-10-08 08:46:24,789 - DEBUG [ProcessThread:-1:commitproces...@169] - 
Processing request:: sessionid:0x32b8b03e21e000c type:ping 
cxid:0xfffe zxid:0xfffe txntype:unknown reqpath:n/a
2010-10-08 08:46:24,789 - DEBUG [CommitProcessor:3:finalrequestproces...@78] - 
Processing request:: sessionid:0x32b8b03e21e000c type:ping 
cxid:0xfffe zxid:0xfffe txntype:unknown reqpath:n/a
2010-10-08 08:46:24,789 - DEBUG [CommitProcessor:3:finalrequestproces...@160] - 
sessionid:0x32b8b03e21e000c type:ping cxid:0xfffe 
zxid:0xfffe txntype:unknown reqpath:n/a
2010-10-08 08:46:27,986 - DEBUG [ProcessThread:-1:commitproces...@169] - 
Processing request:: sessionid:0x32b8b03e21e0001 type:ping 
cxid:0xfffe zxid:0xfffe txntype:unknown reqpath:n/a
2010-10-08 08:46:38,000 - INFO  [SessionTracker:zookeeperser...@315] - Expiring 
session 0x32b8b03e21e0004, timeout of 1ms exceeded
2010-10-08 08:46:46,471 - INFO  [SessionTracker:zookeeperser...@315] - Expiring 
session 0x32b8b03e21e0003, timeout of 1ms exceeded
2010-10-08 08:47:00,083 - INFO  [SessionTracker:zookeeperser...@315] - Expiring 
session 0x32b8b03e21e0001, timeout of 1ms exceeded

It looks to me like the pipeline is getting stalled after 0x32b8b03e21e000c. 
Notice that 0x32b8b03e21e0001 sends in a ping which gets to the commit 
processor, but you never see the "final request processor" messages (before the 
session expires). Notice that 30seconds are elapsing here.

> Zookeeper drops connections under moderate IO load
> --
>
> Key: ZOOKEEPER-885
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-885
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.2.2, 3.3.1
> Environment: Debian (Lenny)
> 1Gb RAM
> swap disabled
> 100Mb heap for zookeeper
>Reporter: Alexandre Hardy
>Priority: Critical
> Attachments: tracezklogs.tar.gz, tracezklogs.tar.gz, 
> WatcherTest.java, zklogs.tar.gz
>
>
> A zookeeper server under minimum load, with a number of clients watching 
> exactly one node will fail to maintain the connection when the machine is 
> subjected to moderate IO load.
> In a specific test example we had three zookeeper servers running on 
> dedicated machines with 45 clients connected, watching exactly one node. The 
> clients would disconnect after moderate load was added to each of the 
> zookeeper servers with the command:
> {noformat}
> dd if=/dev/urandom of=/dev/mapper/nimbula-test
> {noformat}
> The {{dd}} command transferred data at a rate of about 4Mb/s.
> The same thing happens with
> {noformat}
> dd if=/dev/zero of=/dev/mapper/nimbula-test
> {noformat}
> It seems strange that such a moderate load should cause instability in the 
> connection.
> Very few other processes were running, the machines were setup to test the 
> connection instability we have experienced. Clients performed no other read 
> or mutation operations.
> Although the documents state that minimal competing IO load should present on 
> the zookeeper server, it seems reasonable that moderate IO should not cause 
> problems in this case.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (ZOOKEEPER-885) Zookeeper drops connections under moderate IO load

2010-10-13 Thread Patrick Hunt (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920646#action_12920646
 ] 

Patrick Hunt commented on ZOOKEEPER-885:


Flavio can you take a look at the updated logs? thanks.

> Zookeeper drops connections under moderate IO load
> --
>
> Key: ZOOKEEPER-885
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-885
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.2.2, 3.3.1
> Environment: Debian (Lenny)
> 1Gb RAM
> swap disabled
> 100Mb heap for zookeeper
>Reporter: Alexandre Hardy
>Priority: Critical
> Attachments: tracezklogs.tar.gz, tracezklogs.tar.gz, 
> WatcherTest.java, zklogs.tar.gz
>
>
> A zookeeper server under minimum load, with a number of clients watching 
> exactly one node will fail to maintain the connection when the machine is 
> subjected to moderate IO load.
> In a specific test example we had three zookeeper servers running on 
> dedicated machines with 45 clients connected, watching exactly one node. The 
> clients would disconnect after moderate load was added to each of the 
> zookeeper servers with the command:
> {noformat}
> dd if=/dev/urandom of=/dev/mapper/nimbula-test
> {noformat}
> The {{dd}} command transferred data at a rate of about 4Mb/s.
> The same thing happens with
> {noformat}
> dd if=/dev/zero of=/dev/mapper/nimbula-test
> {noformat}
> It seems strange that such a moderate load should cause instability in the 
> connection.
> Very few other processes were running, the machines were setup to test the 
> connection instability we have experienced. Clients performed no other read 
> or mutation operations.
> Although the documents state that minimal competing IO load should present on 
> the zookeeper server, it seems reasonable that moderate IO should not cause 
> problems in this case.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (ZOOKEEPER-894) add Package o.a.Zookeeper.Client

2010-10-13 Thread Patrick Hunt (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920645#action_12920645
 ] 

Patrick Hunt commented on ZOOKEEPER-894:


seems reasonable to me, however can you name the package o.a.zookeeper.client 
instead? (note all lower case. plz update the summary)

> add Package o.a.Zookeeper.Client
> 
>
> Key: ZOOKEEPER-894
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-894
> Project: Zookeeper
>  Issue Type: Sub-task
>Reporter: Thomas Koch
>
> I'd like to move classes that are not part of the API but belong to the ZK 
> Client into a separate Client package. These classes are:
> - Inner classes that should become normal classes:
>   Zookeeper.ZkWatchManager
>   Zookeeper.WatchRegistration
>   ClientCnxn.SendThread (should become a Runnable anyhow)
>   ClientCnxn.EventThread
>   ClientCnxn.Package
> - Classes now in the Zookeeper package:
>   ClientCnxn -> Client.Cnxn
>   ClientCnxnSocket* -> Client.CnxnSocket*
>   ... Maybe some others that can be moved without breaking the API
> - Classes yet to be written:
>   PendingQueue (?)
>   OutgoingQueue (?)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Assigned: (ZOOKEEPER-893) ZooKeeper high cpu usage when invalid requests

2010-10-13 Thread Patrick Hunt (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt reassigned ZOOKEEPER-893:
--

Assignee: Thijs Terlouw

> ZooKeeper high cpu usage when invalid requests
> --
>
> Key: ZOOKEEPER-893
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-893
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.3.1
> Environment: Linux 2.6.16
> 4x Intel(R) Xeon(R) CPU X3320  @ 2.50GHz
> java version "1.6.0_17"
> Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
> Java HotSpot(TM) Server VM (build 14.3-b01, mixed mode)
>Reporter: Thijs Terlouw
>Assignee: Thijs Terlouw
>Priority: Critical
> Fix For: 3.3.2, 3.4.0
>
> Attachments: ZOOKEEPER-893.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> When ZooKeeper receives certain illegally formed messages on the internal 
> communication port (:4181 by default), it's possible for ZooKeeper to enter 
> an infinite loop which causes 100% cpu usage. It's related to ZOOKEEPER-427, 
> but that patch does not resolve all issues.
> from: src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java 
> the two affected parts:
> ===
> int length = msgLength.getInt();  
>   
> if(length <= 0) { 
>   
> throw new IOException("Invalid packet length:" + length); 
>   
> } 
> ===
> ===
> while (message.hasRemaining()) {  
>   
> temp_numbytes = channel.read(message);
>   
> if(temp_numbytes < 0) {   
>   
> throw new IOException("Channel eof before end");  
>   
> } 
>   
> numbytes += temp_numbytes;
>   
> } 
> ===
> how to replicate this bug:
> perform an nmap portscan against your zookeeper server: "nmap -sV -n 
> your.ip.here -p4181"
> wait for a while untill you see some messages in the logfile and then you 
> will see 100% cpu usage. It does not recover from this situation. With my 
> patch, it does not occur anymore

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (ZOOKEEPER-893) ZooKeeper high cpu usage when invalid requests

2010-10-13 Thread Patrick Hunt (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-893:
---

Fix Version/s: 3.4.0
   3.3.2

> ZooKeeper high cpu usage when invalid requests
> --
>
> Key: ZOOKEEPER-893
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-893
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.3.1
> Environment: Linux 2.6.16
> 4x Intel(R) Xeon(R) CPU X3320  @ 2.50GHz
> java version "1.6.0_17"
> Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
> Java HotSpot(TM) Server VM (build 14.3-b01, mixed mode)
>Reporter: Thijs Terlouw
>Priority: Critical
> Fix For: 3.3.2, 3.4.0
>
> Attachments: ZOOKEEPER-893.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> When ZooKeeper receives certain illegally formed messages on the internal 
> communication port (:4181 by default), it's possible for ZooKeeper to enter 
> an infinite loop which causes 100% cpu usage. It's related to ZOOKEEPER-427, 
> but that patch does not resolve all issues.
> from: src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java 
> the two affected parts:
> ===
> int length = msgLength.getInt();  
>   
> if(length <= 0) { 
>   
> throw new IOException("Invalid packet length:" + length); 
>   
> } 
> ===
> ===
> while (message.hasRemaining()) {  
>   
> temp_numbytes = channel.read(message);
>   
> if(temp_numbytes < 0) {   
>   
> throw new IOException("Channel eof before end");  
>   
> } 
>   
> numbytes += temp_numbytes;
>   
> } 
> ===
> how to replicate this bug:
> perform an nmap portscan against your zookeeper server: "nmap -sV -n 
> your.ip.here -p4181"
> wait for a while untill you see some messages in the logfile and then you 
> will see 100% cpu usage. It does not recover from this situation. With my 
> patch, it does not occur anymore

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Reopened: (ZOOKEEPER-804) c unit tests failing due to "assertion cptr failed"

2010-10-13 Thread Patrick Hunt (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt reopened ZOOKEEPER-804:



Reopening due to issue raised with the patch (see earlier comment).

Is this an issue that should be addressed or ok to resolve?

> c unit tests failing due to "assertion cptr failed"
> ---
>
> Key: ZOOKEEPER-804
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-804
> Project: Zookeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.0
> Environment: gcc 4.4.3, ubuntu lucid lynx, dual core laptop (intel)
>Reporter: Patrick Hunt
>Assignee: Michi Mutsuzaki
>Priority: Critical
> Fix For: 3.3.2, 3.4.0
>
> Attachments: ZOOKEEPER-804.patch
>
>
> I'm seeing this frequently:
>  [exec] Zookeeper_simpleSystem::testPing : elapsed 18006 : OK
>  [exec] Zookeeper_simpleSystem::testAcl : elapsed 1022 : OK
>  [exec] Zookeeper_simpleSystem::testChroot : elapsed 3145 : OK
>  [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started : 
> elapsed 25687 : OK
>  [exec] zktest-mt: 
> /home/phunt/dev/workspace/gitzk/src/c/src/zookeeper.c:1952: 
> zookeeper_process: Assertion `cptr' failed.
>  [exec] make: *** [run-check] Aborted
>  [exec] Zookeeper_simpleSystem::testHangingClient
> Mahadev can you take a look?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (ZOOKEEPER-893) ZooKeeper high cpu usage when invalid requests

2010-10-13 Thread Patrick Hunt (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-893:
---

Status: Patch Available  (was: Open)

> ZooKeeper high cpu usage when invalid requests
> --
>
> Key: ZOOKEEPER-893
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-893
> Project: Zookeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.3.1
> Environment: Linux 2.6.16
> 4x Intel(R) Xeon(R) CPU X3320  @ 2.50GHz
> java version "1.6.0_17"
> Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
> Java HotSpot(TM) Server VM (build 14.3-b01, mixed mode)
>Reporter: Thijs Terlouw
>Assignee: Thijs Terlouw
>Priority: Critical
> Fix For: 3.3.2, 3.4.0
>
> Attachments: ZOOKEEPER-893.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> When ZooKeeper receives certain illegally formed messages on the internal 
> communication port (:4181 by default), it's possible for ZooKeeper to enter 
> an infinite loop which causes 100% cpu usage. It's related to ZOOKEEPER-427, 
> but that patch does not resolve all issues.
> from: src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java 
> the two affected parts:
> ===
> int length = msgLength.getInt();  
>   
> if(length <= 0) { 
>   
> throw new IOException("Invalid packet length:" + length); 
>   
> } 
> ===
> ===
> while (message.hasRemaining()) {  
>   
> temp_numbytes = channel.read(message);
>   
> if(temp_numbytes < 0) {   
>   
> throw new IOException("Channel eof before end");  
>   
> } 
>   
> numbytes += temp_numbytes;
>   
> } 
> ===
> how to replicate this bug:
> perform an nmap portscan against your zookeeper server: "nmap -sV -n 
> your.ip.here -p4181"
> wait for a while untill you see some messages in the logfile and then you 
> will see 100% cpu usage. It does not recover from this situation. With my 
> patch, it does not occur anymore

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (ZOOKEEPER-891) Allow non-numeric version strings

2010-10-13 Thread Patrick Hunt (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Hunt updated ZOOKEEPER-891:
---

Fix Version/s: (was: 4.0.0)

I noticed a similar issue when I attempted to move us to the scheme hadoop core 
is using - x.y.z-dev as the default version. We should address this. Slating 
for 3.4.0

> Allow non-numeric version strings
> -
>
> Key: ZOOKEEPER-891
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-891
> Project: Zookeeper
>  Issue Type: Improvement
>  Components: build
>Reporter: Eli Collins
>Priority: Minor
> Fix For: 3.4.0
>
>
> Non-numeric version strings (eg -dev) or -are not currently accepted, you 
> either get an error (Invalid version number format, must be "x.y.z") or if 
> you pass x.y.z-dev or x.y.z+1 you'll get a NumberFormatException.  Would be 
> useful to allow non-numeric versions. 
> {noformat}
> version-info:
>  [java] All version-related parameters must be valid integers!
>  [java] Exception in thread "main" java.lang.NumberFormatException: For 
> input string: "3-dev"
>  [java]   at 
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
>  [java]   at java.lang.Integer.parseInt(Integer.java:458)
>  [java]   at java.lang.Integer.parseInt(Integer.java:499)
>  [java]   at 
> org.apache.zookeeper.version.util.VerGen.main(VerGen.java:131)
>  [java] Java Result: 1
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (ZOOKEEPER-890) C client invokes watcher callbacks multiple times

2010-10-13 Thread Patrick Hunt (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920619#action_12920619
 ] 

Patrick Hunt commented on ZOOKEEPER-890:


Should this issue be closed then? (also sounds like 2 other issues should be 
opened which were detailed in the comments - docs and python)

> C client invokes watcher callbacks multiple times
> -
>
> Key: ZOOKEEPER-890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-890
> Project: Zookeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.3.1
> Environment: Mac OS X 10.6.5
>Reporter: Austin Shoemaker
>Priority: Critical
> Attachments: watcher_twice.c, ZOOKEEPER-890.patch
>
>
> Code using the C client assumes that watcher callbacks are called exactly 
> once. If the watcher is called more than once, the process will likely 
> overwrite freed memory and/or crash.
> collect_session_watchers (zk_hashtable.c) gathers watchers from 
> active_node_watchers, active_exist_watchers, and active_child_watchers 
> without removing them. This results in watchers being invoked more than once.
> Test code is attached that reproduces the bug, along with a proposed patch.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (ZOOKEEPER-860) Add alternative search-provider to ZK site

2010-10-13 Thread Otis Gospodnetic (JIRA)

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920547#action_12920547
 ] 

Otis Gospodnetic commented on ZOOKEEPER-860:


For what it's worth, the following issues for the same functionality on other 
sub-projects have been committed so far: PIG-1661 HBASE-2886 HDFS-1367 


> Add alternative search-provider to ZK site
> --
>
> Key: ZOOKEEPER-860
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-860
> Project: Zookeeper
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Alex Baranau
>Assignee: Alex Baranau
>Priority: Minor
> Fix For: 3.4.0
>
> Attachments: ZOOKEEPER-860.patch
>
>
> Use search-hadoop.com service to make available search in ZK sources, MLs, 
> wiki, etc.
> This was initially proposed on user mailing list 
> (http://search-hadoop.com/m/sTZ4Y1BVKWg1). The search service was already 
> added in site's skin (common for all Hadoop related projects) before (as a 
> part of [AVRO-626|https://issues.apache.org/jira/browse/AVRO-626]) so this 
> issue is about enabling it for ZK. The ultimate goal is to use it at all 
> Hadoop's sub-projects' sites.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



What's the QA strategy of ZooKeeper?

2010-10-13 Thread Thomas Koch
Hi,

after filling 13 refactoring issues against the Java Client code[1], I started 
to dig into the server site code to understand the last issues with the Netty 
stuff.
I feel bad. It's this feeling of "I don't wanna hurt you, but...". ZooKeeper 
is quite an important piece of the Hadoop ecosystem containing some of the 
most complicated pieces of code. And it'll only get more complex with more 
features.

I'd propose to have a word about quality assurance. Is there already a 
strategy to ensure the ongoing maintainability of ZK? Is there a code style 
guide, a list of Dos-And-Donts (where I'd like to add some points)?

Should PMD be added to Hudson? What is the level of FindBugs? Should it be 
raised?

Some of the points I'd like to add to a style guide:

- Don't write methods longer then 20-40 lines of code

- Are you sure you want to use inner classes?

- If there is a new operator in a method? Could the method maybe already 
receive the object as a parameter?

- Are you sure you want to use system properties? They are like global 
variables and the IDE does not know about them

- Are you sure you want to extend a class? Often an aggregation is more 
elegant.

- Don't nest ifs and loops deeper then 2 or 3 levels. If you do so, you should 
better break your code into more methods.

- Use Enums or constants instead of plain status integers

- please document your intentions in code comments. You don't need to comment 
the what? but the why?.

Do you agree with me, that there is a need for better code quality in 
ZooKeeper? If so, it's not really scalable if a manic like me fights like Don 
Quichotte to clean up the code. All developers would need to establish a sense 
for clean code and constantly improve the code.

[1] https://issues.apache.org/jira/browse/ZOOKEEPER-835

Best regards,

Thomas Koch, http://www.koch.ro


[jira] Updated: (ZOOKEEPER-894) add Package o.a.Zookeeper.Client

2010-10-13 Thread Thomas Koch (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Koch updated ZOOKEEPER-894:
--

Description: 
I'd like to move classes that are not part of the API but belong to the ZK 
Client into a separate Client package. These classes are:

- Inner classes that should become normal classes:
  Zookeeper.ZkWatchManager
  Zookeeper.WatchRegistration
  ClientCnxn.SendThread (should become a Runnable anyhow)
  ClientCnxn.EventThread
  ClientCnxn.Package

- Classes now in the Zookeeper package:
  ClientCnxn -> Client.Cnxn
  ClientCnxnSocket* -> Client.CnxnSocket*
  ... Maybe some others that can be moved without breaking the API

- Classes yet to be written:
  PendingQueue (?)
  OutgoingQueue (?)

> add Package o.a.Zookeeper.Client
> 
>
> Key: ZOOKEEPER-894
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-894
> Project: Zookeeper
>  Issue Type: Sub-task
>Reporter: Thomas Koch
>
> I'd like to move classes that are not part of the API but belong to the ZK 
> Client into a separate Client package. These classes are:
> - Inner classes that should become normal classes:
>   Zookeeper.ZkWatchManager
>   Zookeeper.WatchRegistration
>   ClientCnxn.SendThread (should become a Runnable anyhow)
>   ClientCnxn.EventThread
>   ClientCnxn.Package
> - Classes now in the Zookeeper package:
>   ClientCnxn -> Client.Cnxn
>   ClientCnxnSocket* -> Client.CnxnSocket*
>   ... Maybe some others that can be moved without breaking the API
> - Classes yet to be written:
>   PendingQueue (?)
>   OutgoingQueue (?)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Created: (ZOOKEEPER-894) add Package o.a.Zookeeper.Client

2010-10-13 Thread Thomas Koch (JIRA)
add Package o.a.Zookeeper.Client


 Key: ZOOKEEPER-894
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-894
 Project: Zookeeper
  Issue Type: Sub-task
Reporter: Thomas Koch




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (ZOOKEEPER-823) update ZooKeeper java client to optionally use Netty for connections

2010-10-13 Thread Thomas Koch (JIRA)

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Koch updated ZOOKEEPER-823:
--

Attachment: testWatchAutoResetWithPending_FAILURE

I did 10 builds now in total with the last patch. 8 builds were succesful, 2 
failed with different test cases, see the *_FAILURE files. Both failures 
happened in the NettyNettySuiteTest.
It's hard to debug this kind of Heisenbugs that happen only very seldom. It 
would make things easier, if I could start cleaning up the code to better 
identify spots where things could go wrong.

> update ZooKeeper java client to optionally use Netty for connections
> 
>
> Key: ZOOKEEPER-823
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-823
> Project: Zookeeper
>  Issue Type: New Feature
>  Components: java client
>Reporter: Patrick Hunt
>Assignee: Patrick Hunt
> Fix For: 3.4.0
>
> Attachments: NettyNettySuiteTest.rtf, 
> TEST-org.apache.zookeeper.test.NettyNettySuiteTest.txt.gz, 
> testDisconnectedAddAuth_FAILURE, testWatchAutoResetWithPending_FAILURE, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, ZOOKEEPER-823.patch, 
> ZOOKEEPER-823.patch
>
>
> This jira will port the client side connection code to use netty rather than 
> direct nio.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.