Re: [VOTE] Release ZooKeeper 3.3.0 (candidate 0)

2010-03-27 Thread Stack
Thanks Patrick for the detail (Support of the quality below puts us
hbasistas at ease about our deciscion to bet the farm on zk making our
next major release).
St.Ack

On Thu, Mar 25, 2010 at 3:57 PM, Patrick Hunt ph...@apache.org wrote:
 Some background on this: In order to add new features (sometimes to fix
 bugs) we need to change the client lib in a non-b/w compatible way, this is
 infrequent, but there's just no way around this in some cases. At the server
 level we always ensure (and even in extreme cases this might not be
 possible, but so far it has) that a new server can talk to an old (at least
 1 version back) client. Additionally we also ensure that a new server can
 talk to an old server, this allows rolling upgrade of the ensemble. WRT
 this approach I'm probably not telling you anything you don't already know
 from your own/prior projects.

 In zk THIS IS CRITICAL to our primary project level goals of high
 availability. It would be laughable if we bill ourselves as highly
 available but sorry, you need to shut everything down then upgrade
 everything to a new version. That's just not acceptable.

 The roadmap has some detail on this, but it's out of date from our current
 practices. We also need to include this information in our release notes.
 http://wiki.apache.org/hadoop/ZooKeeper/Roadmap
 https://issues.apache.org/jira/browse/ZOOKEEPER-727

 Stack, in your specific case you are seeing that 3.3 client works fine with
 3.2 server. In 3.3 we added a new API method to the client, which sends a
 new message type to the server. As long as you don't use this method
 (getchildren2) it will probably work fine. However, we don't officially
 support this configuration as we don't design for this case (the changes)
 and we don't test for this. It may be that there was some semantic change at
 the protocol (client-server protocol) level, that may not be exposed except
 in unusual cases. Perhaps if we had more resources we could verify this case
 (3.3C with 3.2S) but today we do not, so essentially it would be use at
 your own risk.

 Hope this helps. If you have further insights, esp wrt HBase using ZK please
 feel free to comment.

 Regards,

 Patrick

 Stack wrote:

 Patrick just let me know that newer client talking to older server is
 not supported.  I didn't know that.  Thanks for pointing it out.  Was
 sort of surprised it worked at all so just noted this aspect of my zk
 3.3.0 RC0 eval.

 Congrats on new release lads,
 St.Ack

 P.S. Below is backup of my assertion 3.3.0 client basically works
 against 3.2.2 ensemble:


 In client log I see this:

 2010-03-25 12:23:57,670 INFO org.apache.zookeeper.ZooKeeper: Client
 environment:zookeeper.version=3.3.0-925362, built on 03/19/2010 18:38
 GMT

 ...

 Then this:

 2010-03-25 12:23:57,672 INFO org.apache.zookeeper.ZooKeeper:
 Initiating client connection,

 connectString=sv2borg165:2181,sv2borg166:2181,sv2borg167:2181,sv2borg169:2181,sv2borg164:2181
 sessionTimeout=6 watcher=Thread[Thread-0,5,main]
 2010-03-25 12:23:57,683 INFO org.apache.zookeeper.ClientCnxn: Opening
 socket connection to server sv2borg166/10.20.20.166:2181
 2010-03-25 12:23:57,686 INFO org.apache.zookeeper.ClientCnxn: Socket
 connection established to sv2borg166/10.20.20.166:2181, initiating
 session

 ..

 Over on 166 I see...

 2010-03-25 12:23:57,697 INFO
 org.apache.zookeeper.server.NIOServerCnxn: Connected to
 /10.20.20.185:46331 lastZxid 0
 2010-03-25 12:23:57,725 INFO
 org.apache.zookeeper.server.NIOServerCnxn: Creating new session
 0x3266d5140d70759
 2010-03-25 12:23:57,726 INFO
 org.apache.zookeeper.server.NIOServerCnxn: Finished init of
 0x3266d5140d70759 valid:true
 2010-03-25 12:25:07,305 INFO
 org.apache.zookeeper.server.NIOServerCnxn: Processing stat command
 from /10.20.20.185:60661
 2010-03-25 12:25:07,306 WARN
 org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of
 session 0x0 due to java.io.IOException: Responded to info probe
 2010-03-25 12:25:07,306 INFO
 org.apache.zookeeper.server.NIOServerCnxn: closing session:0x0
 NIOServerCnxn: java.nio.channels.SocketChannel[connected
 local=/10.20.20.166:2181 remote=/10.20.20.185:60661]

 ...

 If I do stat over there I see Zookeeper version: 3.2.2-888565, built
 on 12/08/2009 21:51 GMT...


 On Thu, Mar 25, 2010 at 2:00 PM, Patrick Hunt ph...@apache.org wrote:

 Stack, you can't use a new client with an old server. We support b/w
 compatibility at the server level (new server works with old client) but
 not
 the other way around. You would have to upgrade the server and client at
 the
 same time, or upgrade the servers (rolling upgrade) then upgrade the
 clients.

 Patrick

 Stack wrote:

 +1

 All hbase tests pass with 3.3.0 in place.  I ran small loading and
 nothing odd looking.  Looks like no issue having a zk 3.3.0 client
 talk to a 3.2.2 ensemble.

 Requires small mods to hbase other than dropping new zk jar into
 hbase/lib in place of zk 3.2.2: HBASE-2380.

 St.Ack

 On Fri, Mar 19, 2010 at 

Re: [VOTE] Release ZooKeeper 3.3.0 (candidate 0)

2010-03-25 Thread Patrick Hunt
With 5 +1's (3 from PMC members) and no -1's the vote passes. 
Congratulations all!


I will work to finish the release process and send announcement emails 
when it is completed.


Regards,

Patrick

Stack wrote:

+1

All hbase tests pass with 3.3.0 in place.  I ran small loading and
nothing odd looking.  Looks like no issue having a zk 3.3.0 client
talk to a 3.2.2 ensemble.

Requires small mods to hbase other than dropping new zk jar into
hbase/lib in place of zk 3.2.2: HBASE-2380.

St.Ack

On Fri, Mar 19, 2010 at 12:43 PM, Patrick Hunt ph...@apache.org wrote:

I have created a candidate build for ZooKeeper 3.3.0. Over 180 JIRAs are
addressed in this release.

*** Please download, test and VOTE before the
*** vote closes 1pm PT on Wednesday, March 24.***

http://people.apache.org/~phunt/zookeeper-3.3.0-candidate-0/

Should we release this?

Patrick








Re: [VOTE] Release ZooKeeper 3.3.0 (candidate 0)

2010-03-25 Thread Patrick Hunt
Some background on this: In order to add new features (sometimes to fix 
bugs) we need to change the client lib in a non-b/w compatible way, this 
is infrequent, but there's just no way around this in some cases. At the 
server level we always ensure (and even in extreme cases this might not 
be possible, but so far it has) that a new server can talk to an old (at 
least 1 version back) client. Additionally we also ensure that a new 
server can talk to an old server, this allows rolling upgrade of the 
ensemble. WRT this approach I'm probably not telling you anything you 
don't already know from your own/prior projects.


In zk THIS IS CRITICAL to our primary project level goals of high 
availability. It would be laughable if we bill ourselves as highly 
available but sorry, you need to shut everything down then upgrade 
everything to a new version. That's just not acceptable.


The roadmap has some detail on this, but it's out of date from our 
current practices. We also need to include this information in our 
release notes.

http://wiki.apache.org/hadoop/ZooKeeper/Roadmap
https://issues.apache.org/jira/browse/ZOOKEEPER-727

Stack, in your specific case you are seeing that 3.3 client works fine 
with 3.2 server. In 3.3 we added a new API method to the client, which 
sends a new message type to the server. As long as you don't use this 
method (getchildren2) it will probably work fine. However, we don't 
officially support this configuration as we don't design for this case 
(the changes) and we don't test for this. It may be that there was some 
semantic change at the protocol (client-server protocol) level, that 
may not be exposed except in unusual cases. Perhaps if we had more 
resources we could verify this case (3.3C with 3.2S) but today we do 
not, so essentially it would be use at your own risk.


Hope this helps. If you have further insights, esp wrt HBase using ZK 
please feel free to comment.


Regards,

Patrick

Stack wrote:

Patrick just let me know that newer client talking to older server is
not supported.  I didn't know that.  Thanks for pointing it out.  Was
sort of surprised it worked at all so just noted this aspect of my zk
3.3.0 RC0 eval.

Congrats on new release lads,
St.Ack

P.S. Below is backup of my assertion 3.3.0 client basically works
against 3.2.2 ensemble:


In client log I see this:

2010-03-25 12:23:57,670 INFO org.apache.zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.3.0-925362, built on 03/19/2010 18:38
GMT

...

Then this:

2010-03-25 12:23:57,672 INFO org.apache.zookeeper.ZooKeeper:
Initiating client connection,
connectString=sv2borg165:2181,sv2borg166:2181,sv2borg167:2181,sv2borg169:2181,sv2borg164:2181
sessionTimeout=6 watcher=Thread[Thread-0,5,main]
2010-03-25 12:23:57,683 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server sv2borg166/10.20.20.166:2181
2010-03-25 12:23:57,686 INFO org.apache.zookeeper.ClientCnxn: Socket
connection established to sv2borg166/10.20.20.166:2181, initiating
session

..

Over on 166 I see...

2010-03-25 12:23:57,697 INFO
org.apache.zookeeper.server.NIOServerCnxn: Connected to
/10.20.20.185:46331 lastZxid 0
2010-03-25 12:23:57,725 INFO
org.apache.zookeeper.server.NIOServerCnxn: Creating new session
0x3266d5140d70759
2010-03-25 12:23:57,726 INFO
org.apache.zookeeper.server.NIOServerCnxn: Finished init of
0x3266d5140d70759 valid:true
2010-03-25 12:25:07,305 INFO
org.apache.zookeeper.server.NIOServerCnxn: Processing stat command
from /10.20.20.185:60661
2010-03-25 12:25:07,306 WARN
org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of
session 0x0 due to java.io.IOException: Responded to info probe
2010-03-25 12:25:07,306 INFO
org.apache.zookeeper.server.NIOServerCnxn: closing session:0x0
NIOServerCnxn: java.nio.channels.SocketChannel[connected
local=/10.20.20.166:2181 remote=/10.20.20.185:60661]

...

If I do stat over there I see Zookeeper version: 3.2.2-888565, built
on 12/08/2009 21:51 GMT...


On Thu, Mar 25, 2010 at 2:00 PM, Patrick Hunt ph...@apache.org wrote:

Stack, you can't use a new client with an old server. We support b/w
compatibility at the server level (new server works with old client) but not
the other way around. You would have to upgrade the server and client at the
same time, or upgrade the servers (rolling upgrade) then upgrade the
clients.

Patrick

Stack wrote:

+1

All hbase tests pass with 3.3.0 in place.  I ran small loading and
nothing odd looking.  Looks like no issue having a zk 3.3.0 client
talk to a 3.2.2 ensemble.

Requires small mods to hbase other than dropping new zk jar into
hbase/lib in place of zk 3.2.2: HBASE-2380.

St.Ack

On Fri, Mar 19, 2010 at 12:43 PM, Patrick Hunt ph...@apache.org wrote:

I have created a candidate build for ZooKeeper 3.3.0. Over 180 JIRAs are
addressed in this release.

*** Please download, test and VOTE before the
*** vote closes 1pm PT on Wednesday, March 24.***

http://people.apache.org/~phunt/zookeeper-3.3.0-candidate-0/


Re: [VOTE] Release ZooKeeper 3.3.0 (candidate 0)

2010-03-24 Thread Flavio Junqueira
+1, ran unit tests plus some manual tests, including some tests  
involving observers.


On Mar 23, 2010, at 5:56 PM, Benjamin Reed wrote:


+1 ran some load tests and everything looks good. no measurable change
in performance from last release.

On 03/19/2010 12:43 PM, Patrick Hunt wrote:
I have created a candidate build for ZooKeeper 3.3.0. Over 180  
JIRAs are

addressed in this release.

*** Please download, test and VOTE before the
*** vote closes 1pm PT on Wednesday, March 24.***

http://people.apache.org/~phunt/zookeeper-3.3.0-candidate-0/

Should we release this?

Patrick












Re: [VOTE] Release ZooKeeper 3.3.0 (candidate 0)

2010-03-24 Thread Patrick Hunt

+1

In addition to the std stuff (unit tests etc...) I've run a number of 
deployment tests, including clusters of size 1/3/5/6/7/9/11/12/13 and 
they are all working fine.


Patrick

Patrick Hunt wrote:
I have created a candidate build for ZooKeeper 3.3.0. Over 180 JIRAs are 
addressed in this release.


*** Please download, test and VOTE before the
*** vote closes 1pm PT on Wednesday, March 24.***

http://people.apache.org/~phunt/zookeeper-3.3.0-candidate-0/

Should we release this?

Patrick







Re: [VOTE] Release ZooKeeper 3.3.0 (candidate 0)

2010-03-24 Thread Mahadev Konar
+1  

Did some unit testing on different machine configs. Also, did some smoke
testing with a cluster size of 3, running some updates against it and
bringing one machine down at a time.

Seems to work!

Thanks
mahadev


On 3/24/10 9:36 AM, Patrick Hunt ph...@apache.org wrote:

 +1
 
 In addition to the std stuff (unit tests etc...) I've run a number of
 deployment tests, including clusters of size 1/3/5/6/7/9/11/12/13 and
 they are all working fine.
 
 Patrick
 
 Patrick Hunt wrote:
 I have created a candidate build for ZooKeeper 3.3.0. Over 180 JIRAs are
 addressed in this release.
 
 *** Please download, test and VOTE before the
 *** vote closes 1pm PT on Wednesday, March 24.***
 
 http://people.apache.org/~phunt/zookeeper-3.3.0-candidate-0/
 
 Should we release this?
 
 Patrick
 
 
 
 
 



Re: [VOTE] Release ZooKeeper 3.3.0 (candidate 0)

2010-03-23 Thread Benjamin Reed
+1 ran some load tests and everything looks good. no measurable change 
in performance from last release.


On 03/19/2010 12:43 PM, Patrick Hunt wrote:

I have created a candidate build for ZooKeeper 3.3.0. Over 180 JIRAs are
addressed in this release.

*** Please download, test and VOTE before the
*** vote closes 1pm PT on Wednesday, March 24.***

http://people.apache.org/~phunt/zookeeper-3.3.0-candidate-0/

Should we release this?

Patrick