Re: Errors with Python bindings

2010-07-16 Thread Patrick Hunt
Hi Rich, the version string looks useful to have, thanks! Would you mind 
submitting this via jira? Do a svn diff (looks like you did already), 
create a jira and attach the diff, then click submit link on the jira. 
We'll review and work on getting it into a future release.

http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute

Thanks!

Patrick

On 07/15/2010 05:24 PM, Rich Schumacher wrote:

Hey Henry,

Good to know!  I was under the impression the 3.3.0 release had the updated 
bindings but it seems I was mistaken.  I'll get those built and then see what 
happens.  Just curious, have you ever run into or heard of this?  A quick 
Google search didn't return anything interesting.

As for the version in the Python bindings, how about this trivial patch:

Index: src/c/zookeeper.c
===
--- src/c/zookeeper.c   (revision 964617)
+++ src/c/zookeeper.c   (working copy)
@@ -1510,6 +1510,11 @@
PyModule_AddObject(module, ZooKeeperException, ZooKeeperException);
Py_INCREF(ZooKeeperException);

+  char version_str[];
+  sprintf(version_str, %i.%i.%i, ZOO_MAJOR_VERSION, ZOO_MINOR_VERSION, 
ZOO_PATCH_VERSION);
+
+  PyModule_AddStringConstant(module, __version__, version_str);
+
ADD_INTCONSTANT(PERM_READ);
ADD_INTCONSTANT(PERM_WRITE);
ADD_INTCONSTANT(PERM_CREATE);


On Jul 14, 2010, at 2:57 PM, Henry Robinson wrote:


Hi Rich -

No, there's not a very easy way to verify the Python bindings version afaik
- would be a useful feature to have though.

My first suggestion is to move to the bindings shipped with 3.3.1 - we fixed
a lot of problems with the Python bindings which improved their stability a
lot. Could you try that and then let us know if you continue to see
problems?

cheers,
Henry

On 14 July 2010 13:14, Rich Schumacherrich.s...@gmail.com  wrote:


I'm running a Tornado webserver and using ZooKeeper to store some metadata
and occasionally the ZooKeeper connection will error out irrevocably.  Any
subsequent calls to ZooKeeper from this process will result in a
SystemError.

Here is the relevant portion of the Python traceback:
snip...
File /usr/lib/pymodules/python2.5/zuul/storage/zoo.py, line 69, in call
   return getattr(zookeeper, name)(self.handle, *args)
SystemError: NULL result without error in PyObject_Call

I found this in the ZooKeeper server logs:

2010-07-13 06:52:46,488 - INFO  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:nioservercnxn$fact...@251] - Accepted socket
connection from /10.2.128.233:54779
2010-07-13 06:52:46,489 - INFO  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:nioserverc...@742] - Client attempting to renew
session 0x429b865a6270003 at /10.2.128.233:54779
2010-07-13 06:52:46,489 - INFO  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:lear...@95] - Revalidating client: 299973596915630083
2010-07-13 06:52:46,793 - INFO
[QuorumPeer:/0:0:0:0:0:0:0:0:2181:nioserverc...@1424] - Invalid session
0x429b865a6270003 for client /10.2.128.233:54779, probably expired
2010-07-13 06:52:46,794 - INFO  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:nioserverc...@1286] - Closed socket connection for
client /10.2.128.233:54779 which had sessionid 0x429b865a6270003


The ZooKeeper ensemble is healthy; each node responds as expected to the
four letter word commands and a simple restart of the Tornado processes
fixes this.

My question is, if this really is due to session expiration why is a
SessionExpiredException not raised?  Another question, is there an easy way
to determine the version of the ZooKeeper Python bindings I'm using?  I
built the 3.3.0 bindings but I just want to be able to verify that.

Thanks for the help,

Rich





--
Henry Robinson
Software Engineer
Cloudera
415-994-6679




cleanup ZK takes 40-60 seconds

2010-07-16 Thread Vishal K
Hi,

We have embedded ZK server in our application. We start a thread in our
application and call QuorumPeerMain.InitializeArgs().

When cleaning-up ZK we call QuorumPeerMain.shutdown() and wait for the
thread that is calling InitializeArgs() to finish. These two steps are
taking around 60 seconds. I could probably not wait for InitializeArgs() to
finish and that might speed up things.

However, I am not sure why the cleanup should take such a long time. Can
anyone comment on this?

Thanks.
-Vishal


Re: cleanup ZK takes 40-60 seconds

2010-07-16 Thread Ted Dunning
I can't comment on the cleanup time, but I can suggest that it is normally
not a very good idea to embed Zookeeper in your application.  If your
application really is distributed, then having ZK survive the demise of any
particular instance is a really nice thing.  If ZK goes away with your
application then you lose a lot of the power of having a reliable and
independent coordination service.

I have, as they say, been there and done that.  It was not a happy
experience.

You know what you are doing much more than I possibly could, so embedding ZK
might actually make sense.  I really don't think so, though.

On Fri, Jul 16, 2010 at 6:28 PM, Vishal K vishalm...@gmail.com wrote:

 However, I am not sure why the cleanup should take such a long time. Can
 anyone comment on this?



RE: cleanup ZK takes 40-60 seconds

2010-07-16 Thread Benjamin Reed
how big is your database? it would be good to know the timing of the two calls. 
shutdown should take very little time.

sent from my droid

-Original Message-
From: Vishal K [vishalm...@gmail.com]
Received: 7/16/10 6:31 PM
To: zookeeper-user@hadoop.apache.org [zookeeper-u...@hadoop.apache.org]
Subject: cleanup ZK takes 40-60 seconds

Hi,

We have embedded ZK server in our application. We start a thread in our
application and call QuorumPeerMain.InitializeArgs().

When cleaning-up ZK we call QuorumPeerMain.shutdown() and wait for the
thread that is calling InitializeArgs() to finish. These two steps are
taking around 60 seconds. I could probably not wait for InitializeArgs() to
finish and that might speed up things.

However, I am not sure why the cleanup should take such a long time. Can
anyone comment on this?

Thanks.
-Vishal