Re: Error running contrib tests
Hey! I have been working with the zkpython module for that last couple of weeks. After the initial problem with ImportError: libzookeeper_mt.so.2: cannot open shared object file: No such file or directory I just added LD_LIBRARY_PATH=/usr/local/lib to the .bashrc file and everything worked fine. For pydev I had to add it as an Environment variable, to make it work. But when moving away from raw python to to using it in a fcgi context I was not able to get it to work under any circumstances. We were running this using python 2.5, which seemed to be one of the problems, the other one that I've found was the build script in zookeeper_home/src/contrib/ zkpython. when running sudo ant install everyhing works fine but when trying to import it in python 2.6 I still got the same error. Changing the setup.py file to point to the correct libraries and running sudo python setup.py install everything works fine and no more errors. Just wanted to share this with the community since I've spend quite some time trying to ge this to work. Regards Erik
Re: Error running contrib tests
What version of ZooKeeper is this? If it's 3.2.1 take a look at this: binding fixes in 3.2.2 https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=truepid=12310801component=12312860fixfor=12314335resolution=1sorter/field=prioritysorter/order=DESC You might give the 3.2 branch (in svn) a try asap. Feel free to enter JIRAs if you notice issues, we'll be happy to address them. Patrick Erik Holstad wrote: Hey! I have been working with the zkpython module for that last couple of weeks. After the initial problem with ImportError: libzookeeper_mt.so.2: cannot open shared object file: No such file or directory I just added LD_LIBRARY_PATH=/usr/local/lib to the .bashrc file and everything worked fine. For pydev I had to add it as an Environment variable, to make it work. But when moving away from raw python to to using it in a fcgi context I was not able to get it to work under any circumstances. We were running this using python 2.5, which seemed to be one of the problems, the other one that I've found was the build script in zookeeper_home/src/contrib/ zkpython. when running sudo ant install everyhing works fine but when trying to import it in python 2.6 I still got the same error. Changing the setup.py file to point to the correct libraries and running sudo python setup.py install everything works fine and no more errors. Just wanted to share this with the community since I've spend quite some time trying to ge this to work. Regards Erik
c client on win32
Hi, Has anybody managed to get the c client / dll compiled on Win32, and if so, how? I did a quick pass with MunGW and Cygwin, and they failed horribly. I'd like to load the DLL to bind it to a scripting language on windows as well. Thanks, Maarten
Re: c client on win32
Maarten, I just tried this with cygwin and it fails for me too. It seems that cygwin does not support getaddrinfo! Please create a JIRA and I'll see what we can do. Patrick Maarten Koopmans wrote: Hi, Has anybody managed to get the c client / dll compiled on Win32, and if so, how? I did a quick pass with MunGW and Cygwin, and they failed horribly. I'd like to load the DLL to bind it to a scripting language on windows as well. Thanks, Maarten
Re: c client on win32
Well I can tell you that the C API is the more heavily used API of the two (c/java) inside Yahoo, it's also the basis of the python and perl bindings. The issue here is that no one seems to have tried it on windows in quite some time. I believe Cygwin 1.7 (currently in beta) does have getaddrinfo support btw (I have not tried that). MinGW sounds like a good goal, if you'd like to create a jira and provide some patches we'd be happy to work with you to achieve. Regards, Patrick Maarten Koopmans wrote: Patrick, I'll stick to the Java API - the C API feels to much of a second class citizen. Besides, I think we might wnat to try with the beta of Cygwin first before filing it in Jira. Ultimately the goal (IMHO) should be MinGW support(?) --Maarten Patrick Hunt schreef: Maarten, I just tried this with cygwin and it fails for me too. It seems that cygwin does not support getaddrinfo! Please create a JIRA and I'll see what we can do. Patrick Maarten Koopmans wrote: Hi, Has anybody managed to get the c client / dll compiled on Win32, and if so, how? I did a quick pass with MunGW and Cygwin, and they failed horribly. I'd like to load the DLL to bind it to a scripting language on windows as well. Thanks, Maarten
Re: c client on win32
Good to know. I'll check on Cygwin 1.7 and switch to Linux otherwise. I want to do a quick REBOL binding (so I need to figure out the C calls that are minimal, and preferably have no callback function pointers). If I can get ZooKeeper talking via C (or via TCP, but the protocol doesn't seem to be specified) from REBOL, I have some very cool things coming. But maybe we should talk off-list aboutt that (will be open sourced though). --Maarten Patrick Hunt schreef: Well I can tell you that the C API is the more heavily used API of the two (c/java) inside Yahoo, it's also the basis of the python and perl bindings. The issue here is that no one seems to have tried it on windows in quite some time. I believe Cygwin 1.7 (currently in beta) does have getaddrinfo support btw (I have not tried that). MinGW sounds like a good goal, if you'd like to create a jira and provide some patches we'd be happy to work with you to achieve. Regards, Patrick Maarten Koopmans wrote: Patrick, I'll stick to the Java API - the C API feels to much of a second class citizen. Besides, I think we might wnat to try with the beta of Cygwin first before filing it in Jira. Ultimately the goal (IMHO) should be MinGW support(?) --Maarten Patrick Hunt schreef: Maarten, I just tried this with cygwin and it fails for me too. It seems that cygwin does not support getaddrinfo! Please create a JIRA and I'll see what we can do. Patrick Maarten Koopmans wrote: Hi, Has anybody managed to get the c client / dll compiled on Win32, and if so, how? I did a quick pass with MunGW and Cygwin, and they failed horribly. I'd like to load the DLL to bind it to a scripting language on windows as well. Thanks, Maarten
Re: c client on win32
Feel free to submit for inclusion in ZK's src/contrib. Good Luck, Patrick Maarten Koopmans wrote: Good to know. I'll check on Cygwin 1.7 and switch to Linux otherwise. I want to do a quick REBOL binding (so I need to figure out the C calls that are minimal, and preferably have no callback function pointers). If I can get ZooKeeper talking via C (or via TCP, but the protocol doesn't seem to be specified) from REBOL, I have some very cool things coming. But maybe we should talk off-list aboutt that (will be open sourced though). --Maarten Patrick Hunt schreef: Well I can tell you that the C API is the more heavily used API of the two (c/java) inside Yahoo, it's also the basis of the python and perl bindings. The issue here is that no one seems to have tried it on windows in quite some time. I believe Cygwin 1.7 (currently in beta) does have getaddrinfo support btw (I have not tried that). MinGW sounds like a good goal, if you'd like to create a jira and provide some patches we'd be happy to work with you to achieve. Regards, Patrick Maarten Koopmans wrote: Patrick, I'll stick to the Java API - the C API feels to much of a second class citizen. Besides, I think we might wnat to try with the beta of Cygwin first before filing it in Jira. Ultimately the goal (IMHO) should be MinGW support(?) --Maarten Patrick Hunt schreef: Maarten, I just tried this with cygwin and it fails for me too. It seems that cygwin does not support getaddrinfo! Please create a JIRA and I'll see what we can do. Patrick Maarten Koopmans wrote: Hi, Has anybody managed to get the c client / dll compiled on Win32, and if so, how? I did a quick pass with MunGW and Cygwin, and they failed horribly. I'd like to load the DLL to bind it to a scripting language on windows as well. Thanks, Maarten
Please disregard - Re: Exception on close of connection (WAS - Re: c client on win32)
Please disregard. Sorry for the noise (Patrick, of note, I am seeing this session timeout on a cluster other than Zhenyus). St.Ack On Fri, Nov 20, 2009 at 4:24 PM, stack st...@duboce.net wrote: Sorry, I had a bad subject on the below question. St.Ack On Fri, Nov 20, 2009 at 4:22 PM, stack st...@duboce.net wrote: Below an excerpt from a single node zk quorum that was at heart of a small hbase cluster. Unfortunately the log is not at DEBUG level (I've asked the gentleman to up the log level meantime). What it seems to be reporting is that an exception while closing a session caused it to timeout all connected sessions. Here is the line that mentions the exception on close of session. There is no stack trace: 2009-11-20 03:41:04,766 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x124bc250d700790 due to java.io.IOException: Read error Is it correct that an error at this stage throws out all connected sessions? Thanks, St.Ack 2009-11-20 00:00:04,948 INFO org.apache.zookeeper.server.NIOServerCnxn: Connected to /10.1.20.101:50716 lastZxid 0 2009-11-20 00:00:04,982 INFO org.apache.zookeeper.server.NIOServerCnxn: Creating new session 0x1250f26319f0016 2009-11-20 00:00:05,051 INFO org.apache.zookeeper.server.NIOServerCnxn: Finished init of 0x1250f26319f0016 valid:true 2009-11-20 00:00:05,051 WARN org.apache.zookeeper.server.PrepRequestProcessor: Got exception when processing sessionid:0x1250f26319f0016 type:create c xid:0x1 zxid:0xfffe txntype:unknown n/a org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists at org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114) 2009-11-20 00:00:40,150 WARN org.apache.zookeeper.server.PrepRequestProcessor: Got exception when processing sessionid:0x1250f26319f0016 type:create c xid:0x4 zxid:0xfffe txntype:unknown n/a org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists at org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114) 2009-11-20 00:00:50,428 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x1250f26319f0016 due to java.io.IOExceptio n: Read error 2009-11-20 00:00:50,429 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x1250f26319f0016 NIOServerCnxn: java.nio.channels.SocketChann el[connected local=/10.1.20.101:2181 remote=/10.1.20.101:50716] 2009-11-20 00:01:22,002 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x1250f26319f0016 2009-11-20 00:01:22,002 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x1250f26319f0016 2009-11-20 00:01:22,002 INFO org.apache.zookeeper.server.PrepRequestProcessor: Processed session termination request for id: 0x1250f26319f0016 2009-11-20 03:41:04,766 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x124bc250d700790 due to java.io.IOExceptio n: Read error 2009-11-20 03:41:04,864 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x1250f26319f 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x1250f26319f 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x124bc250d7007a2 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x124bc250d7007a2 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x124bc250d700794 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x124bc250d700794
Re: Please disregard - Re: Exception on close of connection (WAS - Re: c client on win32)
Oops too late. ;-) I'm perplexed as to why you see all these expirations though. Are you killing your clients, ie not cleaning up the ZK session gracefully via close()? Patrick stack wrote: Please disregard. Sorry for the noise (Patrick, of note, I am seeing this session timeout on a cluster other than Zhenyus). St.Ack On Fri, Nov 20, 2009 at 4:24 PM, stack st...@duboce.net wrote: Sorry, I had a bad subject on the below question. St.Ack On Fri, Nov 20, 2009 at 4:22 PM, stack st...@duboce.net wrote: Below an excerpt from a single node zk quorum that was at heart of a small hbase cluster. Unfortunately the log is not at DEBUG level (I've asked the gentleman to up the log level meantime). What it seems to be reporting is that an exception while closing a session caused it to timeout all connected sessions. Here is the line that mentions the exception on close of session. There is no stack trace: 2009-11-20 03:41:04,766 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x124bc250d700790 due to java.io.IOException: Read error Is it correct that an error at this stage throws out all connected sessions? Thanks, St.Ack 2009-11-20 00:00:04,948 INFO org.apache.zookeeper.server.NIOServerCnxn: Connected to /10.1.20.101:50716 lastZxid 0 2009-11-20 00:00:04,982 INFO org.apache.zookeeper.server.NIOServerCnxn: Creating new session 0x1250f26319f0016 2009-11-20 00:00:05,051 INFO org.apache.zookeeper.server.NIOServerCnxn: Finished init of 0x1250f26319f0016 valid:true 2009-11-20 00:00:05,051 WARN org.apache.zookeeper.server.PrepRequestProcessor: Got exception when processing sessionid:0x1250f26319f0016 type:create c xid:0x1 zxid:0xfffe txntype:unknown n/a org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists at org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114) 2009-11-20 00:00:40,150 WARN org.apache.zookeeper.server.PrepRequestProcessor: Got exception when processing sessionid:0x1250f26319f0016 type:create c xid:0x4 zxid:0xfffe txntype:unknown n/a org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists at org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114) 2009-11-20 00:00:50,428 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x1250f26319f0016 due to java.io.IOExceptio n: Read error 2009-11-20 00:00:50,429 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x1250f26319f0016 NIOServerCnxn: java.nio.channels.SocketChann el[connected local=/10.1.20.101:2181 remote=/10.1.20.101:50716] 2009-11-20 00:01:22,002 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x1250f26319f0016 2009-11-20 00:01:22,002 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x1250f26319f0016 2009-11-20 00:01:22,002 INFO org.apache.zookeeper.server.PrepRequestProcessor: Processed session termination request for id: 0x1250f26319f0016 2009-11-20 03:41:04,766 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x124bc250d700790 due to java.io.IOExceptio n: Read error 2009-11-20 03:41:04,864 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x1250f26319f 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x1250f26319f 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x124bc250d7007a2 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x124bc250d7007a2 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x124bc250d700794 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x124bc250d700794
Re: Please disregard - Re: Exception on close of connection (WAS - Re: c client on win32)
I think now I can explain the session expirations; hbase cilents especially up in a map/reduce task can exit without closing the zk session. Will fix. St.Ack On Fri, Nov 20, 2009 at 4:45 PM, Patrick Hunt ph...@apache.org wrote: Yes, right, that's what I meant to say - what is causing the client to die, throwing read error on the server side, and then later you end up with the session expiration because the client was not closed gracefully. (thanks mahadev) Patrick Mahadev Konar wrote: That should be the case since the server gets an exception reading from the socket - meaning the client went away (not gracefully) and that leads the server to expire the session in 30 seconds. mahadev On 11/20/09 4:35 PM, Patrick Hunt ph...@apache.org wrote: Oops too late. ;-) I'm perplexed as to why you see all these expirations though. Are you killing your clients, ie not cleaning up the ZK session gracefully via close()? Patrick stack wrote: Please disregard. Sorry for the noise (Patrick, of note, I am seeing this session timeout on a cluster other than Zhenyus). St.Ack On Fri, Nov 20, 2009 at 4:24 PM, stack st...@duboce.net wrote: Sorry, I had a bad subject on the below question. St.Ack On Fri, Nov 20, 2009 at 4:22 PM, stack st...@duboce.net wrote: Below an excerpt from a single node zk quorum that was at heart of a small hbase cluster. Unfortunately the log is not at DEBUG level (I've asked the gentleman to up the log level meantime). What it seems to be reporting is that an exception while closing a session caused it to timeout all connected sessions. Here is the line that mentions the exception on close of session. There is no stack trace: 2009-11-20 03:41:04,766 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x124bc250d700790 due to java.io.IOException: Read error Is it correct that an error at this stage throws out all connected sessions? Thanks, St.Ack 2009-11-20 00:00:04,948 INFO org.apache.zookeeper.server.NIOServerCnxn: Connected to /10.1.20.101:50716 lastZxid 0 2009-11-20 00:00:04,982 INFO org.apache.zookeeper.server.NIOServerCnxn: Creating new session 0x1250f26319f0016 2009-11-20 00:00:05,051 INFO org.apache.zookeeper.server.NIOServerCnxn: Finished init of 0x1250f26319f0016 valid:true 2009-11-20 00:00:05,051 WARN org.apache.zookeeper.server.PrepRequestProcessor: Got exception when processing sessionid:0x1250f26319f0016 type:create c xid:0x1 zxid:0xfffe txntype:unknown n/a org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists at org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces sor.java:245) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j ava:114) 2009-11-20 00:00:40,150 WARN org.apache.zookeeper.server.PrepRequestProcessor: Got exception when processing sessionid:0x1250f26319f0016 type:create c xid:0x4 zxid:0xfffe txntype:unknown n/a org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists at org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces sor.java:245) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j ava:114) 2009-11-20 00:00:50,428 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x1250f26319f0016 due to java.io.IOExceptio n: Read error 2009-11-20 00:00:50,429 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x1250f26319f0016 NIOServerCnxn: java.nio.channels.SocketChann el[connected local=/10.1.20.101:2181 remote=/10.1.20.101:50716] 2009-11-20 00:01:22,002 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x1250f26319f0016 2009-11-20 00:01:22,002 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x1250f26319f0016 2009-11-20 00:01:22,002 INFO org.apache.zookeeper.server.PrepRequestProcessor: Processed session termination request for id: 0x1250f26319f0016 2009-11-20 03:41:04,766 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x124bc250d700790 due to java.io.IOExceptio n: Read error 2009-11-20 03:41:04,864 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x1250f26319f 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x1250f26319f 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x124bc250d7007a2 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x124bc250d7007a2 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.SessionTrackerImpl: Expiring session 0x124bc250d700794 2009-11-20 03:41:04,927 INFO org.apache.zookeeper.server.ZooKeeperServer: Expiring session 0x124bc250d700794