FYI - Menagerie - ZooKeeper-based Java Concurrency Libraries

2011-01-06 Thread Patrick Hunt
This looks very cool: https://github.com/openUtility/menagerie Patrick

Re: killing zookeeper when used with CI with version 3.2.0+

2011-01-11 Thread Patrick Hunt
I believe jira 346 is saying that JMX could be used, not that it currently has that capability. I took a look at the current beans and I can't find a current method that implements shutdown (kill). As of today the best way to kill the server is as Mahadev suggests. Patrick On Tue, Jan 11, 2011

Re: ZK Client won't time out when quorum irrevocably goes away

2011-02-03 Thread Patrick Hunt
this assertion before and it always irked me a bit, but Ryan's scenario is a great thought experiment (well, though experiment for US, not for him).  Why can't those clients decide the session is expired after 3 days when the timeout is 3 minutes? On Thu, Feb 3, 2011 at 4:01 PM, Patrick Hunt ph

Re: tickTime and sessionTimeout

2011-02-11 Thread Patrick Hunt
Those are pretty short timeouts, many sources of delay could be causing this. Network jitter/latency, GC/swap (server or client), IO write latency, etc... See if any of this might be your issue: https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting I can clearly see this sequence

Re: garbage collection of ZooKeeper class

2011-02-14 Thread Patrick Hunt
Ok, great! Patrick On Mon, Feb 14, 2011 at 1:23 PM, Johannes Zillmann jzillm...@googlemail.com wrote: Hi Patrick, On Feb 14, 2011, at 6:36 PM, Patrick Hunt wrote: What version are you using? Perhaps you are hitting this one? https://issues.apache.org/jira/browse/ZOOKEEPER-795 Looks like

Re: Zookeeper for generating sequential IDs

2011-02-25 Thread Patrick Hunt
Have you looked at snowflake? http://engineering.twitter.com/2010/06/announcing-snowflake.html Patrick On Fri, Feb 25, 2011 at 9:43 AM, Ted Dunning ted.dunn...@gmail.com wrote: If your id's don't need to be exactly sequential or if the generation rate is less than a few thousand per second,

Re: Question about the Barrier Java example on the ZooKeeper documentation

2011-03-08 Thread Patrick Hunt
On Tue, Mar 8, 2011 at 5:59 AM, Flavio Junqueira f...@yahoo-inc.com wrote: I believe the goal of the examples was never to be a complete solutions to barriers or queues, but just to give a quick bootstrap to beginners. It is true, though, that the documentation page does not make that claim,

Re: Identify ephemeral node owner

2011-03-29 Thread Patrick Hunt
Hi Travis, as Mahadev mentioned, when a client establishes a session with a server the host id is logged to the server log: 2011-03-29 14:41:59,741 - INFO [SyncThread:0:NIOServerCnxn@1580] - Established session 0x12f0390e752 with negotiated timeout 3 for client /127.0.0.1:47789 You

Re: Hung up zookeeper client close?

2011-04-04 Thread Patrick Hunt
What version of ZK is being used? See: https://issues.apache.org/jira/browse/ZOOKEEPER-846 Patrick On Mon, Apr 4, 2011 at 10:48 AM, Stack st...@duboce.net wrote: Have you lot ever seen a hang on client close?  A user over in hbaselandia is seeing such a thing reportedly (See below -- I'm

Re: Serious problem processing hearbeat on login stampede

2011-04-14 Thread Patrick Hunt
cluster failover time significantly. Thank you, Patrick. ps. We actually push ping request to FinalRequestProcessor as soon as the packet identifies itself as ping. No dice. 2011. 4. 14., 오전 12:21, Patrick Hunt 작성: Hi Chang, it sounds like you may have an issue with your cluster

Re: observers in occasionally disconnected data centers

2011-05-04 Thread Patrick Hunt
patch is now attached to the issue: https://issues.apache.org/jira/browse/ZOOKEEPER-784 I do *not* make any claim to have understood the contents of this patch; all I did was synch everything and fix the obvious log4j/slf4j change.  Now what? On Tue, May 3, 2011 at 5:46 PM, Patrick Hunt ph

Re: Announcement - ZK IDGenerator code released to open source (finally!) :-)

2011-05-19 Thread Patrick Hunt
Hi David, great to see you making this available to the community! Regards. Patrick On Thu, May 19, 2011 at 11:08 AM, David Rosenstrauch dar...@darose.net wrote: On 05/19/2011 01:56 PM, Ted Dunning wrote: Nice. Some nits: a) why sourceforge and not github?  SF makes it harder to browse

Re: Timing issue when running embedded?

2011-05-19 Thread Patrick Hunt
Hi Gunnar this is great detective work. It certainly sounds like it might be some timing issue or possible bug in ZK exposed by this embedded case. A few questions: 1) in this dev/embedded case you only have a single zk server, correct? 2) you have 2 clients in this case, one creating the znode

Re: Zookeeper Disconnects repeatedly and often is unable to reconnect which expires session

2011-05-25 Thread Patrick Hunt
Excellent suggestions from Ted, you should also take a look at the troubleshooting guide on the wiki: https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting On Wed, May 25, 2011 at 10:17 AM, Ted Dunning ted.dunn...@gmail.com wrote: That is a fairly short timeout. Generally, the

Welcome Ted Dunning to the ZooKeeper PMC!

2011-05-26 Thread Patrick Hunt
Hi Folks, I'm happy to report that the PMC has voted and Ted has happily accepted to become a ZooKeeper PMC member! Ted, welcome aboard! Please feel free to mention a little bit about yourself, and congrats! Patrick

Re: Zookeeper server as OSGI bundle/feature

2011-05-31 Thread Patrick Hunt
Hi Oliver, it should definitely be possible. See https://issues.apache.org/jira/browse/ZOOKEEPER-425 also see the build.xml file or the generated zookeeper jar file - it includes OSGi specific detail. Regards, Patrick On Tue, May 31, 2011 at 8:18 AM, Oliver Wulff owna...@gmail.com wrote: Hi

Re: Debian packager orphaning ZooKeeper

2011-06-10 Thread Patrick Hunt
On Fri, Jun 10, 2011 at 12:58 PM, Gustavo Niemeyer gust...@niemeyer.net wrote: Hi Patrick, Hi Gustavo. Definitely some valid points in there, obv not everyone agrees with all of them. ;-) Thomas also sent some details earlier on the dev list, which you can find here:

Re: Debian packager orphaning ZooKeeper

2011-06-10 Thread Patrick Hunt
On Fri, Jun 10, 2011 at 2:18 PM, Gustavo Niemeyer gust...@niemeyer.net wrote: I had a long talk with Thomas at Buzzwords. He is far more congenial in person than on the mailing lists. Whether he's kind or not isn't really the concern at this point.  The point is that there are real issues

Re: How to check if the zookeeper client is authenticated?

2011-06-20 Thread Patrick Hunt
? If it is then I have to catch the exception, process it (if it is actually authentication exception) and re-try my call (e.g. create znode) until it is successful. Not easy and obvious, isn't it? Regards, Semyon -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent

Re: session expiry

2011-06-24 Thread Patrick Hunt
2011/6/22 lei liu liulei...@gmail.com: In addition to, could you tell me some articles aobut session expiry? checkout items 1/2/3 in the faq, they highlight a number of issues: https://cwiki.apache.org/confluence/display/ZOOKEEPER/FAQ also the session section of the programmer guide has

Re: Serious problem processing hearbeat on login stampede

2011-07-05 Thread Patrick Hunt
Vishal brought up an issue at the ZK post-summit meetup that might also be (partially?) resolved by this patch. Thanks again Chang Song! Patrick 2011/7/1 Chang Song tru64...@me.com: No problem. Glad to contribute. Thanks a lot. 2011. 7. 2., 오전 1:03, Ted Dunning 작성: Thanks for the

Re: Zookeeper for Windows Server 2003 64 bit enterprise edition

2011-07-20 Thread Patrick Hunt
On Wed, Jul 20, 2011 at 12:06 AM, Ted Dunning ted.dunn...@gmail.com wrote: Zookeeper is written in Java and is thus very portable. Well there's also the C client issue (not avail on windows), support has been added for windows in the upcoming 3.4.0 release:

Re: Zookeeper/Hbase storage type on EC2

2011-07-21 Thread Patrick Hunt
On Thu, Jul 21, 2011 at 1:21 AM, Yves Langisch y...@langisch.ch wrote: I just need a statement if it makes sense to use ephemeral storage for ZK at all (in conjunction with Hbase if the answer depends on the use case)? Any help is appreciated. On 19.07.2011 19:37, Yves Langisch wrote: I

Re: Connecting to ZK from windows box takes significantly longer than from linux

2011-07-22 Thread Patrick Hunt
curious if everything works fine for other windows 7 users and we are special. On 07/22/2011 06:28 PM, Patrick Hunt wrote: Are you using the java or the c client binding? What version? I'd encourage you to enter a jira (and submit a patch if you figure it out). Regards, Patrick On Fri, Jul

Re: Zookeeper/Hbase storage type on EC2

2011-07-25 Thread Patrick Hunt
) - Original Message - From: Patrick Hunt ph...@apache.org To: user@zookeeper.apache.org Cc: Sent: Thursday, July 21, 2011 3:56 PM Subject: Re: Zookeeper/Hbase storage type on EC2 On Thu, Jul 21, 2011 at 1:21 AM, Yves Langisch y...@langisch.ch wrote:  I just need a statement if it makes sense

Re: Testing SessionExipred based on FAQ

2011-07-27 Thread Patrick Hunt
timeouts programmatically? On Fri, Jul 22, 2011 at 8:57 AM, Will Johnson willjohnsonsea...@gmail.comwrote: Created: https://issues.apache.org/jira/browse/ZOOKEEPER-1132 On Thu, Jul 21, 2011 at 6:25 PM, Patrick Hunt ph...@apache.org wrote: I think the FAQ is out of date, iirc it's probably due

Re: Unusual exception

2011-07-27 Thread Patrick Hunt
This is saying that the client (hbase in this case) is no longer connected to the server (tcp connection down) for some reason. This can happen for a number of reasons - typically the client process was killed, host restarted or partitioned from the server, etc There's nothing really to solve

Re: set wacher failed

2011-07-29 Thread Patrick Hunt
The docs; programmer guide, API, and FAQ are good places to start. Read through those first. You might also find it helpful to look at the recipe implementations (src/recipes) which implement common patterns including error handling. Regards, Patrick On Fri, Jul 29, 2011 at 1:23 AM, lei liu

Re: devops/admin/client question: What do you do when you rollback?

2011-08-04 Thread Patrick Hunt
has closed socket, closing socket connection and attempting reconnect (branch 3.3.3) I will poke around and see if I can figure out a nicer way to indicate this condition. The expired state is perfectly fine for me in my use case. C -Original Message- From: Patrick Hunt [mailto:ph

Re: devops/admin/client question: What do you do when you rollback?

2011-08-04 Thread Patrick Hunt
, using that field as an encoded error msg, thoughts? C On Aug 4, 2011 6:10 PM, Patrick Hunt ph...@apache.org wrote: Our error reporting server-client has always been weak. It's a PITA to debug in production because a lot of times when the client gets bounced it's not clear from the client

Re: zookeeper data backup

2011-08-05 Thread Patrick Hunt
We make only a fleeting reference to backup here in the admin docs http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_maintenance You might enter a JIRA. Having incomplete snapshots is fine - it's a common case that the server has to handle. Patrick On Fri, Aug 5, 2011 at 9:06 AM,

Re: zk keeps disconnecting and reconnecting

2011-08-25 Thread Patrick Hunt
The client seeing the problem in this case is 3.3.0, I see this based on the line number in the stack trace not matching up with 3.3.3, with 3.3.0 it's this line: event.setPath(serverPath.substring(chrootPath.length())); so for some reason your chroot path is negative in length?

Re: anyone familiar with the zookeeper perl client?

2011-08-25 Thread Patrick Hunt
Perhaps the client is failing while writing the request? (that's what the eof seems to indicate) Does the server side exception always look the same? What is going on in the client? (version? OS?, etc...?) Patrick On Fri, Aug 12, 2011 at 7:10 AM, Fournier, Camille F. camille.fourn...@gs.com

Re: ZooKeeper Limitation

2011-08-25 Thread Patrick Hunt
FYI: https://issues.apache.org/jira/browse/ZOOKEEPER-1162 We should probably try to mitigate this issue, it keeps coming up. Patrick On Sat, Aug 13, 2011 at 12:07 PM, Ted Dunning ted.dunn...@gmail.com wrote: Yes. But getting a list of children can happen in other contexts. You should not be

Re: Strange responses from server

2011-08-26 Thread Patrick Hunt
ruok checks if the server is listening on the port, that it's up and functional, not that it's joined a quorum stat provides details on the server wrt the quorum, not serving requests means that it is not currently a member of the quorum. You'd need to check the logs to see why it's not

Re: Strange responses from server

2011-08-26 Thread Patrick Hunt
. Diagnosing the network is an important step.  Diagnosing quorum formation is the next step. On Fri, Aug 26, 2011 at 11:31 AM, Jordan Zimmerman jzimmer...@netflix.comwrote: Thanks - that makes ruok essentially useless. Grrr... On 8/26/11 11:10 AM, Patrick Hunt ph...@apache.org wrote: ruok

Re: LinkedIn ZooKeeper Group

2011-08-26 Thread Patrick Hunt
As a group Zookeepers in general don't seem very social. (zero results for zookeeper groups) I don't see why not: http://www.linkedin.com/groups?about=gid=4061896trk=anet_ug_grppro Enjoy, Patrick On Fri, Aug 26, 2011 at 2:17 PM, Ben Hall b...@zynga.com wrote: Is there one?  I couldn't find

Re: sync()

2011-08-30 Thread Patrick Hunt
See this section of the programmers guide: http://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#ch_zkGuarantees specifically the section Simultaneously Consistent Cross-Client Views Patrick On Tue, Aug 30, 2011 at 9:52 AM, Jordan Zimmerman jzimmer...@netflix.com wrote: When/why

Re: sync()

2011-08-30 Thread Patrick Hunt
All reads. On Tue, Aug 30, 2011 at 10:12 AM, Jordan Zimmerman jzimmer...@netflix.com wrote: Thanks - if I understand correctly, this only applies to reads. Is that reads on data only (getData) or both data and a list of a Znode's children (getChildren)? On 8/30/11 10:07 AM, Patrick Hunt ph

Re: zk keeps disconnecting and reconnecting

2011-08-31 Thread Patrick Hunt
Based on past experience I believe it's going to take a fix release or two before 3.4 is rock solid, I personally think we should do a 3.3.4. Notice there are 6 blockers currently listed in 3.3.4

Re: sync()

2011-08-31 Thread Patrick Hunt
' in the ZK API sense. How does one invoke sync() from the C API? On Aug 30, 2011, at 1:18 PM, Patrick Hunt wrote: All reads. On Tue, Aug 30, 2011 at 10:12 AM, Jordan Zimmerman jzimmer...@netflix.com wrote: Thanks - if I understand correctly, this only applies to reads. Is that reads

Re: 'kill' isn't very deadly

2011-09-06 Thread Patrick Hunt
you might also look at bin/zkServer.sh, either to drive things or as an example of how to start/stop. On Tue, Sep 6, 2011 at 10:43 AM, Philip Smith philip_sm...@apple.com wrote: kill -9 Review the man page to understand how a process responds to the different signals. On Sep 6, 2011, at

Re: 'kill' isn't very deadly

2011-09-06 Thread Patrick Hunt
as saying it stops the instance. On 9/6/11 10:49 AM, Patrick Hunt ph...@apache.org wrote: you might also look at bin/zkServer.sh, either to drive things or as an example of how to start/stop. On Tue, Sep 6, 2011 at 10:43 AM, Philip Smith philip_sm...@apple.com wrote: kill -9 Review

Re: Dump all changes on ZK to file

2011-09-14 Thread Patrick Hunt
Consider submitting a patch. Patrick On Tue, Sep 13, 2011 at 11:44 PM, kishore g g.kish...@gmail.com wrote: Hi Ben, I was able to get more information by changing the LogFormatter code. Record txn = SerializeUtils.deserializeTxn(iab, hdr); Based on the txn type, I could cast it to

Re: SessionExpiredException. help please

2011-09-26 Thread Patrick Hunt
On Fri, Sep 23, 2011 at 4:46 PM, Yuhan Zhang yzh...@onescreen.com wrote: also, is auto-reconnection available through zooKeeper API? I'd like to keep a very long session/connection. Is this a good practice with zooKeeper? The client will always try to reconnect if it becomes disconnected,

Re: Exception while reading Snapshot file

2011-10-05 Thread Patrick Hunt
You are missing the header, see: org.apache.zookeeper.server.persistence.FileSnap.deserialize(DataTree, MapLong, Integer, InputArchive) Patrick On Tue, Oct 4, 2011 at 3:00 PM, kishore g g.kish...@gmail.com wrote: Hi, I am trying to load data from a ZK snapshot file into datatree. Here is the

Re: Total Outage of all ZooKeeper Instances

2011-10-12 Thread Patrick Hunt
We use a handful of exit(#) codes, none of which are -7 though. (we don't use 7 either). Perhaps this is some issue with the wrapper? Can you take a look? Are you loading any other code into the JVM that could account for this exit code? It's not clear to me from your description if the servers

Re: is there a 40s connection timeout that expires a session?

2011-10-19 Thread Patrick Hunt
The actual session timeout is negotiated btw the client and server, there is both a min and a max enforced by the server. See this section on session details: http://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#ch_zkSessions One of the parameters to the ZooKeeper client library call

Re: current leader

2011-10-19 Thread Patrick Hunt
We've talked before about putting that information into /zookeeper (ie as a znode). As long as auth can be applied to these znodes, similar to the rest of the space, it seems like a good place to me. Patrick On Wed, Oct 19, 2011 at 3:27 PM, Mahadev Konar maha...@hortonworks.com wrote: I think

Re: Zookeeper 3.4

2011-10-25 Thread Patrick Hunt
Voting is currently under way on the dev list: http://markmail.org/message/tpqkq57kl7tcp7xn On Tue, Oct 25, 2011 at 12:21 PM, Jun Rao jun...@gmail.com wrote: Hi, Has the release date for ZK 3.4 be determined? Thanks, Jun On Thu, Sep 29, 2011 at 7:47 PM, Dan Mihai Dumitriu

Re: How to connect zookeeper jmx by jconsole

2011-10-26 Thread Patrick Hunt
You probably want to read the ZK JMX docs to get started: http://zookeeper.apache.org/doc/r3.3.3/zookeeperJMX.html (short answ: it depends) On Wed, Oct 26, 2011 at 6:01 AM, nileader nilea...@gmail.com wrote: How to connect zookeeper jmx by jconsole. What's the jmx uri of zookeeprer.

Re: Zookeeper 3.4

2011-10-26 Thread Patrick Hunt
? Thanks, Jun On Tue, Oct 25, 2011 at 12:44 PM, Patrick Hunt ph...@apache.org wrote: Voting is currently under way on the dev list: http://markmail.org/message/tpqkq57kl7tcp7xn On Tue, Oct 25, 2011 at 12:21 PM, Jun Rao jun...@gmail.com wrote: Hi, Has the release date for ZK 3.4

Re: Zookeeper session losing some watchers

2011-11-02 Thread Patrick Hunt
On Wed, Nov 2, 2011 at 10:16 AM, Neha Narkhede neha.narkh...@gmail.com wrote: We've been seeing a problem with our zookeeper servers lately, where all of a sudden a session loses some of the watchers registered on some of the znodes. Let me explain our Kafka-ZK setup. We have a Kafka cluster

Re: Question regarding the maximum number of ZNODES a zookeeper

2011-11-09 Thread Patrick Hunt
On Wed, Nov 9, 2011 at 1:14 PM, Aniket Chakrabarti chakr...@cse.ohio-state.edu wrote: I am trying to load a huge matrix(100,000 by 500) to my zookeeper instance. Each element of the matrix is a ZNODE and value of each element is a digit(0-9). But I'm only able to load around 1000 x 500 nodes.

Re: error code in transaction log

2011-11-09 Thread Patrick Hunt
On Tue, Nov 8, 2011 at 8:29 AM, kishore g g.kish...@gmail.com wrote: Hi, I found these errors in the zk transaction log entry time:1320711820906 session:0x133808f05d5 cxid:0x3 zxid:0x10017 type:error err:-110 This generally happens before the setting the acl time:1320711821268

Re: Question regarding the maximum number of ZNODES a zookeeper

2011-11-09 Thread Patrick Hunt
Good points Ted. Aniket, you might try using visualvm or something similar to monitor heap usage during your test: http://visualvm.java.net/ It will give you some insight into how quickly the heap is being used up. You're reporting approx 4k/znode. That seems way high to me. Are you sure that

Re: Question regarding the maximum number of ZNODES a zookeeper

2011-11-10 Thread Patrick Hunt
On Thu, Nov 10, 2011 at 2:48 AM, Vitalii Tymchyshyn tiv...@gmail.com wrote: 10.11.11 06:53, Aniket Chakrabarti написав(ла): There is a limit on how many nodes can single parent node have if you use list operation. The limit is not direct, but rather it's a limit on response size. It can be

Re: ephemeral node not deleted after client session closed

2011-11-10 Thread Patrick Hunt
, Patrick Hunt ph...@apache.org wrote: ps. was there any instability in the quorum itself during this time period? Attaching logs to 1208 would be helpful. Patrick On Thu, Nov 10, 2011 at 11:50 AM, Patrick Hunt ph...@apache.org wrote: Great report, some addl questions: 1) Why is the session closed

Re: ephemeral node not deleted after client session closed

2011-11-10 Thread Patrick Hunt
On Thu, Nov 10, 2011 at 12:20 PM, Patrick Hunt ph...@apache.org wrote: Q: what are you clients doing? It's weird that a create would come from the client after the session has been closed. Do you have multiple threads sharing a session? The client (checked java) seems to protect against

Re: ephemeral node not deleted after client session closed

2011-11-10 Thread Patrick Hunt
According to 1208 this is 3.3.3. On Thu, Nov 10, 2011 at 1:34 PM, Camille Fournier cami...@apache.org wrote: This is zk 3.3.3? From my phone On Nov 10, 2011 4:02 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Nov 10, 2011 at 12:20 PM, Patrick Hunt ph...@apache.org wrote: Q: what are you

Re: setting zookeeper heap size

2011-11-10 Thread Patrick Hunt
In general ZK does not bound the heap size. The default is to use whatever the JVM prefers, which is typically based on available host memory. You can override this using JVMFLAGS. Patrick On Thu, Nov 10, 2011 at 1:47 PM, Ed Sexton sextonhad...@gmail.com wrote: Hello Group- I am running CDH

Re: ephemeral node not deleted after client session closed

2011-11-10 Thread Patrick Hunt
any such entries. So I'm guessing there was no election happening. Do you have thoughts, though, on how easy it would be to reproduce this bug, to verify the bug fix ? Thanks, Neha On Thu, Nov 10, 2011 at 2:08 PM, Patrick Hunt phu...@gmail.com wrote: On Thu, Nov 10, 2011 at 1:52 PM, Neha

Re: setting zookeeper heap size

2011-11-10 Thread Patrick Hunt
, 2011 4:59 PM, Patrick Hunt ph...@apache.org wrote: In general ZK does not bound the heap size. The default is to use whatever the JVM prefers, which is typically based on available host memory. You can override this using JVMFLAGS. Patrick On Thu, Nov 10, 2011 at 1:47 PM, Ed Sexton sextonhad

Re: ephemeral node not deleted after client session closed

2011-11-10 Thread Patrick Hunt
Ok, patch posted that fixes this (1208). Committers please take a look. Neha you might want to give a patched version a try. Awesome job helping to document and track down this issue. Thanks! Patrick On Thu, Nov 10, 2011 at 4:43 PM, Patrick Hunt ph...@apache.org wrote: See my update to 1208

Re: ephemeral node not deleted after client session closed

2011-11-11 Thread Patrick Hunt
as your patch. Will be a good opportunity for me to start understanding the zookeeper codebase. Thanks again, Neha On Thursday, November 10, 2011, Patrick Hunt ph...@apache.org wrote: Ok, patch posted that fixes this (1208). Committers please take a look. Neha you might want to give

Re: ZooKeeper Book?

2011-11-14 Thread Patrick Hunt
You should see my twitter search string. Damn those ppl who keep talking about standing in the queue for the movie! apache zookeeper does a reasonable job on Amazon. No book so far. Patrick On Sun, Nov 13, 2011 at 8:31 PM, Mike Schilli m...@perlmeister.com wrote: Aside from the Hadoop book

Re: Missing session state handling in most Leader Election implementations

2011-11-14 Thread Patrick Hunt
On Mon, Nov 14, 2011 at 3:25 AM, Jérémie BORDIER jeremie.bord...@gmail.com wrote: Good to hear. I was really wondering if I was missing something but took the risk to sound like an idiot on the list, really happy to have pointed that out. This stuff is hard, we try to answer any question

Re: how to set the starting value of the created sequence?

2011-11-14 Thread Patrick Hunt
How about using something like this instead? http://svn.apache.org/viewvc/incubator/flume/trunk/flume-core/src/main/java/com/cloudera/flume/master/ZooKeeperCounter.java?view=markup A second option would be to store the offset as the data for the znode (or in the parent), etc... Patrick On Mon,

Re: How can I get ip from session id.

2011-11-15 Thread Patrick Hunt
Client socket connections are local to a server, so you'd need to run cons on all servers to get the full sense of the connection activity Patrick On Mon, Nov 14, 2011 at 10:12 PM, nileader nilea...@gmail.com wrote: hi, everyone. When i exec        $  echo wchc | nc 192.168.2.176 2181

Re: How can I get ip from session id.

2011-11-15 Thread Patrick Hunt
. It would be nice if ZK logged the ip in the createSession transaction, so that we can get the ip from zk transaction log. On Tue, Nov 15, 2011 at 8:45 AM, Patrick Hunt ph...@apache.org wrote: Client socket connections are local to a server, so you'd need to run cons on all servers to get

Re: Any one have experience of tuning zookeeper's performance by set zookeeper zoo.cfg

2011-11-23 Thread Patrick Hunt
Typically you want to tune because you have some specific use case in mind. I suspect you're not getting any responses on this query because it's so vague. What issue are you facing? fwiw ZK has limited tuning knobs specifically so that you don't need to worry about tuning in most cases. Patrick

Re: More sync questions

2011-11-23 Thread Patrick Hunt
On Thu, Nov 10, 2011 at 10:29 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: A while back I asked about sync() and got responses that said it's only needed for reads (getData, getChildren). I was looking through the source and it appears that this does _not_ apply to exsists(). Am I

Re: [ANNOUNCE] Apache ZooKeeper 3.3.4

2011-11-29 Thread Patrick Hunt
Corrected link to release notes: http://zookeeper.apache.org/releases.html Patrick On Tue, Nov 29, 2011 at 9:53 AM, Patrick Hunt ph...@apache.org wrote: The Apache ZooKeeper team is proud to announce Apache ZooKeeper version 3.3.4 ZooKeeper is a high-performance coordination service

Re: How can I get ip from session id.

2011-11-29 Thread Patrick Hunt
It all depends on whether someone interested enough in seeing the feature added steps up and does the work. In other words, if you want to see it in see it in. Patrick On Mon, Nov 28, 2011 at 10:41 PM, nileader nilea...@gmail.com wrote: Will zookeeper add the feature

Re: Any command zookeeper administrator can use to close or reject client connetion with ip level?

2011-11-29 Thread Patrick Hunt
iptables is very flexible. There's even this which I think is pretty amazing (rate limit new connections): https://access.redhat.com/kb/docs/DOC-18730 On Mon, Nov 28, 2011 at 3:11 AM, nileader nilea...@gmail.com wrote: Any command zookeeper administrator can use to close or reject client

Re: java.nio.channels.CancelledKeyException in ZK 3.4

2011-11-30 Thread Patrick Hunt
I believe it's this: https://issues.apache.org/jira/browse/ZOOKEEPER-1237 Timing issue btw multiple threads (client closing and the server writing). The problem is that the socket has been closed prior to the server getting out the close session response to the client. Patrick On Wed, Nov 30,

Re: If i update my conf/zoo.cfg config, must i restart my zk server?

2011-11-30 Thread Patrick Hunt
fyi fix releases only include fixes. We typically put the new features into minor/major updates. Helps to maintain Operation's sanity. ;-) Also keep in mind that rolling restart is dead simple and works. 107 will be a nice option to have but is overkill in most scenarios. Patrick On Tue, Nov

Re: Invalid config

2011-12-01 Thread Patrick Hunt
It's this: Caused by: java.lang.IllegalArgumentException: zoo.cfg file is missing you need to pass the correct location of zoo.cfg when running the server. Patrick On Thu, Dec 1, 2011 at 12:33 PM, René Weinert rene_wein...@hotmail.com wrote: Hi! I tried to set up Zookeeper for the first

Re: Invalid config

2011-12-01 Thread Patrick Hunt
It's this: Caused by: java.lang.IllegalArgumentException: zoo.cfg file is missing you need to pass the correct location of zoo.cfg when running the server. Patrick On Thu, Dec 1, 2011 at 12:33 PM, René Weinert rene_wein...@hotmail.com wrote: Hi! I tried to set up Zookeeper for the first time

Re: Unusual exception

2011-12-02 Thread Patrick Hunt
What do the server logs indicate? On Thu, Dec 1, 2011 at 11:49 PM, adje ad.gaasb...@gmail.com wrote: I'm new to Zookeeper. The first tests all worked but now I'm getting the same exception as the previous posters. Unfortunately I don't see the answer. My situation is that I have the zookeeper

Re: Abandoned exists() watches are forever

2011-12-02 Thread Patrick Hunt
Good catch. I don't think that we can do anything automatically to resolve this. However there is a jira pending which would allow you to at least remove the watch when this does occur: https://issues.apache.org/jira/browse/ZOOKEEPER-442 Patrick On Fri, Dec 2, 2011 at 1:36 PM, Robert Crocombe

Re: Abandoned exists() watches are forever

2011-12-02 Thread Patrick Hunt
Cool. EOD exists watches are forever is a true statement though. That's the contract. This feature would allow you to explicitly remove the watch. On Fri, Dec 2, 2011 at 2:34 PM, Robert Crocombe rcroc...@gmail.com wrote: On Fri, Dec 2, 2011 at 2:58 PM, Patrick Hunt ph...@apache.org wrote: Good

Re: Determine if the distributed mode of ZK functions correctly

2011-12-06 Thread Patrick Hunt
I created the zk-smoketest project explicitly to help ops with this problem: https://github.com/phunt/zk-smoketest Patrick On Tue, Dec 6, 2011 at 9:39 AM, Florian Leibert f...@leibert.de wrote: Hi - I was wondering if there was an easy way to check if a given zookeeper server has joined an

Re: Determine if the distributed mode of ZK functions correctly

2011-12-06 Thread Patrick Hunt
... But this helps! Thanks! On Tue, Dec 6, 2011 at 11:39 AM, Patrick Hunt ph...@apache.org wrote: I created the zk-smoketest project explicitly to help ops with this problem: https://github.com/phunt/zk-smoketest Patrick On Tue, Dec 6, 2011 at 9:39 AM, Florian

Re: missing data after restarting+expanding a ZK 3.4.0 cluster

2011-12-06 Thread Patrick Hunt
This is being caused by a regression introduced by ZOOKEEPER-1136, see my comments on https://issues.apache.org/jira/browse/ZOOKEEPER-1319 This is a serious regression, I've talked with Mahadev and we'll be rolling a 3.4.1 soon to address it. (either later this week or early next) Patrick On

Re: 3.3.2 - 3.4.0 NodeExistsException when reading snapshot

2011-12-07 Thread Patrick Hunt
small correction: zkCli.sh -server host:port On Wed, Dec 7, 2011 at 10:19 AM, Mahadev Konar maha...@hortonworks.com wrote: Stefan,  It mostly looks like https://issues.apache.org/jira/browse/ZOOKEEPER-1319, but just to confirm can you do this - do a get on the same znode

Re: ZooKeeper 3.4.0 release.

2011-12-07 Thread Patrick Hunt
I believe this can effect not just 3.3-3.4 upgrades but also normal operation of a 3.4 cluster. I'd suggest holding off on 3.4 until 3.4.1 is available. (Mahadev and I are working on it for RC later this week, a patch should be available soon). Patrick On Wed, Dec 7, 2011 at 10:25 AM, Mahadev

Re: how to set zookeeper session time to forever?

2011-12-07 Thread Patrick Hunt
The client bindings are responsible for maintaining a heartbeat with the service: http://zookeeper.apache.org/doc/r3.3.4/zookeeperProgrammers.html#Java+Binding there's nothing you need to do here from your application's perspective. If the TCP connection is lost btw the client and server (or any

Re: ZooKeeper 3.4.0 release.

2011-12-07 Thread Patrick Hunt
Good idea Mahadev. Let's just be (more) explicit in our messaging about what's production ready and what's not. Patrick On Wed, Dec 7, 2011 at 1:56 PM, iv...@apache.org wrote: Sounds good to me. -Ivan On Wed, Dec 07, 2011 at 01:50:19PM -0800, Mahadev Konar wrote: Actually, I think Ill

Re: 3.3.2 - 3.4.0 NodeExistsException when reading snapshot

2011-12-08 Thread Patrick Hunt
. Stefan Zier On Dec 7, 2011, at 10:27 AM, Patrick Hunt wrote: small correction: zkCli.sh -server host:port On Wed, Dec 7, 2011 at 10:19 AM, Mahadev Konar maha...@hortonworks.com wrote: Stefan,  It mostly looks like https://issues.apache.org/jira/browse/ZOOKEEPER-1319, but just

Re: maven-metadata in Maven Central still points to 3.3.1

2011-12-12 Thread Patrick Hunt
The mirroring seems broken, here's the source: http://people.apache.org/repo/m2-ibiblio-rsync-repository/org/apache/zookeeper/zookeeper/maven-metadata.xml Patrick On Mon, Dec 12, 2011 at 2:44 AM, Berk D. Demir b...@mindcast.org wrote: On Sat, Dec 10, 2011 at 6:08 PM, Mahadev Konar

Re: maven-metadata in Maven Central still points to 3.3.1

2011-12-12 Thread Patrick Hunt
No idea - we have no control over third-party mirroring. Sounds like your best bet would be to set p.a.o as a repo in your pom. Patrick On Mon, Dec 12, 2011 at 1:37 PM, Berk D. Demir b...@mindcast.org wrote: On Mon, Dec 12, 2011 at 9:03 AM, Patrick Hunt ph...@apache.org wrote: The mirroring

Re: maven-metadata in Maven Central still points to 3.3.1

2011-12-12 Thread Patrick Hunt
who does control mirroring on repo1.maven.org? Anyone know? Is this Apache? (you might check wth Apache Infra) Patrick On Mon, Dec 12, 2011 at 2:04 PM, Patrick Hunt ph...@apache.org wrote: No idea - we have no control over third-party mirroring. Sounds like your best bet would be to set p.a.o

Re: nodes were erased after restart.

2011-12-13 Thread Patrick Hunt
Are you using /tmp to store your datadir? Patrick On Tue, Dec 13, 2011 at 1:55 PM, Yuhan Zhang yzh...@onescreen.com wrote: Hi all, I'm using zooKeeper as a message queue, creating each task as a persistent_sequential node under a folder (another node created in persistent mode). It works

Re: nodes were erased after restart.

2011-12-13 Thread Patrick Hunt
NP. You mentioned reboot, it's a pretty common misconfig. :-) Regards, Patrick On Tue, Dec 13, 2011 at 2:05 PM, Yuhan Zhang yzh...@onescreen.com wrote: ah, you are right! that's why it has been erased. Thanks Patrick. :) Yuhan On Tue, Dec 13, 2011 at 2:00 PM, Patrick Hunt ph...@apache.org

Re: Running zookeeper on multiple EC2 instances

2011-12-14 Thread Patrick Hunt
Take a look at whirr. http://whirr.apache.org/ That said, it's not clear to me what you mean. every time a new instance is spawned zk will run on it. Surely you don't mean a zk server (the max you might run is say 5 or perhaps 7 servers), however if it's a client then you'd be including the zk

Re: Running zookeeper on multiple EC2 instances

2011-12-14 Thread Patrick Hunt
from what I understood is I basically have to run the whirr launch cluster command 10 times? Yes. That will do it. Patrick On 12/15/2011 12:27 AM, Patrick Hunt wrote: Take a look at whirr. http://whirr.apache.org/ That said, it's not clear to me what you mean. every time a new instance

Re: New to zookeeper

2011-12-19 Thread Patrick Hunt
The admin guide is not helpful in this regard? If there are bits missing the community would benefit if you entered jiras highlighting the shortfall. Regards, Patrick On Mon, Dec 19, 2011 at 12:27 AM, David Song heven...@gmail.com wrote:  You can see Chapter 14 Zookeeper in Hadoop: The

Re: connection lost

2011-12-19 Thread Patrick Hunt
fyi That linger change was included in 3.3.4 and 3.4.1. Patrick On Mon, Dec 19, 2011 at 8:44 AM, Benjamin Reed br...@apache.org wrote: which version of the server are you using. this sounds like the linger bug that was fixed a while back. ben On Sun, Dec 18, 2011 at 6:22 PM, Camille

Re: Syntax for zkCli.sh commands

2011-12-21 Thread Patrick Hunt
Set it to true or 1 to register a watch in the shell. Patrick On Wed, Dec 21, 2011 at 11:35 AM, AJ s5a...@gmail.com wrote: ok. Can I get the syntax for the 'watch' option in the get and ls commands. Thanks. -- View this message in context:

  1   2   3   4   5   >