Re: Dynamic adding/removing ZK servers on client

2010-05-03 Thread Patrick Hunt
On 05/03/2010 07:03 AM, Dave Wright wrote: I've got a situation where I essentially need dynamic cluster membership, which has been talked about in ZOOKEEPER-107 but doesn't look like it's going to happen any time soon. Could you provide some insight into why you need this? Just so we have

Re: Pathological ZK cluster: 1 server verbosely WARN'ing, other 2 servers pegging CPU

2010-04-30 Thread Patrick Hunt
. For example high max latency might indicate that something (usually swap/gc) is causing the server to respond slowly in some cases. Patrick On 04/28/2010 10:47 PM, Patrick Hunt wrote: Hi Aaron, some questions/comments below: On 04/28/2010 06:29 PM, Aaron Crow wrote: We were running version 3.2.2

Re: Question on maintaining leader/membership status in zookeeper

2010-04-30 Thread Patrick Hunt
I believe Lei's concern is that the leader and all slaves can talk to ZK, but the slaves cannot talk to the leader. As a result no work can be done. However nothing will happen on the ZK side since everyone is heartbeating properly. Mahadev I think you came up with a pretty good solution.

Re: zookeeper consistency model?

2010-04-29 Thread Patrick Hunt
Hi Chen, this section of the docs has details on our guarantees, in particular see the note: http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkGuarantees Patrick On 04/29/2010 07:42 AM, Flavio Junqueira wrote: Hi Chen, Let's say that the value of a znode /test is

Re: Misbehaving zk servers

2010-04-29 Thread Patrick Hunt
Is there any good (simple/fast/bulletproof) way to monitor the FD use inside the jvm? If so we could stop accepting new client connections once we get close to the os imposed limit... The test would have to be a bulletproof one though - we wouldn't want to end up in some worse situation (where

Re: zookeeper consistency model?

2010-04-29 Thread Patrick Hunt
Ted, this is my browser homepage ;-) http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing Patrick On 04/29/2010 09:14 AM, Ted Dunning wrote: In general, the guarantee is that B will do exactly as you say it will read the new value or the old value. Your question depends on a

Re: Embedding ZK in another application

2010-04-29 Thread Patrick Hunt
On 04/29/2010 11:22 AM, Chad Harrington wrote: On Thu, Apr 29, 2010 at 8:49 AM, Patrick Huntph...@apache.org wrote: This is not foolproof however. We found that in general this would work, however there were some infrequent cases where a restarted server would fail to initialize due to the

Re: bug: wrong heading in recipes doc

2010-04-22 Thread Patrick Hunt
Hi Adam, would you mind creating a JIRA? That's the best way to address this type of issue. Thanks! https://issues.apache.org/jira/browse/ZOOKEEPER Patrick On 04/22/2010 11:30 AM, Adam Rosien wrote: http://hadoop.apache.org/zookeeper/docs/r3.3.0/recipes.html#sc_recoverableSharedLocks uses the

Re: bug: wrong heading in recipes doc

2010-04-22 Thread Patrick Hunt
No problem. https://issues.apache.org/jira/browse/ZOOKEEPER-752 I've seen alot of traffic on infrastruct...@apache, you might try there, I'm sure they could help you out. Regards, Patrick On 04/22/2010 01:26 PM, Adam Rosien wrote: I would, but the Apache JIRA has been f***ed since the

Re: Client reconnection after a connection loss

2010-04-21 Thread Patrick Hunt
Hi Julien, take a look at the FAQ for some background: http://wiki.apache.org/hadoop/ZooKeeper/FAQ#A3 Basically when your client connects to the server it provides a timeout value. If the server doesn't hear from your client within this timeout period (we do heartbeating for you in a

Re: Would this work?

2010-04-20 Thread Patrick Hunt
There are a small handful of cases where the server code will system.exit. This is typically only if quorum communication fails in some weird, unrecoverable way. We've been working to remove this (mainly so zk can be deployed in a container) but there are still a few cases left. I don't see

Re: Recovery issue - how to debug?

2010-04-19 Thread Patrick Hunt
Usually the server logs will shed light on such issues. If we had access to them it might be easier to speculate. Patrick On 04/19/2010 09:22 AM, Mahadev Konar wrote: Hi Hao, As Vishal already asked, how are you determining if the writes are being received? Also, what was the status of

Re: znode cversion decreasing?

2010-04-12 Thread Patrick Hunt
Hi Kevin, The server increments a znode's cversion by one each time a change to it's child list is made. Every znode has it's own cversion. It should never decrease. If you delete a znode and create it a new then the cversion is reset for that znode. The cversion also happens to be used for

Re: feed queue fetcher with hadoop/zookeeper/gearman?

2010-04-12 Thread Patrick Hunt
See this environment http://bit.ly/4ekN8G. Subsequently I used the 3 server setup, each configured with 8gig of heap in the jvm and 4 CPUs/jvm (I think I used 10second session timeouts for this) for some additional testing that I've not written up yet. I was able to run ~500 clients (same test

Re: znode cversion decreasing?

2010-04-12 Thread Patrick Hunt
We did have a case where the user setup 3 servers, each was standalone. :-) Doesn't look like that's the problem here though given you only specify 1 server in the connect string (although as mahadev mentioned you don't need to worry about that aspect). After it goes 7-11-9, does it ever go

Re: znode cversion decreasing?

2010-04-12 Thread Patrick Hunt
On 04/12/2010 03:58 PM, Kevin Webb wrote: On Mon, 12 Apr 2010 15:09:20 -0700 Patrick Huntph...@apache.org wrote: We did have a case where the user setup 3 servers, each was standalone. :-) Doesn't look like that's the problem here though given you only specify 1 server in the connect string

Re: znode cversion decreasing?

2010-04-12 Thread Patrick Hunt
Probably reaching for straws but could you print path, just to confirm it's what you know it is? Patrick On 04/12/2010 02:53 PM, Kevin Webb wrote: On Mon, 12 Apr 2010 14:33:44 -0700 Mahadev Konarmaha...@yahoo-inc.com wrote: Hi Kevin, Thanks for the info. Could you cut and paste the code

Re: how to build an mt version of shared lib for zookeeper under cygwin?

2010-04-08 Thread Patrick Hunt
That is weird. Ben tried with 1.7 as part of the release and it was fine. Looks like this issue has happened to others, but this wasn't very recent: http://cygwin.com/ml/cygwin/2009-05/msg00063.html Perhaps you can try upgrading your cygwin install to the latest (ensure you have the latest)

Re: how to build an mt version of shared lib for zookeeper under cygwin?

2010-04-07 Thread Patrick Hunt
Are you running cygwin 1.7 or 1.5? I believe there was a change btw versions of cygwin on how pthreads was supported. Patrick On 04/07/2010 03:08 AM, zd.wbh wrote: hi, all: Is it possible to build an mt version of zookeeper dll lib? I've tried, cygzookeeper_st-2.dll come out successfully,

Re: how to build an mt version of shared lib for zookeeper under cygwin?

2010-04-07 Thread Patrick Hunt
when you run configure what does it say for pthreads? My cygwin says this: checking for pthread_mutex_lock in -lpthread... yes Do you have pthreads libs in /usr/lib? I see: $ ls /usr/lib/*pth* /usr/lib/libpthread.a* You could also try running configure and the build, capture all the output

Re: the error

2010-03-31 Thread Patrick Hunt
Hi Li, when you say 17 threads reading a znode, do you mean that you have 17 threads each creating a session and using that session to read a znode? If so it's probably due to this: http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html#sc_advancedConfiguration see the parameter

Re: Solitication for logging/debugging requirements

2010-03-29 Thread Patrick Hunt
Take a look at the logging page in the docs: http://hadoop.apache.org/zookeeper/docs/current/zookeeperInternals.html#sc_logging Some good guidelines in there. Basically we log things at info level that are interesting/informational but not logged so frequently that they fill the log. WARN is

[ANNOUNCE] Apache ZooKeeper 3.3.0

2010-03-26 Thread Patrick Hunt
The Apache ZooKeeper team is proud to announce Apache ZooKeeper version 3.3.0. ZooKeeper is a high-performance coordination service for distributed applications. It exposes common services - such as naming, configuration management, synchronization, and group services - in a simple interface

Re: permanent ZSESSIONMOVED

2010-03-18 Thread Patrick Hunt
:39, Patrick Hunt ph...@apache.org wrote: Yea, that's great. (no complaints on my end, just wasn't sure what you meant, wanted to make sure I was clear.). Can you identify some sort of pattern? We're discussing on our end, but this is the first time we've heard of such an issue (outside 3.2.0 bug

Re: permanent ZSESSIONMOVED

2010-03-17 Thread Patrick Hunt
, Łukasz On Tue, Mar 16, 2010 at 20:39, Patrick Hunt ph...@apache.org wrote: Yea, that's great. (no complaints on my end, just wasn't sure what you meant, wanted to make sure I was clear.). Can you identify some sort of pattern? We're discussing on our end, but this is the first time we've heard

Re: permanent ZSESSIONMOVED

2010-03-16 Thread Patrick Hunt
Can you verify that you are using 3.2.2 on all servers? You can do this by running the stat command against each of your servers and look at the very top of the output (we include the version of zk server there). http://bit.ly/dglVld Are you using synchronous or async operations in your

Re: permanent ZSESSIONMOVED

2010-03-16 Thread Patrick Hunt
after reconnect, what's the second part saying?) Regards, Patrick Łukasz Osipiuk wrote: On Tue, Mar 16, 2010 at 19:22, Patrick Hunt ph...@apache.org wrote: Łukasz Osipiuk wrote: On Tue, Mar 16, 2010 at 17:18, Patrick Hunt ph...@apache.org wrote: Can you verify that you are using 3.2.2 on all

Re: permanent ZSESSIONMOVED

2010-03-16 Thread Patrick Hunt
] - FOLLOWING Additionally if you use the stat 4letter word you will see the current status of the server, leader or follower. (JMX as well) You might also find this useful: http://github.com/phunt/zktop Patrick Łukasz Osipiuk wrote: On Tue, Mar 16, 2010 at 20:05, Patrick Hunt ph...@apache.org wrote

Re: permanent ZSESSIONMOVED

2010-03-16 Thread Patrick Hunt
Agree. Patrick Ted Dunning wrote: This is very different from most uses of ZK. Normally Zookeeper is used on a private network with almost no packet loss. Your high rate of loss may be tickling a protocol bug that other people just never see. On Tue, Mar 16, 2010 at 12:24 PM, Łukasz Osipiuk

Re: permanent ZSESSIONMOVED

2010-03-16 Thread Patrick Hunt
. (rather than trying to restart the client as you have been doing). Not sure if you can try this (production env?) but it would be an interesting additional data point if you can give it a try. Regards, Patrick Patrick Hunt wrote: Yes, if you search back (older entries) in the server log you

Re: maxClientCnxns

2010-03-15 Thread Patrick Hunt
I'm afraid the docs are wrong, it's being fixed in 3.3.0: https://issues.apache.org/jira/browse/ZOOKEEPER-681 set the value to 0 if you want unlimited. Patrick Michael Bauland wrote: Hi, I've got another question regarding the connection limit. I sometimes get the following warning in my

Re: Managing multi-site clusters with Zookeeper

2010-03-15 Thread Patrick Hunt
Michael Bauland wrote: - When I connect with a client to the Zookeeper ensemble I provide the three IP addresses of my three Zookeeper servers. Does the client then choose one of them arbitrarily or will it always try to connect to the first one first? I'm asking since I would like to have my

Re: Managing multi-site clusters with Zookeeper

2010-03-08 Thread Patrick Hunt
IMO latency is the primary issue you will face, but also keep in mind reliability w/in a colo. Say you have 3 colos (obv can't be 2), if you only have 3 servers, one in each colo, you will be reliable but clients w/in each colo will have to connect to a remote colo if the local fails. You

Re: Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread Patrick Hunt
See the troubleshooting page, some apropos detail there (esp relative to virtual env). http://wiki.apache.org/hadoop/ZooKeeper/Troubleshooting ZK servers are sensitive to IO (disk/network) latency. As long as you aren't very sensitive latency requirements it should be fine. If the machine

Re: Managing multi-site clusters with Zookeeper

2010-03-08 Thread Patrick Hunt
be resumed when the link reappears. I need to get this running in the lab and start pulling out wires. regards, Martin On 8 March 2010 17:39, Patrick Hunt ph...@apache.org wrote: IMO latency is the primary issue you will face, but also keep in mind reliability w/in a colo. Say you have 3 colos

Re: Usage of myId

2010-03-05 Thread Patrick Hunt
re previous discussion see this thread: http://www.mail-archive.com/zookeeper-user@hadoop.apache.org/msg00916.html Patrick Nick Dimiduk wrote: I think automating this assignment has been discussed in the past. IIRC, some combination of MAC address and pid or timestamp would be sufficient to

Re: zookeeper utils

2010-03-03 Thread Patrick Hunt
This is def. something we should add to the recipes (docs code lib), Henry/David can you create a jira for this? Patrick Henry Robinson wrote: Just to illustrate one of the primitives you're looking for: an AtomicInteger equivalent would be fairly easy to construct, with nearly identical

Re: how to lock one-of-many ?

2010-02-24 Thread Patrick Hunt
on the locks directory to make the wait as long as possible AND as short as possible. On Wed, Feb 24, 2010 at 8:53 AM, Patrick Hunt ph...@apache.org wrote: Anyone interested in locking an explicit resource attempts to create an ephemeral node in /locks with the same ### as they resource they want access

Re: Bit of help debugging a TIMED OUT session please

2010-02-23 Thread Patrick Hunt
Stack you might look at the following: 1) why does server 14 have such a low recv count? Received: 194 while the other servers are at 3.7k + received. Did server 14 fail at some point? Or it's network? This may have caused the timeout seen by the client: --snippet-

Re: Eclipse Plug-in for ZooKeeper released

2010-02-22 Thread Patrick Hunt
Wow, this is impressive. I haven't gotten a chance to try it yet but it looks great. Thanks Mark! If you are interested to have it included in the official ZooKeeper release please submit a patch for contrib. It would be great to get this included and pushed out to users as part of a release.

Re: Zab documentation ?

2010-02-19 Thread Patrick Hunt
Mahadev can you put a link to that up on the wiki? Patrick Mahadev Konar wrote: Hi Kay, There is a paper Brief Announcement Zab A Practical Totally Ordered Broadcast Protocol. in DISC 2009 By ben and flavio which has the details of zab in it. Thanks mahadev On 2/18/10 6:07 PM, Kay Kay

Re: zookeeper for gearman?

2010-02-19 Thread Patrick Hunt
Thomas, I've looked at integrating the two, so far as to download the gearman source and examine it a bit. I didn't see a huge near-term win implementing a plugin as gearman already has support for drizzle/memcached/sqlite4/pq. While ZK could be used to provide highly reliable/available

Re: When session expired event fired?

2010-02-08 Thread Patrick Hunt
Here's one that I know of that's open - LinkedIn's Norbert http://wiki.github.com/rhavyn/norbert/ Patrick neptune wrote: Thanks for your quick answer. I will make cluster member ship service using zookeeper. If a node in a cluster can't connect to zookeeper cluster, the node killed by oneself

Re: When session expired event fired?

2010-02-08 Thread Patrick Hunt
this? How should I handle SESSION_EXPIRED? http://wiki.apache.org/hadoop/ZooKeeper/FAQ#A3 Benjamin Reed wrote: i was looking through the docs to see if we talk about handling session expired, but i couldn't find anything. we should probably open a jira to add to the docs, unless i missed

Re: When session expired event fired?

2010-02-08 Thread Patrick Hunt
for more the the session timeout ... That's why I use Expired event for cluster membership. In my case client is a node in a cluster. 2010/2/9 Patrick Hunt ph...@apache.org this? How should I handle SESSION_EXPIRED? http://wiki.apache.org/hadoop/ZooKeeper/FAQ#A3 Benjamin Reed wrote: i

Re: When session expired event fired?

2010-02-08 Thread Patrick Hunt
the ZooKeeper service for more the the session timeout ... That's why I use Expired event for cluster membership. In my case client is a node in a cluster. 2010/2/9 Patrick Hunt ph...@apache.org this? How should I handle SESSION_EXPIRED? http://wiki.apache.org/hadoop/ZooKeeper/FAQ#A3 Benjamin Reed

Re: ZOOKEEPER-22 and release 3.3

2010-02-03 Thread Patrick Hunt
While this is a very useful improvement it sounds to me like the prudent thing to do given the short time to 3.3.0. If you want we can shoot for 3.4.0 soon after 3.3.0 goes out (with 22 as one of the primary features). Patrick Mahadev Konar wrote: Hi all, I had been working on zookeeper-22

Re: question regarding connectionloss

2010-02-02 Thread Patrick Hunt
You should never see connection loss except in the case where you have some network partition or some other issue that causes communication issues btw the client and server. (client swapping? server swapping or either having GC pause issues? etc...) Are you monitoring your hosts/network/jvms,

Re: stop script not working

2010-01-27 Thread Patrick Hunt
This is zk 3.2.2? Are you sure that's all it said on start? When I start I see: $ bin/zkServer.sh start JMX enabled by default Using config: /home/phunt/dev/workspace/gitzk/bin/../conf/zoo.cfg Starting zookeeper ... STARTED you might have some error case, for example if I start zk w/o having a

Re: Server exception when closing session

2010-01-25 Thread Patrick Hunt
GC and disk IO (transactional log in particular) will cause significant latency in some cases. See this for details on the types of things you should look at: http://wiki.apache.org/hadoop/ZooKeeper/Troubleshooting I've seen cases where the JVM will pause for 2+ minutes for GC, in some cases

Re: ZooKeeper Dashboard: Error: No module named zookeeper_dashboard.zkadmin

2010-01-25 Thread Patrick Hunt
Rename phunt-zookeeper_dashboard-43ce91a to zookeeper_dashboard (or change the django files inside phunt-zookeeper_dashboard-43ce91a to use this as the module name) Patrick Eric Scheie wrote: I am having trouble getting the ZooKeeper Dashboard up and running. I installed Django-1.1.1 and

Re: Killing a zookeeper server

2010-01-25 Thread Patrick Hunt
(myid is 2) and we started it around 10:03:21 Thx! J-D On Mon, Jan 25, 2010 at 10:49 AM, Patrick Hunt ph...@apache.org wrote: 1) Capture the logs from all 5 servers 2) give the config for the down server, also indicate that it's server id is. 3) if possible it would be interesting to see

Re: Killing a zookeeper server

2010-01-25 Thread Patrick Hunt
JD, there's something _very_ unusual in your setup. Are you running official released ZooKeeper code or something else? Either there is a misconfiguration on the other servers (the configs for the other servers is exactly the same as 222 right?), or perhaps some patches to ZK codebase that

Re: Server exception when closing session

2010-01-25 Thread Patrick Hunt
experience. Lots of times that bad user experience is somewhat covered up by your load balancer and other general redundancy to you may notice it first from ZK forcing you to think about these things. On Mon, Jan 25, 2010 at 9:43 AM, Patrick Hunt ph...@apache.org wrote: GC and disk IO (transactional

Re: Killing a zookeeper server

2010-01-14 Thread Patrick Hunt
any of these and analyze them just let me know. Thanks for the help, Nick Bailey On Jan 12, 2010, at 8:20 PM, Patrick Hunt ph...@apache.org wrote: Nick Bailey wrote: In my last email I failded to include a log line that may be revelent as well 2010-01-12 18:33:10,658 [QuorumPeer:/0.0.0.0:2181

Re: Killing a zookeeper server

2010-01-14 Thread Patrick Hunt
Btw, here's an excellent example of these 4letterwords being used in a monitoring application ;-) zktop - http://bit.ly/1iMZdg Patrick Patrick Hunt wrote: ruok basically is polling to see if the ZK process is ok, which it is, it's just that zk is not part of a quorum (which is potentially

Re: Recommendations for zookeeper deployment

2010-01-12 Thread Patrick Hunt
Mekaraj, Prashant wrote: Hi, http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html is a great resource. It's rare to see a open source project think so much about practical enterprise deployment and this is much appreciated. Thanks! There are a few more recommendations that

Re: Killing a zookeeper server

2010-01-12 Thread Patrick Hunt
12 servers? That's alot, if you dont' mind my asking why so many? Typically we recommend 5 - that way you can have one down for maintenance and still have a failure that doesn't bring down the cluster. The electing a leader is probably the restarted machine attempting to re-join the ensemble

Re: Killing a zookeeper server

2010-01-12 Thread Patrick Hunt
. -Original Message- From: Patrick Hunt ph...@apache.org Sent: Tuesday, January 12, 2010 5:40pm To: zookeeper-user@hadoop.apache.org, nicholas.bai...@rackspace.com Subject: Re: Killing a zookeeper server 12 servers? That's alot, if you dont' mind my asking why so many? Typically we recommend 5

Re: blog post about zk and group membership

2010-01-11 Thread Patrick Hunt
Adam, this is awesome! I've retweeted it on my twitter account: http://twitter.com/phunt/status/7637054275 A few comments: 1) you are correct about the off the shelf part, but really alot of that is our marketing blurb, we paint with a broad stroke. I think it would make sense to include this

Re: Why is not win32 usable in production?

2010-01-08 Thread Patrick Hunt
The primary reason is testing. Secondly there just hasn't been much (any really) interest in win32 for production, it's not used on that platform and confidence is low as a result. We'd need to find one or more users/developers that are interested in testing and fixing any issues found on

Re: Share Zookeeper instance and Connection Limits

2009-12-16 Thread Patrick Hunt
Thiago Borges wrote: I read the documentation at zoo site and can't find some text about sharing/limits of zoo clients connections. No limits in particular to ZK itself (given enough memory) - usually the limitations are due to the max number of file descriptors the host OS allows. Often

Re: size of data / number of znodes

2009-12-15 Thread Patrick Hunt
See this recent benchmark I did: http://bit.ly/4ekN8G In this case I have 20 clients doing 10k zodes each (200k znodes of size 100 bytes each with 1million watches). However I have tested similar setup with 400 clients (so 4 million znodes and 20million watches). As Ben mentioned there are

[ANNOUNCE] Apache ZooKeeper 3.1.2

2009-12-14 Thread Patrick Hunt
The Apache ZooKeeper team is proud to announce Apache ZooKeeper version 3.1.2. ZooKeeper is a high-performance coordination service for distributed applications. It exposes common services - such as naming, configuration management, synchronization, and group services - in a simple interface

[ANNOUNCE] Apache ZooKeeper 3.2.2

2009-12-14 Thread Patrick Hunt
The Apache ZooKeeper team is proud to announce Apache ZooKeeper version 3.2.2. ZooKeeper is a high-performance coordination service for distributed applications. It exposes common services - such as naming, configuration management, synchronization, and group services - in a simple interface

Re: Starting Zookeeper on Amazon EC2

2009-12-10 Thread Patrick Hunt
Btw, the docs for a particular version of ZK are always included in the release (under docs toplevel directory). The docs on the apache site are always for the latest stable release (currently 3.2.1). That's why you saw the discrepancy. Patrick Something Something wrote: Switched to 3.2.1.

Re: Starting Zookeeper on Amazon EC2

2009-12-10 Thread Patrick Hunt
One thing missing from the list is that you need to have myid files on each server that correspond to the servers id (specified in the conf file). http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html#sc_zkMulitServerSetup You'd have a config on each system that looks something

Re: Starting Zookeeper on Amazon EC2

2009-12-10 Thread Patrick Hunt
Would you be willing to create a zk on ec2 page on the wiki? http://wiki.apache.org/hadoop/ZooKeeper/ZooKeeperOnEC2 Not sure on the exactly content -- basically outline what you did to get ZK running. Something you think would be helpful to other users trying to deploy ZK on ec2 for the first

Re: SLF4J for logging

2009-12-04 Thread Patrick Hunt
I pinged hadoop general@ a few months ago and got lukewarm reception (generally having multiple logger backends sux). But no one really seemed too interested one way or the other: http://bit.ly/71M6tK I'm not sure if anyone is thinking about this wrt hadoop as a whole... For example, what's

Re: Failure to Launch

2009-12-01 Thread Patrick Hunt
Hi Alan, turns out there were a number of issues with the scripts on cygwin, see the patch attached to the JIRA that will be applied to 3.3.0. https://issues.apache.org/jira/browse/ZOOKEEPER-606 See the tar attached to this email, give these scripts a try (replace your entire bin directory with

Re: Failure to Launch

2009-11-25 Thread Patrick Hunt
We don't use xerces, perhaps it's picking up your default classpath? Try unsetting your default classpath and try that again. unset CLASSPATH or (the following should also work I think) CLASSPATH= bin/zkServer.sh start Patrick Ratner, Alan S (IS) wrote: I have been unsuccessful at getting

Re: Error running contrib tests

2009-11-24 Thread Patrick Hunt
Yes, that was my thinking as well - something like configure does (autotools) where it will tell you some details after calling it (although in this case the output is after install). Sounds good! Patrick Mahadev Konar wrote: I would suggest adding it to the readme no? Modifying /etc files

Re: c client on win32

2009-11-24 Thread Patrick Hunt
on checking the static flag for g++ in configure in 3.2.1 Weird. Maarten Koopmans schreef: I applied the patch to the trunk, but somehow autoconf fails on the fresh checkout (or my brain fails me, more likely ;-) What do you do on checkout to get a build of the trunk? --Maarten Patrick Hunt

Re: Asking zk cluster how its configured and whats this expired about?

2009-11-24 Thread Patrick Hunt
stack wrote: I want to ask a running zk cluster what its configuration is -- ticktime, session timeout, etc. -- but do not see how. There are the four letter words. Dump and stat do not print what I want. I took a look in logs -- the leader in particular -- and do not see vitals dumped out.

Re: Asking zk cluster how its configured and whats this expired about?

2009-11-24 Thread Patrick Hunt
stack wrote: On Tue, Nov 24, 2009 at 1:33 PM, Patrick Hunt ph...@apache.org wrote: We can definitely add this, please create a JIRA. ZOOKEEPER-595 Great, thanks! I was also wondering what this expire stuff in the dump output is about? Those are the expiration sets, or buckets. Each

[Fwd: Re: 1st Hadoop India User Group meet]

2009-11-23 Thread Patrick Hunt
See attached - Hadoop India User Group meet up on 28th November 2009 in Noida, India. Aby Abraham will be presenting on ZooKeeper. Patrick ---BeginMessage--- Here is the updated agenda for Hadoop India User Group meet up on 28th November 2009 in Noida- - Introductions - Sessions- -

Re: Using ZooKeeper for Configuration Managment

2009-11-23 Thread Patrick Hunt
Config management and group membership are two of the simplest use cases for ZK, we don't have a recipe here as it's pretty basic (not that we couldn't/shouldn't) The java example in our docs is pretty close, it sets a watch on a node and monitors for changes. This is the typical pattern for

Re: c client on win32

2009-11-23 Thread Patrick Hunt
fyi, this patch allows compilation under cygwin but the tests are currently not passing (probably not handling the space in windows directory names correctly, but haven't had a chance to track it down). This should go into 3.3.0 https://issues.apache.org/jira/browse/ZOOKEEPER-586 Maarten

Re: c client on win32

2009-11-23 Thread Patrick Hunt
tomorrow. --Maarten Op 23 nov 2009 om 18:38 heeft Patrick Hunt ph...@apache.org het volgende geschreven:\ fyi, this patch allows compilation under cygwin but the tests are currently not passing (probably not handling the space in windows directory names correctly, but haven't had a chance to track

Re: Error running contrib tests

2009-11-20 Thread Patrick Hunt
What version of ZooKeeper is this? If it's 3.2.1 take a look at this: binding fixes in 3.2.2 https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=truepid=12310801component=12312860fixfor=12314335resolution=1sorter/field=prioritysorter/order=DESC You might give the 3.2 branch (in svn)

Re: c client on win32

2009-11-20 Thread Patrick Hunt
Maarten, I just tried this with cygwin and it fails for me too. It seems that cygwin does not support getaddrinfo! Please create a JIRA and I'll see what we can do. Patrick Maarten Koopmans wrote: Hi, Has anybody managed to get the c client / dll compiled on Win32, and if so, how? I did a

Re: c client on win32

2009-11-20 Thread Patrick Hunt
citizen. Besides, I think we might wnat to try with the beta of Cygwin first before filing it in Jira. Ultimately the goal (IMHO) should be MinGW support(?) --Maarten Patrick Hunt schreef: Maarten, I just tried this with cygwin and it fails for me too. It seems that cygwin does not support

Re: c client on win32

2009-11-20 Thread Patrick Hunt
function pointers). If I can get ZooKeeper talking via C (or via TCP, but the protocol doesn't seem to be specified) from REBOL, I have some very cool things coming. But maybe we should talk off-list aboutt that (will be open sourced though). --Maarten Patrick Hunt schreef: Well I can tell you

Re: Please disregard - Re: Exception on close of connection (WAS - Re: c client on win32)

2009-11-20 Thread Patrick Hunt
Oops too late. ;-) I'm perplexed as to why you see all these expirations though. Are you killing your clients, ie not cleaning up the ZK session gracefully via close()? Patrick stack wrote: Please disregard. Sorry for the noise (Patrick, of note, I am seeing this session timeout on a

Re: Observers!

2009-11-18 Thread Patrick Hunt
I'm really psyched to see this go into the 3.3 code line, kudos to Henry and the rest of the team in getting this landed! Patrick ps. gentle reminder - the user list is primarily for answering user questions on released software, many users don't follow future developments and can be

[Fwd: Hadoop User Group (Bay Area) - next Wednesday (Nov 18th) at Yahoo!]

2009-11-17 Thread Patrick Hunt
Tomorrow is the BA HUG, if anyone is interested to talk with Mahadev or I f2f regarding ZooKeeper we'll both be in attendance. Patrick ---BeginMessage--- Hi all, We are one week away from the next Bay Area Hadoop User Group - Yahoo! Sunnyvale Campus, next Wednesday (Nov 18th) at 6PM We have

[ATTN] If you upgraded from v2 of ZooKeeper to v3 please read this.

2009-11-17 Thread Patrick Hunt
We've found an issue with the migration tool used to migrate users from version 2 of ZooKeeper to version 3. This tool was provided to users who upgraded from the SourceForge v2 ZK, after we moved to being a subproject of Apache Hadoop (which is the same time that we incremented the version

Re: Some thoughts on Zookeeper after using it for a while in the CXF/DOSGi subproject

2009-11-11 Thread Patrick Hunt
and I'll report back... David 2009/11/10 Patrick Hunt ph...@apache.org I couldn't find a JIRA for removing the sys exits so I created one: https://issues.apache.org/jira/browse/ZOOKEEPER-575 there's also this which seems like it should be easy for someone who knows osgi container jar format

Re: ZK on EC2

2009-11-10 Thread Patrick Hunt
is generally about what you saw. We have been able to sustain 20-30MB/s into EC2 to a single node which should be harder than moving data between nodes. I have heard rumors that others were able to get double what I got for incoming transfer. On Mon, Nov 9, 2009 at 9:47 PM, Patrick Hunt ph...@apache.org

Re: ZK on EC2

2009-11-10 Thread Patrick Hunt
that we blamed EC2 for some of our own GC misconfiguration. I would not use our configuration here as canonical since we didn't apply a whole lot of brainpower to this problem. On Tue, Nov 10, 2009 at 9:29 AM, Patrick Hunt ph...@apache.org wrote: Ted, could you provide your configuration information

Re: Authentication, encryption, and dynamic membership

2009-11-10 Thread Patrick Hunt
Ben has been looking at using netty instead of our own direct NIO code. My understanding is that it supports the encryption/auth between servers, but I'm not 100% on that -- Ben, can you elaborate? Patrick Gustavo Niemeyer wrote: Dear ZooKeepers, I'm quite interested in the features related

Re: Some thoughts on Zookeeper after using it for a while in the CXF/DOSGi subproject

2009-11-10 Thread Patrick Hunt
I couldn't find a JIRA for removing the sys exits so I created one: https://issues.apache.org/jira/browse/ZOOKEEPER-575 there's also this which seems like it should be easy for someone who knows osgi container jar format requirements: https://issues.apache.org/jira/browse/ZOOKEEPER-425 Now we

Re: ZK on EC2

2009-11-09 Thread Patrick Hunt
I'm really interested to know how ec2 compares wrt disk and network performance to what I've documented here under the hardware section: http://wiki.apache.org/hadoop/ZooKeeper/ServiceLatencyOverview#Hardware Is it possible for someone to compare the network and disk performance (scp, dd,

Re: ZK on EC2

2009-11-09 Thread Patrick Hunt
-mumble:~# On Mon, Nov 9, 2009 at 4:54 PM, Patrick Hunt ph...@apache.org wrote: I'm really interested to know how ec2 compares wrt disk and network performance to what I've documented here under the hardware section: http://wiki.apache.org/hadoop/ZooKeeper/ServiceLatencyOverview#Hardware

ZooKeeper Service Latency Overview

2009-10-28 Thread Patrick Hunt
Some users have asked what performance one should expect from a ZK cluster based on number of cores, client load, disk performance, etc... I put together the following survey that looks at performance on hardware with varying core counts using a shared 7200RPM disk. Many of our users don't

Re: Cluster Configuration Issues

2009-10-23 Thread Patrick Hunt
on a few other ports as well and it's giving me the same issue. Do I need to tamper with port permissions? -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Friday, October 23, 2009 1:23 AM To: zookeeper-user@hadoop.apache.org; Mark Vigeant Subject: Re: Cluster Configuration

Re: Cluster Configuration Issues

2009-10-23 Thread Patrick Hunt
Great to hear. Good luck. Patrick Mark Vigeant wrote: Oh awesome! Thanks Patrick! Everything works now. Thanks a lot everyone for all your help! -Mark -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Friday, October 23, 2009 11:58 AM To: Mark Vigeant Cc

ZooKeeper Dashboard now available

2009-10-22 Thread Patrick Hunt
I've created a ZooKeeper Dashboard using Django and the zkpython bindings, it's available on github: http://bit.ly/1kjQy0 It currently provides some basic information about the ensemble: * Cluster summary * Individual server detail * Client connection detail * Navigate examine

Re: Cluster Configuration Issues

2009-10-22 Thread Patrick Hunt
guys for all of your immediate help! My machines are currently down for other reasons so I'll be able to try things again, send out the errors I'm getting and such in a day or so (hopefully!) thanks again! -Mark -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent

Re: Cluster Configuration Issues

2009-10-22 Thread Patrick Hunt
bummer, donno what to say, what version of python are you using? works for me in 2.5 and 2.6 (linux jaunty) $ ls LICENSE.txt README.html start.py start.pyc stop.py stop.pyc test3 zoocfg.py zoocfg.pyc ptestREADME.textile start.py.bak start.tmpl stop.py.bak

<    1   2   3   >