FW: [ANN] katta-0.1.0 release - distribute lucene indexes in a grid

2008-09-17 Thread Mahadev Konar
-- Forwarded Message From: Stefan Groschupf [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] Date: Wed, 17 Sep 2008 17:06:19 -0700 To: [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: [ANN] katta-0.1.0 release - distribute lucene indexes in a grid After 5

Re: ZooKeeper Roadmap - 3.1.0 and beyond.

2008-11-18 Thread Mahadev Konar
Hi Krishna, Sorry for the delayed response. The responses are in line. On 11/18/08 12:02 PM, Krishna Sankar (ksankar) [EMAIL PROTECTED] wrote: Have a couple of questions on the proposed multi-tenancy feature (pardon me if they are obvious, as I am slowly getting up to speed): a) First,

Re: ZooKeeper 3.0 Fix Release slated for end of this week.

2008-12-01 Thread Mahadev Konar
Hi Jake, We are in the process of releasing. The release is up for vote. As soon as the release passes, we will put up the release. Here is the email by Pat. On 11/24/08 5:50 PM, Patrick Hunt [EMAIL PROTECTED] wrote: From: Patrick Hunt [EMAIL PROTECTED] Date: November 24, 2008 5:50:24 PM

Re: NullPointerException stopping and starting Zookeeper servers

2008-12-08 Thread Mahadev Konar
Hi Thomas, This looks like a bug. Can you open a jira mentioning what the problem is and how to recreate it? Thanks mahadev On 12/8/08 11:33 AM, Thomas Vinod Johnson [EMAIL PROTECTED] wrote: Hi, I have a replicated zookeeper services consisting of 3 zookeeper (3.0.1) servers all running on

Re: What happens when a server loses all its state?

2008-12-16 Thread Mahadev Konar
Hi Thomas, More generally, is it a safe assumption to make that the ZooKeeper service will maintain all its guarantees if a minority of servers lose persistent state (due to bad disks, etc) and restart at some point in the future? Yes that is true. mahadev Thanks. Mahadev Konar wrote

Re: Any practical limits to the number of znodes?

2008-12-29 Thread Mahadev Konar
Hi Jon, We do not have any limit on the number of znodes in Zookeeper. Its mainly limited by memory, since it keeps the whole namespace in memory. Also the lock mechanism is a good use case for zookeeper. It is listed as one of the recipes for zookeeper.

Re: myid....

2009-01-05 Thread Mahadev Konar
You are right Kevin. The myid file is required for creation. I think we had it well documented. Ill check and see if its not documented well enough and open a jira in case its not. mahadev On 1/5/09 9:58 AM, Kevin Burton bur...@spinn3r.com wrote: Doesn't look like it automatically creates

Re: InterruptedException

2009-01-06 Thread Mahadev Konar
Hi Kevin, The interrupt exception would be thrown in case any other thread tries to interrupt zookeeper threads during a client call (its not really interrupting the server but interrupting the client threads). Its like any synchronous operation that waits throwing an interrupted exception if

Re: event re-issue on reconnect?

2009-01-06 Thread Mahadev Konar
Does onData mean a datawatch? onConnect onData path: /foo, version: 4, data: '2333' onDisconnect onConnect onData path: /foo, version: 4, data: '2333' Are these the sequence of events that you get on the client? mahadev On 1/6/09 5:03 PM, Kevin Burton bur...@spinn3r.com wrote: I have an

Re: Simpler ZooKeeper event interface....

2009-01-07 Thread Mahadev Konar
Hi Vinod, I think what Ben meant was this-- The client will never know of a session expiration until and unless its connected to one of the servers. So the leader cannot demote itself since its connected to one of the servers. It might have lost its session (which all the others except itself

Re: ouch, zookeeper infinite loop

2009-01-07 Thread Mahadev Konar
The version of Jute we use is really an ancient version of recordio ser/deser library in hadoop. We do want to move to some better(versioned/fast/well accepted) ser/deser library. mahadev On 1/7/09 12:08 PM, Kevin Burton bur...@spinn3r.com wrote: Ah... you think it was because it was empty?

Re: Does session expiration only happen during total ensemble failure or network split?

2009-01-07 Thread Mahadev Konar
Kevin, the case you mention, the session is not really expired unless the quorum decides to expire it. So the client assuming that the session expired would be wrong to say. It is possible that as soon as you bring up the servers, the client reconnects with the same session and the session is

Re: A modest proposal for simplifying zookeeper :)

2009-01-09 Thread Mahadev Konar
Hi Kevin, It would be great to have such high level interfaces. It could be something that you could contribute :) . We havent had the bandwidth to provide such interfaces for zookeeper. It would be great to have all such recipes as a part of contrib package of zookeeper. mahadev On 1/9/09

Re: Maximum number of children

2009-01-12 Thread Mahadev Konar
I was going to suggest bucketing with predifined hashes. /root/template/data/hashbucket/hash For the issue raised by Joshua regarding the length of the output from the server -- This is a bug. We seem to allow any number of children ( int) of a node and the getchildren call fails to return the

Delaying 3.2 release by 2 to 3 weeks?

2009-01-15 Thread Mahadev Konar
Hi all, I needed to get quotas in zookeeper 3.2.0 and wanted to see if delaying the release by 2-3 weeks is ok with everyone? Here is the jira for it - http://issues.apache.org/jira/browse/ZOOKEEPER-231 Please respond if you have any issues with the delay. thanks mahadev

Re: Delaying 3.1 release by 2 to 3 weeks?

2009-01-15 Thread Mahadev Konar
That was release 3.1 and not 3.2 :) mahadev On 1/15/09 4:26 PM, Mahadev Konar maha...@yahoo-inc.com wrote: Hi all, I needed to get quotas in zookeeper 3.2.0 and wanted to see if delaying the release by 2-3 weeks is ok with everyone? Here is the jira for it - http://issues.apache.org

Re: Standard redistributable set of primitives?

2009-01-16 Thread Mahadev Konar
Hi Tom, IT does sound like a reasonable idea. If you want to go ahead and implement one of those, we would be happy to help out and get it into Zookeeper. We havent had the bandwidth to put in these recipes in Zookeeper code base. Please go ahead and create a jira if you want to work on it.

Re: Delaying 3.1 release by 2 to 3 weeks?

2009-01-16 Thread Mahadev Konar
From: Mahadev Konar [maha...@yahoo-inc.com] Sent: Thursday, January 15, 2009 4:32 PM To: zookeeper-...@hadoop.apache.org Cc: zookeeper-user@hadoop.apache.org Subject: Re: Delaying 3.1 release by 2 to 3 weeks? That was release 3.1 and not 3.2 :) mahadev On 1/15/09 4:26 PM

Re: Testing Zookeeper

2009-02-10 Thread Mahadev Konar
HI Joshua, Feel free to open a jira and attach a patch. Please take a look at how to contribute: http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute Thanks mahadev On 2/10/09 11:34 AM, Joshua Tuberville joshuatubervi...@eharmony.com wrote: To test our zookeeper usage we built a utility

Re: Dealing with session expired

2009-02-12 Thread Mahadev Konar
Hi Tom, We prefer to discard the zookeeper instance if a session expires. Maintaining a one to one relationship between a client handle and a session makes it much simpler for users to understand the existence and disappearance of ephemeral nodes and watches created by a zookeeper client.

Re: Dealing with session expired

2009-02-12 Thread Mahadev Konar
On Thu, Feb 12, 2009 at 2:12 PM, Mahadev Konar maha...@yahoo-inc.com wrote: Hi Tom, We prefer to discard the zookeeper instance if a session expires. Maintaining a one to one relationship between a client handle and a session makes it much simpler for users to understand the existence

Re: Watcher guarantees

2009-02-13 Thread Mahadev Konar
If client sets a watcher on a znode by doing a getData operation is it guaranteed to get the next change after the value it read, or can a change be missed? The watch is just a notification that the node changed. If you do a getData on the node, you their might have been more updates on the

Re: Recommended session timeout

2009-02-23 Thread Mahadev Konar
Hi Joey, here is a link to information on session timeouts. http://hadoop.apache.org/zookeeper/docs/r3.0.1/zookeeperProgrammers.html#ch_ zkSessions The session timeouts depends on how sensitive you want your application to be. A very low session timeout like (1-2 seconds) might lead to your

Re: Recommended session timeout

2009-02-23 Thread Mahadev Konar
) can be found here: http://hadoop.apache.org/zookeeper/docs/r3.1.0/zookeeperProgrammers.html#ch_z kSessions Patrick Mahadev Konar wrote: Hi Joey,  here is a link to information on session timeouts. http://hadoop.apache.org/zookeeper/docs/r3.0.1/zookeeperProgrammers.html#ch_

Re: Anyone using Zookeeper in AWS (Amazon Cloud)?

2009-02-26 Thread Mahadev Konar
Hi Greg, As for cross datacenters, we have tested zookeeper cross data centers and it works fine. The only thing is that you might have to tweak synclimit and tickTime to a little higher values for Zookeeper. http://hadoop.apache.org/zookeeper/docs/r3.1.0/zookeeperAdmin.html#sc_config uration

Re: How large an ensemble can one build with Zookeeper?

2009-03-03 Thread Mahadev Konar
HI Chad, The maximum number of zookeeper servers we have tested with is 13. Even with 13 the performance starts to degrade very quickly (compared to ensemble of 5 and 7). I am not sure we have the current numbers (we have made 3x or so performance improvements) but with the old number in

Re: Distributed Lock Libraries

2009-03-06 Thread Mahadev Konar
Hi Fernando, Our 3.2 release is focussed more on having such recipes in a clieaner and reliable way. One of the jiras that focusses on this is http://issues.apache.org/jira/browse/ZOOKEEPER-78. We hope to add more of such recipes in 3.2. Thanks mahadev On 3/6/09 2:30 PM, Fernando Padilla

Re: Dynamic addition of servers to Zookeeper cluster

2009-03-13 Thread Mahadev Konar
Hi Raghu, You are right that the cluster configuration is a static one. To deal with this problem, we usually suggest that you change your configs for every server and then re hup all of them at the same time (almost the same time I mean). The clients would lose connections to the servers but

Re: Dynamic addition of servers to Zookeeper cluster

2009-03-13 Thread Mahadev Konar
By re hup I mean restart. Thanks mahadev On 3/13/09 4:54 PM, Mahadev Konar maha...@yahoo-inc.com wrote: Hi Raghu, You are right that the cluster configuration is a static one. To deal with this problem, we usually suggest that you change your configs for every server and then re hup all

Re: Semantics of ConnectionLoss exception

2009-03-25 Thread Mahadev Konar
Hi Nitay, - Does this event happening mean my ephemeral nodes will go away? No. the client will try connecting to other servers and if its not able to reconnect to the servers within the remaining session timeout. If the client is not able to connect within the remaining session timeout, the

Re: Semantics of ConnectionLoss exception

2009-03-26 Thread Mahadev Konar
Isn't it the case that the client won't get session expired until it's able to connect to a server, right? So what might happen is that the client loses connection to the server, the server eventually expires the client and deletes ephemerals (notifying all watchers) but the client won't

Re: Semantics of ConnectionLoss exception

2009-03-26 Thread Mahadev Konar
ephemeral nodes are gone? Why not deliver the Session Expired event on the client side after the right time has passed without communication to any server? On Thu, Mar 26, 2009 at 10:58 AM, Mahadev Konar maha...@yahoo-inc.comwrote: Isn't it the case that the client won't get session expired

Re: Some one send me some demo of programming with C client API for Zookeeper

2009-04-16 Thread Mahadev Konar
Please take a look at src/c/src/cli.c for some examples on zookeeper c client usage. Also you can see the test cases. Also http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperProgrammers.html Will give you some exmaple code for c clients. mahadev On 4/16/09 2:30 AM, Qian Ye

Re: Server-client connection timeout

2009-04-21 Thread Mahadev Konar
Hi raghu, http://wiki.apache.org/hadoop/ZooKeeper/FAQ Explains on what timeouts mean for a zookeeper client. A timeout does not mean a closed session. The client will reconnect to another server and then renew the session. A closed session will make the zoookeeper handle invalid. Hope this

Re: ZooKeeper's Atomic Broadcast Leader Election Algorithms

2009-04-21 Thread Mahadev Konar
Hi Jason, You should be a able to get some idea from the set of presentations at http://wiki.apache.org/hadoop/ZooKeeper/ZooKeeperPresentations. Also, please use zookeeper-...@hadoop.apache.org for questions which are not related to zookeeper users. mahadev On 4/21/09 2:32 PM, Jason Dusek

Re: Does anyone have done some strict performance testing on current Zookeeper?

2009-04-23 Thread Mahadev Konar
Hi Qian, I can give you an example of one of our systems that uses zookeeper (crawling for our Yahoo! search engine). It has of the order of 4K clients - expecting to grow to 8K to 12K. Their write load is around 100 writes/sec (this is pretty low) and 4K reads/sec (max reads) with an ensemble

Re: Multiple ZooKeeper client instances

2009-04-24 Thread Mahadev Konar
HI Satish, A zookeeper client usually has a very small footprint for memory and cpu. The mutithreaded version of zookeeper client creates an internal thread to do the io and callbacks. I would suggest using the same zookeeper client across the objects to have less number of threads in your

Re: Dynamic server addition/deletion

2009-05-01 Thread Mahadev Konar
Hi Raghu and Ted, Theire is already an open jira on this -- http://issues.apache.org/jira/browse/ZOOKEEPER-107 You can go through the suggestions on it and can continue the discussion on the jira. Please feel free to add your ideas to the jira. Also, I don't think anyone is working on it (to

Re: Moving ZooKeeper Servers

2009-05-04 Thread Mahadev Konar
Hi Satish, Is the re generation of state in production something that is not acceptable? Copying over the whole datadir and datalogdir as it is maintaining the dirctory structure would be necessary. Also, in general this is a bad idea (just to warn you) since you would have to be careful with

Re: Moving ZooKeeper Servers

2009-05-06 Thread Mahadev Konar
from the pre-production cluster and write to the production one is much more sound. If you can't do that, you may need to rethink your processes since they are likely to be delicate for other reasons as well. On Mon, May 4, 2009 at 2:35 PM, Mahadev Konar maha...@yahoo-inc.com wrote: So

Re: Some thoughts on Zookeeper after using it for a while in the CXF/DOSGi subproject

2009-05-29 Thread Mahadev Konar
Hi David, I second this. If you folks want to host at http://scala-tools.org we'd be happy to host this non-Scala but super mega interesting and valuable project. We'll even do builds on our hudson server. We have had plans to publish our release on maven repos but havent had the time to

Re: ZooKeeper heavy CPU utilisation

2009-06-02 Thread Mahadev Konar
Hi Satish, Can you attach this trace to a jira? Please open one for this. Also, can you do the following - For all the threads for the zookeeper server you are seeing the problem on, Can you do an strace on all the threads and see which thread is spinning? Also, can you upload the configs of

Re: Errors during shutdown/startup of ZooKeeper

2009-06-02 Thread Mahadev Konar
Hi Nitay, This is not an error but should be a warning. I have opened up a jira for it. http://issues.apache.org/jira/browse/ZOOKEEPER-428 The message just says that a client is connecting to a server that is behind that a server is was connected to earlier. The log should be warn and not

Re: Win32 as a production platform

2009-06-04 Thread Mahadev Konar
Hi Marc, The only thing missing would be testing and support. We do most of our testing on linux boxes and for the same reason its easy for us to support the platforms that we use. We do not have access to windows boxes to test and (therefore) support windows as a suggested production platform.

Re: Question about ACL

2009-06-07 Thread Mahadev Konar
I think my last email bounced -- Hi Qian, I think we lack a lot of documentation on ACL’s and how to use your own authentication schemes. 3.2 should have some of it. Zookeeper allows you to add you own authentication schemes. You should be able to write you own authentication scheme like the

Re: Newbie Questions

2009-06-07 Thread Mahadev Konar
objects using ZooKeeper. Is this the recommended approach? Is there some other more elegant way to do this? Satish On Sun, Jun 7, 2009 at 11:58 AM, Mahadev Konar maha...@yahoo-inc.comwrote: HI Grant, I agree with Ted but just to elaborate a little more. Its good to have a single

Re: General Question about Zookeeper

2009-06-25 Thread Mahadev Konar
Hi Harold, As Henry mentioned, what acl's provide you is preventing access to znodes. If someone has access to zookeeper's data stored on zookeeper's server machines, they should be able to resconstruct the data and read it (using zookeeper deserialization code). I am not sure what kind of

Re: General Question about Zookeeper

2009-06-25 Thread Mahadev Konar
Hi Harold, Let me explain the whole concept of ZooKeeper Acls. 1) Zookeeper servers are run using some user id say X 2) zookeeper client use ZooKeeper client libaryr to create zookeeper nodes on zookeeper servers. They could be running as user id C. They can provide acl's to create such nodes

Re: Some questions about Zookeeper 3.2.0

2009-06-30 Thread Mahadev Konar
Hi Qian, Sorry for the delayed response. ZooKeeper guarantees that their is backwards compatibility with minor releases. So, 3.2* is backwards compatible with 3.1*. So, what Ted mentions below related to rolling upgrade should work. Hope this helps. Thanks Mahadev Ted Dunning: A rolling

Re: Help to compile Zookeeper C API on a old system

2009-07-06 Thread Mahadev Konar
Hi Qian, What issues do you face? I have never tried compiling with the configuration below, but I could give it a try in my free time to see if I can get it to compile. mahadev On 7/6/09 7:37 AM, Qian Ye yeqian@gmail.com wrote: Hi all: I'm writing to ask you to do me a favor. It's

Re: zookeeper on ec2

2009-07-06 Thread Mahadev Konar
Hi David, Answers in line: On 7/6/09 4:45 AM, David Graf david.g...@28msec.com wrote: Hello I wanna set up a zookeeper ensemble on amazon's ec2 service. In my system, zookeeper is used to run a locking service and to generate unique id's. Currently, for testing purposes, I am only

Re: Help to compile Zookeeper C API on a old system

2009-07-06 Thread Mahadev Konar
, 2009 at 2:22 AM, Mahadev Konar maha...@yahoo-inc.com wrote: Hi Qian, What issues do you face? I have never tried compiling with the configuration below, but I could give it a try in my free time to see if I can get it to compile. mahadev On 7/6/09 7:37 AM, Qian Ye yeqian

Re: Question about the sequential flag on create.

2009-07-13 Thread Mahadev Konar
Internally children of a node are not guranteed to be stored as sorted via their names. The counter that you mention is just a version number on the parent that is used during creation of children of a node that are created with a Sequential flag. It has nothing to do with how the children of a

Re: Instantiating HashSet for DataNode?

2009-07-14 Thread Mahadev Konar
Hi Erik, I am not sure if that would a considerable opitmization but even if you wanted to do it, it would be much more than just adding a check in the constructor (the serialization/deserialization would need to have specialized code). Right now all the datanodes are treated equally for

Re: zkCleanup.sh is buggy

2009-07-17 Thread Mahadev Konar
Hi Fernando, Please do file a jira ( http://issues.apache.org/jira/browse/ZOOKEEPER ) and the patch below as an attachment to the created jira. Here is how to contribute: http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute thnaks Mahadev On 7/17/09 11:59 AM, Fernando Padilla

Re: Queue code

2009-07-17 Thread Mahadev Konar
Also are there any performance numbers of zookeeeper based queues. How does it compare with JMS. thanks Kishore G Hi Kishore, We do not have any performance number fr queues on zookeeper. I think you can get a rough idea of those numbers from your usage of zookeeper (number of reads/writes

Re: Leader Elections

2009-07-20 Thread Mahadev Konar
Both of the options that Scott mentioned are quite interesting. Quite a few of our users are interested in these two features. I think for 2, we should be able to use observers with a subscription to the master cluster with interested in a special subtree. That avoids too much of cross talk.

Bug in 3.2 release.

2009-07-23 Thread Mahadev Konar
Hi folks, We just discovered a bug in 3.2 release http://issues.apache.org/jira/browse/ZOOKEEPER-484. This bug will affect your clients whenever they switch zookeeper servers - from a zookeeper server that is a follower to a server that is leader. We should have a fix out by next week in

Re: bad svn url : test-patch

2009-07-30 Thread Mahadev Konar
Hi Todd, Yes this happens with the branch 3.2. The test-patch link is broken becasuse of the hadoop split. This file is used for hudson test environment. It isnt used anywhere else, so the svn co otherwise should be fine. We should fix it anyways. Thanks mahadev On 7/30/09 2:57 PM, Todd

Re: c client error message with chroot

2009-08-03 Thread Mahadev Konar
This looks like a bug. Does this happen without doing any reads/writes using the zookeeper handle? Please do open a jira for this. Thanks mahadev On 8/2/09 10:53 PM, Michi Mutsuzaki mi...@cs.stanford.edu wrote: Hello, I'm doing something like this (using zookeeper-3.2.0): zhandle_t*

Re: question about watcher

2009-08-05 Thread Mahadev Konar
Hi Qian, There isnt any such api. We have been thinking abt adding an api on cancelling a cleints watches. We have been thinking about adding a proc filesystem wherein a cleintt will have a list of all the watches. This data can be used to know which clients are watching what znode, but this has

Re: c client - problem with failover

2009-08-28 Thread Mahadev Konar
Hi Lukasz, Comments in line: On 8/28/09 8:24 AM, Łukasz Osipiuk luk...@osipiuk.net wrote: Hi! I my name is Łukasz Osipiuk. I am working for one of major Polish Internet companies. In one of our projects we are intensively using Zookeeper as distributed locking system. We implemented

Re: Runtime Interrogation of the Ensemble

2009-08-31 Thread Mahadev Konar
Hi Todd, You can use jmx to to find such information. Also you can just do this Echo stat | nc localhost clientport To get status from the zookeeper servers. This is all documented in the forrest docs at http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperAdmin.html Hope this helps.

Re: zookeeper on ec2

2009-09-01 Thread Mahadev Konar
Hi Satish, Connectionloss is a little trickier than just retrying blindly. Please read the following sections on this - http://wiki.apache.org/hadoop/ZooKeeper/ErrorHandling And the programmers guide: http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperProgrammers.html To learn more

Re: Infinite ping after calling setData()

2009-09-15 Thread Mahadev Konar
they to long to add to email at this time. Unfortunately I am having completely different issues now with the servers not shutting down. When I get past that and if I run into this issue again I will give more details. Thanks. -Original Message- From: Mahadev Konar

Re: ACL question w/ Zookeeper 3.1.1

2009-09-17 Thread Mahadev Konar
HI todd, From what I understand, you are sayin that a creator_all_acl does not work with auth? I tried the following with CREATOR_ALL_ACL and it seemed to work for me... import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.Watcher;

Re: Start problem of Running Replicated ZooKeeper

2009-09-23 Thread Mahadev Konar
Hi Le, Is there some chance of the these servers not being able to talk to each other? IS the zookeeper prcoess running on debian-1? What error do you see on debian-1? The connection refused error suggests that debian-0 is not able to talk to debian-1 machine. Thanks mahadev On 9/23/09

Re: specifying the location of zookeeper.log

2009-10-16 Thread Mahadev Konar
Hi Leonard, You should be able to set the ZOO_LOG_DIR as an environment variable to get a different log directory. I think you are using bin/zkServer.sh to start the server? Also, please open a jira for this. It would be good to fix the documentation for this. Thanks mahadev On 10/16/09

Re: specifying the location of zookeeper.log

2009-10-16 Thread Mahadev Konar
} \ -cp $CLASSPATH $JVMFLAGS $ZOOMAIN $ZOOCFG I double checked by echo'ing the value of ZOO_LOG_DIR just before the java command. It's set correctly ... but it has no effect on the location of zookeeper.log :-( Leonard On 10/16/09 11:08 AM, Mahadev Konar maha...@yahoo-inc.com wrote

Re: specifying the location of zookeeper.log

2009-10-16 Thread Mahadev Konar
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout log4j.appender.FILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%c...@%l] - %m%n Will let you log to the output directory $dir. Hope that helps! mahadev On 10/16/09 11:35 AM, Mahadev Konar maha...@yahoo-inc.com wrote: Hi Leonard, Looks

Re: specifying the location of zookeeper.log

2009-10-16 Thread Mahadev Konar
and forgotten that I'd done so. Thanks for your attention. Leonard On 10/16/09 11:35 AM, Mahadev Konar maha...@yahoo-inc.com wrote: Hi Leonard, Looks like you are right. bin/zkServer.sh just logs the output to console, so you should be able to redirect to any file you want. No? Anyways

Re: Cluster Configuration Issues

2009-10-20 Thread Mahadev Konar
HI Mark, ZooKeeper does not create the myid file in the data directory. Looking at the config file it looks like it is missing the quorum configuration for other servers. Please take alook at http://hadoop.apache.org/zookeeper/docs/r3.2.1/zookeeperAdmin.html#sc_zkMuli tServerSetup You will

Re: Restarting a single zookeeper Server on the same port within the process

2009-10-22 Thread Mahadev Konar
Hi Siddharth, Usually the time of releasing the port is dependent on the OS. So you can try sleeping a few more seconds to see if the port has been released or it .. Or just poll on the port to see if its in use or not There isnt an easier way to restart on the same port. mahadev On

Re: zookeeper viewer

2009-10-24 Thread Mahadev Konar
Hi Hamoun, Can you please mention which link is broken? Are you a looking for a zookeeper tree browser? Pat created a dashboard for zookeeper at github. Below is the link: http://github.com/phunt/zookeeper_dashboard Also, there is an open jira for a zookeeper browser which you can try

Bugfix release 3.2.2

2009-10-30 Thread Mahadev Konar
Hi all, We are planning to make a bugfix release 3.2.2 which will include a critical bugfix in the c client code. The jira is ZOOKEEPER-562, http://issues.apache.org/jira/browse/ZOOKEEPER-562. If you would like some fix to be considered for this bugfix release please feel free to post on the

Re: zkfuse

2009-11-24 Thread Mahadev Konar
Hi Maarten, zkfuse does not have any support for acls. We havent had much time to focus on zkfuse. Create/read/write/delete/ls are all supported. It was built mostly for infrequent updates and more of a browsing interface on filesystem. I don't think zkfuse is being used in production anywhere.

Re: Starting Zookeeper on Amazon EC2

2009-12-09 Thread Mahadev Konar
Hi, Can you try this? bin/zkCli.sh 127.0.0.1:2181 The -server command was added later as far as I remember. Thanks mahadev On 12/9/09 9:05 AM, Something Something mailinglist...@gmail.com wrote: I am trying to start ZooKeeper on an EC2 instance. Here's what I did: 1) Downloaded

Re: Fetching sequential children

2009-12-23 Thread Mahadev Konar
Hi ohad, there isnt a way to get a selected set of children from the servers. So you will have to get all of them and filter out the unwanted ones. Also, what Steve suggested in the other email might be useful for you. Thanks mahadev On 12/23/09 12:29 AM, Ohad Ben Porat o...@outbrain.com

Re: Killing a zookeeper server

2010-01-13 Thread Mahadev Konar
Hi Adam, That seems fair to file as an improvement. Running 'stat' did return the right stats right? Saying the servers werent able to elect a leader? mahadev On 1/13/10 11:52 AM, Adam Rosien a...@rosien.net wrote: On a related note, it was initially confusing to me that the server

Re: question about locking in java singleton class

2010-01-14 Thread Mahadev Konar
Hi Jaakko, The lock recipe has already been implemented in zookeeper under src/recipes/lock (version 3.* I think). It has code to deal with connectionloss as well. I would suggest that you use the recipe. You can file jira's in case you see some shortcomings/bugs in the code. Thanks mahadev

Re: Question regarding Membership Election

2010-01-14 Thread Mahadev Konar
Hi Vijay, Unfortunately you wont be able to keep running the observer in the other DC if the quorum in the DC 1 is dead. Most of the folks we have talked to also want to avoid voiting across colos. They usually run two instances of Zookeeper in 2 DC's and copy state of zookeeper (using a bridge)

Re: Question regarding Membership Election

2010-01-14 Thread Mahadev Konar
for the bridging process? Regards, /VJ On Thu, Jan 14, 2010 at 12:38 PM, Mahadev Konar maha...@yahoo-inc.comwrote: Hi Vijay, Unfortunately you wont be able to keep running the observer in the other DC if the quorum in the DC 1 is dead. Most of the folks we have talked to also want

Re: Namespace partitioning ?

2010-01-14 Thread Mahadev Konar
Hi kay, the namespace partitioning in zookeeper has been on a back burner for a long time. There isnt any jira open on it. There had been some discussions on this but no real work. Flavio/Ben have had this on there minds for a while but no real work/proposal is out yet. May I know is this

Re: [jira] Commented: (MAHOUT-238) Further Dependency Cleanup

2010-01-22 Thread Mahadev Konar
Unfortunately no.. We are planning to deploy 3.3 as the first version on maven repo. Thanks mahadev On 1/22/10 12:58 PM, Ted Dunning ted.dunn...@gmail.com wrote: Is ZK 3.2.2 in a maven repository somewhere? -- Forwarded message -- From: Drew Farris drew.far...@gmail.com

Re: Server exception when closing session

2010-01-22 Thread Mahadev Konar
Hi Josh, This warning is not of any concern. Just a quick question, is there any reason for you to runn the server on a DEBUG level? Thanks mahadev On 1/22/10 5:19 PM, Josh Scheid jsch...@velocetechnologies.com wrote: Is it normal for client session close() to cause a server exception?

Re: Server exception when closing session

2010-01-22 Thread Mahadev Konar
at: http://wiki.apache.org/hadoop/ZooKeeper/ServiceLatencyOverview You can take a look at that as well and see what the expected performance should be for your workload. Thanks mahadev On 1/22/10 5:40 PM, Josh Scheid jsch...@velocetechnologies.com wrote: On Fri, Jan 22, 2010 at 17:22, Mahadev

Re: Q about ZK internal: how commit is being remembered

2010-01-28 Thread Mahadev Konar
Qian, ZooKeeper gurantees that if a client sees some transaction response, then it will persist but the one's that a client does not see might be discarded or committed. So in case a quorum does not log the transaction, there might be a case wherein a zookeeper server which does not have the

ZOOKEEPER-22 and release 3.3

2010-02-02 Thread Mahadev Konar
Hi all, I had been working on zookeeper-22 and found out that it needs quite a few extensive changes. We will need to do some memory measurements to see if it has any memory impacts or not. Since we are targetting 3.3 release for early march, ZOOKEEPER-22 would be hard to get into 3.3. I am

Re: When session expired event fired?

2010-02-08 Thread Mahadev Konar
Hi, a zookeeper client does not expire a session until and unless it is able to connect to one of the servers. In your case if you kill all the servers, the client is not able to connect to any of the servers and will keep trying to connect to the three servers. It cannot expire a session on its

Re: Ordering guarantees for async callbacks vs watchers

2010-02-10 Thread Mahadev Konar
Hi martin, a call like getchildren(final String path, Watcher watcher, ChildrenCallback cb, Object ctx) Means that set a watch on this node for any further changes on the server. A client will see the response to getchildren data before the above watch is fired. Hope that helps. Thanks

Re: How do I get added child znode?

2010-02-16 Thread Mahadev Konar
Hi Kim, The zookeeper api does not provide an api to get the znode that was added or deleted. You will have to compare the last set of children and new set of children to see which one was added or deleted. Thanks mahadev On 2/16/10 5:47 AM, neptune opennept...@gmail.com wrote: Hi all, I'm

Re: ZooKeeper packages for Ubuntu

2010-02-16 Thread Mahadev Konar
Great to hear this mahadev On 2/16/10 11:55 AM, Gustavo Niemeyer gust...@niemeyer.net wrote: Hello everyone, Thanks to Matthias Klose and Thierry Carrez, we've got ZooKeeper packaged for Ubuntu: https://launchpad.net/~ttx/+archive/ppa This is a Personal Package Archive at

Re: Bit of help debugging a TIMED OUT session please

2010-02-22 Thread Mahadev Konar
at 11:26 AM, Mahadev Konar maha...@yahoo-inc.com wrote: I also looked at the logs. Ted might have a point. It does look like that zookeeper server's are doing fine (though as ted mentions the skew is a little concerning, though that might be due to very few packets served by the first server). Other

Re: how to lock one-of-many ?

2010-02-23 Thread Mahadev Konar
Hi Martin, How about this- you have resources in the a directory (say /locks) each process which needs to lock, lists all the children of this directory and then creates an ephemeral node called /locks/resource1/lock depending on which resource it wants to lock. This ephemeral node will

Re: is there a good pattern for leases ?

2010-02-24 Thread Mahadev Konar
waite@googlemail.com wrote: Hi Mahadev, That is interesting. All I need to do is hold the connection for the required time of a session that created an ephemeral node. Zookeeper is an interesting tool. Thanks again, Martin On 24 February 2010 17:00, Mahadev Konar maha...@yahoo

Re: how to lock one-of-many ?

2010-02-24 Thread Mahadev Konar
Hi martin, Currently you cannot access the server that the client is connected to. This was fixed in this jira http://issues.apache.org/jira/browse/ZOOKEEPER-544 But again this does not tell you if you are connected to the primary or the other followers. So you will anyway have to do some

Re: zookeeper utils

2010-03-02 Thread Mahadev Konar
Hi David, There is an implementation for locks and queues in src/recipes. The documentation residres in src/recipes/{lock/queue}/README.txt. Thanks mahadev On 3/2/10 1:04 PM, David Rosenstrauch dar...@darose.net wrote: Was reading through the zookeeper docs on the web - specifically the

Re: Managing multi-site clusters with Zookeeper

2010-03-07 Thread Mahadev Konar
Hi Martin, As Ted rightly mentions that ZooKeeper usually is run within a colo because of the low latency requirements of applications that it supports. Its definitely reasnoble to use it in a multi data center environments but you should realize the implications of it. The high latency/low

Re: Managing multi-site clusters with Zookeeper

2010-03-07 Thread Mahadev Konar
21:43, Mahadev Konar maha...@yahoo-inc.com wrote: Hi Martin, As Ted rightly mentions that ZooKeeper usually is run within a colo because of the low latency requirements of applications that it supports. Its definitely reasnoble to use it in a multi data center environments but you should

Re: Managing multi-site clusters with Zookeeper

2010-03-08 Thread Mahadev Konar
to better than zk does and still maintain correctness. Do note that you can, probalbly bias client to use a local server. That should make things more efficient. Sent from my iPhone On Mar 7, 2010, at 3:00 PM, Mahadev Konar maha...@yahoo-inc.com wrote: The inter-site links

  1   2   >