Re: need for more conditional write support

are you guys going to put a limit on the size of the updates? can someone do an update over 50 znodes where data value is 500K, for example? if there is a failure during the update, is it okay for just a subset of the znodes to be updated? ben On 12/20/2010 06:56 PM, Qian Ye wrote: Hi all,

Re: Extracting Zab from Zookeeper

it's funny, i was just thinking about this yesterday. no one is working on it, so it is still open. it is a non-trivial piece of work, but i'd be willing to give guidance if you are interested in it. ben ps - btw, this is definitely a d...@zookeeper.apache.org discussion :) On 01/18/2011

Re: Why my session does not expire in this situation?

yeah, that will not work. when you restart zookeeper, it resets the timeouts. to provoke a session expiration you should reopen the session with a different zhandle and immediately close it. ben On 02/03/2011 06:38 AM, Ride Shoot Speak wrote: Hi, I tried to provoke an expired session by

Re: Using ZK for real-time group membership notification

in a distributed setting such an answer is impossible. especially given the theory of relativity and the speed of light. a machine may fail right after sending a heart beat or another may come online right after sending a report. even if zookeeper could provide this you would still have thread

Re: Serious problem processing hearbeat on login stampede

chang, if the problem is on client startup, then it isn't the heartbeat stamped, it is session establishment. the heartbeats are very light weight, so i can't imagine them causing any issues. the two key issues we need to know are: 1) the version of the server you are running, and 2) if you are

Re: Serious problem processing hearbeat on login stampede

, we'll have to discuss on JIRA ;) Thank you. Chang 2011. 4. 15., 오전 2:59, Benjamin Reed 작성: chang, if the problem is on client startup, then it isn't the heartbeat stamped, it is session establishment. the heartbeats are very light weight, so i can't imagine them causing any issues

they are making a zookeeper movie

are we the first apache project that has a movie? http://www.zookeeper-movie.com/

Re: they are making a zookeeper movie

yeah that part is going to be about how to make hadoop use zookeeper :) ben On Wed, May 25, 2011 at 5:37 PM, Ted Dunning ted.dunn...@gmail.com wrote: It has an elephant in it so I think that it isn't JUST a ZK movie. On Wed, May 25, 2011 at 5:27 PM, Benjamin Reed br...@apache.org wrote

Re: Session expiration mechanism

the leader tracks the session and tells the followers to close the session. the followers only summarize heartbeats for the leaders. we need to replicate the session open and closes so that they can be recovered if the leader dies. ben On Tue, Jun 7, 2011 at 2:19 PM, Ariel Weisberg

Re: Mounting a remote Zookeeper

this is a small nit, but i think the partition proposal works a bit more like a mount point than your proposal. when you mount a file system, the mount isn't transparent. two mounted file systems can have files with the same inode number, for example. you also can't do some things like a rename

Re: Debian packager orphaning ZooKeeper

hey i've put up a wiki to collect some of these issues from thomas and others: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ImplementationCriticisms if people have others to add, please feel free to update. ben On Fri, Jun 10, 2011 at 11:55 AM, Patrick Hunt ph...@apache.org wrote: Hi

Re: help on Zookeeper code walk through?

if you are running with multiple servers, it is the leader that declares sessions dead, so the leader will call killSession(). the followers track the liveness of the clients with pings and will periodically send liveness summaries to the leader. see camille's email the specific classes to look

Re: zk keeps disconnecting and reconnecting

i have been wondering about 3.3.4. there are so many great bugs that were fixed in 3.4.0 that it isn't clear what we should put into 3.3.4 or if we should even do it. the chroot bug does seem like a good one to do a 3.3.4 release for. ben On Mon, Aug 29, 2011 at 12:45 PM, Mahadev Konar

Re: pzxid field in stat structure

the pzxid from the users point of view is purely informational. internally we use it to figure out if we need to trigger watches on reconnect. both the pzxid and the cversion are updated when adding or removing children. ben On Tue, Oct 18, 2011 at 10:32 AM, Ted Dunning ted.dunn...@gmail.com

Re: Distributed ZooKeeper cluster design

i agree with camille that mirror breaks a lot of the basic guarantees that you use from zookeeper. with that caveat in mind, there is a patch that enables mirroring: ZOOKEEPER-892. ben On Tue, Dec 13, 2011 at 8:24 AM, Camille Fournier cami...@apache.org wrote: I have to strongly disagree with

Re: connection lost

which version of the server are you using. this sounds like the linger bug that was fixed a while back. ben On Sun, Dec 18, 2011 at 6:22 PM, Camille Fournier cami...@apache.org wrote: That is not what we would expect from the cluster. What do the server logs show? On Sun, Dec 18, 2011 at

Re: Causing ZSESSIONEXPIRED

one easy way is to terminate the session using JMX. ben On Thu, Jan 26, 2012 at 2:00 PM, Jonathan Simms slyp...@gmail.com wrote: Hello all, I'm trying to debug how my app handles an expired session, and i'm having trouble understanding how to cause this error to occur. I'd like to test my

Re: Deployment planning question

what do you want to happen if both datacenters are up, but there is a partition so that they cannot communicate with each other? answering that question may get you closer to an answer. i think your two main options are to: 1) designate a data center that you require to be up for things to work.

Re: Deployment planning question

of partition. On Fri, Feb 3, 2012 at 2:05 PM, Benjamin Reed br...@apache.org wrote: what do you want to happen if both datacenters are up, but there is a partition so that they cannot communicate with each other? answering that question may get you closer to an answer. i think your two main options

Re: use cases for asynchronous API

give me any clue on that? Thanks in advance Pierre Louis Aublin [1] ZooKeeper: Wait-free Coordination for Internet-scale Systems, Patrick Hunt and Mahadev Konar, /Yahoo! Grid;/ Flavio P. Junqueira and Benjamin Reed, /Yahoo! Research/

[DISCUSS] longer release candidate time for higher quality releases

at the meetup yesterday we discussed/debated ways to make sure we have high quality releases. we wanted to put the discussion on the mailing list to see if we should make a policy change. MOTIVATION: it is clear that to have high quality releases, we need quality testing. zookeeper is such a

Re: Getting confused with the recipe for lock

sorry to jump in the middle, but i thought i'd point out a couple of things. at the heart of ZK is Zab, which is an atomic broadcast protocol (it actually has stronger guarantees than just atomic broadcast: it also guarantees primary order). updates go through this protocol which gives us

Re: Zookeeper does not clean up deleted nodes

are you looking at the leader or the follower? the leader keeps the last few transactions in memory to speed up syncing with new followers. that might be what you are seeing. On Mon, Apr 8, 2013 at 3:32 AM, Mathias Hodler mathias.hod...@gmail.comwrote: Hi, I made some tests and it seems like

Re: Zookeeper does not clean up deleted nodes

larger files (about 1MB) the only option is to increase the heap space? I know that zookeeper is designed for small files, but I'm using zookeeper with solr and solr stores all the index configuration with large dictionaries in zookeeper. 2013/4/8 Benjamin Reed br...@apache.org are you looking

Re: Zookeeper born date

if i remember correctly, we released on sourceforge about a year after we started it inside of yahoo. we had an open source tar ball release on the yahoo research website earlier in 2007. if someone still has access to yahoo internal svn they could grab the dates from there. ben On Thu, Jun 13,

Re: avoiding the risk of starting ZooKeeper servers in the same ensemble with different transaction logs

when you say inconsistent transaction log, are you talking about a transaction log from a different ensemble instance? for example, you ran zookeeper and did some things. then you reset the all the servers but one and restarted everything. ben On Tue, Sep 24, 2013 at 5:45 AM, German Blanco

Re: Sync question

the read will be up to date with respect to the sync: the data you read is guaranteed to reflect at least the state of the system when the sync executed. even if you only have one server, so it is always perfectly up-to-date, you can do a read and the server will return the latest value, but by

Re: Staggered start of nodes in an ensemble

and what did the server logs say. (probably over and over again) when server 1 and 2 were up? On Mon, Nov 11, 2013 at 9:09 PM, Arindam Mukherjee arindam.muker...@gmail.com wrote: On Tue, Nov 12, 2013 at 8:43 AM, kishore g g.kish...@gmail.com wrote: can you provide the configurations

Re: Disqualify a node from leader election

camille really has the right solution. you have to let it become the leader and then kill it. here is why: lets says you have servers: A, B, C, D, and E and A is the node that you don't want to be the leader. let's also say that C is a leader and commits transaction x on A, B, and C but before D

Re: zookeeper watch limitation

samoi...@gmail.com wrote: hi, I am reading the book Zookeeper by Flavio Junqueira and Benjamin Reed. And I am now concerned if Zookeeper right tool for our scenario: configuration management. We have ~2000 servers that expected to subscribe to znode change notification: current version number

Re: entire cluster dies with EOFException

On Fri, Jul 4, 2014 at 10:35 PM, Aaron Zimmerman azimmer...@sproutsocial.com wrote: Thanks for getting back to me. Jordan, I don't think we are doing any large nodes or thousands of children. We are using zookeeper for storm and service discovery, so things are pretty modest. Camille,

Re: entire cluster dies with EOFException

any chance you are running out of disk space? On Sun, Jul 6, 2014 at 6:49 AM, Benjamin Reed br...@apache.org wrote: On Fri, Jul 4, 2014 at 10:35 PM, Aaron Zimmerman azimmer...@sproutsocial.com wrote: Thanks for getting back to me. Jordan, I don't think we are doing any large nodes

Re: entire cluster dies with EOFException

the log files seem very large to me. is the snap count really large? or do you have a lot of data in each znode? you might try reducing the snap count to make the log files smaller. On Jul 6, 2014 2:27 PM, Flavio Junqueira fpjunque...@yahoo.com.invalid wrote: But what is it that was causing

Re: Issue with NettyServerCnxn.java

if i remember correctly the case in sendResponse where it is catching the IOException is due to the fact that we are opportunistically trying to send something on a non-blocking channel. if it works, ok, but if we can't send because we are blocked then we will just send later. in the case of

Re: Issue with NettyServerCnxn.java

s. > I will submit a JIRA on this (hopefully today). Either we should not > bubble up any exception by IOException or ZK server should be stopped, as > it is really hard to figure out without turning on tracing what really > happened. > ThanksYuliya > > From: Benjamin Reed <

Re: november meetup at facebook

we are finding another date :) On Wed, Sep 28, 2016 at 9:17 AM, Benjamin Reed <br...@apache.org> wrote: > oh haha i finally get raul's comment! > > On Wed, Sep 28, 2016 at 8:51 AM, David Brower <david.bro...@oracle.com> > wrote: > >> That's election day,

Re: tracing client request/responses

yeah i tried zk-dump, but it ended up missing way too many packets. On Wed, Oct 5, 2016 at 5:50 PM, Raúl Gutiérrez Segalés <r...@itevenworks.net> wrote: > On 5 October 2016 at 17:39, Benjamin Reed <br...@apache.org> wrote: > > > anyone know how to turn on tracing to g

tracing client request/responses

anyone know how to turn on tracing to get a file that i can run through the TraceFormatter? i see the 4 letter works to set tracing masks, but i can't seem to find the switch to actually turn on tracing. thanx ben

Re: Exception using zkCli

it sounds like your host/port may be incorrect. just to be sure you might try using the -server flag: zkCli.sh -server zkhost:zkport (where zkhost is the hostname of your zkserver and zkport is the port number) you can also check if the server is running by using nc (or telnet): echo stat | nc

Re: november meetup at facebook

oh haha i finally get raul's comment! On Wed, Sep 28, 2016 at 8:51 AM, David Brower <david.bro...@oracle.com> wrote: > That's election day, which might create complications. > > -dB > > > On 9/27/2016 10:05 PM, Benjamin Reed wrote: > >> facebook would like

november meetup at facebook

facebook would like to host a zookeeper meetup in our offices in menlo park, ca on november 8th (a tuesday). before sending out an official invitation with details about logistics, i thought i would first do a quick date check and make sure that there isn't a big scheduling conflict that we didn't

november meetup at facebook (take 2)

facebook would like to host a zookeeper meetup in our offices in menlo park, ca on november 17th (a thursday). before sending out an official invitation with details about logistics, i thought i would first do a quick date check and make sure that there isn't a big scheduling conflict that we

Re: ZooKeeper clients does not handle new error codes properly

did we bump the protocol version when we added the new errors? the server could do the conversion when it responds to older clients. On Mon, Oct 3, 2016 at 3:05 AM, Flavio Junqueira wrote: > Hi Arshad, > > It makes sense to me. What if we convert unknown server errors to >

Re: november meetup at facebook (take 2)

ep 30, 2016 2:26 PM, "Marshall McMullen" < > > marshall.mcmul...@gmail.com> > > wrote: > > +1. I would love to attend along with a few of my coworkers and > > this > > date > > works for us. > > On Fri, Sep 30, 2016 at 3:10 PM, Benjamin Reed <br

november 17, 2016 zookeeper meetup at facebook

we would like to invite you all to a zookeeper meetup at the facebook campus on november 17, 2016 starting at 5pm. please use the following link to the facebook event to get details and rsvp. https://www.facebook.com/events/1228722650504268 we need you to rsvp to make sure we plan appropriately

november 17 zk meetup (additional registration info required)

sorry for the late notice, but we have found out from security that we will need an email and phone number to get you in without the usual visitor NDA. if you are coming, please send an email to acon...@fb.com with that information by tonight. if you forget to do this or decide to come at the

Re: november 17 zk meetup (additional registration info required)

-Original Message----- > From: Benjamin Reed [mailto:br...@apache.org] > Sent: 16 November 2016 19:07 > To: user@zookeeper.apache.org; DevZooKeeper <d...@zookeeper.apache.org> > Subject: november 17 zk meetup (additional registration info required) > > sorry for the lat

november 17 zk meetup reminder

this is just a reminder that we have a zookeeper meetup at facebook on november 17. note that usually you have to sign an NDA to get into facebook, but for the meetup we are giving security a list of people for the event so that they can get in without an NDA. that's why we need you to mark you

Re: zookeeper client seems to timeout earlier than it should

clients need to make sure they move off of a dead server on to a new one to keep their connection alive, so generally if the client hasn't heard from the server in 2/3 * sessionTimeout it will try to connect to someone else. if it waited the whole 4 seconds, when connected to an active server it

Re: Acl block detete not working

please check out http://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperProgrammers.html#sc_ACLPermissions. DELETE prevents deletion of children (like CREATE prevents the creation of children). it does not prevent the deletion of the znode itself. ben On Fri, May 12, 2017 at 10:53 PM, Edward

Re: Zookeeper and vrf

not that i know of. how would you envision the library supporting vrf? thanx ben On Mon, Aug 20, 2018 at 11:55 AM, Pramod Srinivasan wrote: > Hi Everyone, any guidance for the question below? > > On 8/16/18, 4:15 PM, "Pramod Srinivasan" wrote: > > Hi Everyone, > > I am using Zookeeper

Re: a misunderstanding of ZAB

the key is what alex said: p1 can be committed or dropped. it was not delivered to a quorum. (if it was, it must be delivered to everyone.) it wasn't skipped. it would be skipped if a new leader did not deliver it and proposed new messages. thus, since it isn't delivered or skipped, skipping it or

Re: Zookeeper client with single address pointing to multiple servers

are you making the assumption that you have a single machine that will always be up? that is not a common assumption these days, which is why solr might be resistant to such a change. you can have a single DNS name resolve to multiple IP addresses and ZooKeeper client will use all those addresses

Re: TLS certificate error does not return a error to client

it sounds like we might be missing a test case. do we not have test case coverage for this one? ben On Fri, Jan 1, 2021 at 8:32 PM Dipti Mulay wrote: > > Hi All, > > I have been using a zookeeper C-client libraries to communicate with the > Zookeeper Cluster(Ensemble). > The communication is

Re: Ephemeral Nodes

can you explain a bit more what you mean by "The zookeeper-client process on standby which uses the same clientid to initialize a session with zookeeper." ? if i understand correctly, you are going to have two clients using the same session which cannot happen. do you only reuse the clientid when

Re: Ephemeral Nodes

> - But what I would like to happen is that D1-B becomes the leader of the lot. > Is this possible in any way ? > > > Hope I am making my self clear. > > Thanks > -Parag > > > On 2/13/21, 12:16 AM, "Benjamin Reed" wrote: > > the simple leader e

Re: Ephemeral Nodes

emeral node is deleted and triggers > > >> leader election again. > > >> > > >> Hope this helps. > > >> > > >> Thanks > > >> -Parag > > >> > > >> > > >> On Fri, 29 Jan 2021 at 13:26, Flavio Junqu

Re: [VOTE] Bookkeeper 4.2.2 release candidate 1

+1 great job ivan! On Tue, Oct 8, 2013 at 6:25 AM, FPJ fpjunque...@yahoo.com wrote: +1, I have checked the release before and that failure was the only thing pending for me. Thanks a lot for your patience and effort, Ivan. -Flavio -Original Message- From: Ivan Kelly

Re: [VOTE] Taking Bookkeeper Subproject to a Top Level Project

+1, awesome! ben On Sat, Oct 18, 2014 at 11:06 PM, Rakesh Radhakrishnan rakeshr.apa...@gmail.com wrote: +1, Great! Best Regards, Rakesh On Fri, Oct 17, 2014 at 11:58 PM, Michi Mutsuzaki mi...@cs.stanford.edu wrote: +1 On Fri, Oct 17, 2014 at 8:55 AM, Greg Asta