Re: Bootstrapping a new cluster and using the reconfig feature

ing for your > tests. > Thanks, > Eric > > On Thu, Dec 30, 2021 at 1:10 PM Alexander Shraer > wrote: > > > The reconfig is in process means something failed during reconfiguration > > and it couldn't complete. Perhaps the new server disconnected in the > mi

Re: Bootstrapping a new cluster and using the reconfig feature

th the leader of > > last commmitted config - try invoking reconfiguration after new servers > are > > connected and synced > > > > I have not set any security at this point. > > > > I am not sure what I am missing at this point, assuming I don't need 2 > > nodes

Re: Bootstrapping a new cluster and using the reconfig feature

This is already possible, since the 3.5.0 release: https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_standaloneEnabled After your single node is up and running, you can connect other nodes to it as described in the reconfig manual. See "Adding servers" in the link

Re: Dynamic Reconfiguration usage

Hi, The only things that can be changed dynamically are the ones in the dynamic configuration file: - list of servers, - their ports, - their roles (follower or observer) - the quorum system definition (majority or hierarchical). AFAIK all other parameters are in the static config file. Thanks,

Re: Clarification on ZooKeeper Timeliness Guarantee

Hi, It sounds tricky to rely on this, because the clocks aren't perfectly in sync across the clients and servers and clock rates may drift. For example, the way syncLimit is counted by the leader may be slower than how B measures it, so the leader might not drop the connection before B's read

Re: upgrade from 3.4.5 to 3.5.6

+1 to what Mate said (I wrote the quoted instructions). On Tue, Mar 24, 2020 at 7:03 AM Szalay-Bekő Máté wrote: > Hi Kuldeep, > > I just want to provide you some background info about our documentation. > The reason to upgrade to 3.4.6 first is to avoid the following error: > > > 2013-01-30

Re: question on ZAB protocol

Yes I believe that this is possible, not only in ZK but in many other systems when your connection to the database fails and you don’t know whether your transaction committed or aborted. Improving this is part of the forever open Zookeeper-22 JIRA. Alex On Sat, Feb 15, 2020 at 6:35 PM

Re: Upgrade guide from 3.4.x to 3.5.x?

Hi, please see “upgrading to 3.5” section here: https://zookeeper.apache.org/doc/r3.5.4-beta/zookeeperReconfig.html On Fri, Feb 14, 2020 at 8:48 PM shrikant kalani wrote: > Hi Allen > > We recently upgrade our Zookeeper clusters from 3.4.13 to 3.5.5. > > Yes the rolling upgrade are possible and

Re: AW: Configuration management for zoo.cfg

etimes to the real file and sometimes to the prefix of the real > file. If there is any history worth keeping, then rolling them over log > style (either with timestamps or config id) is a much more understood > behaviour. > > > > Ari > > > > On 16/12/19 9:53am, Alexan

Re: AW: Configuration management for zoo.cfg

Another potential advantage is that if something bad happened, you would have the latest working config readily available. On Sun, Dec 15, 2019 at 2:53 PM Alexander Shraer wrote: > I wasn't sure whether extracting such information from the log is simple, > and since reconfigurations may

Re: AW: Configuration management for zoo.cfg

t; parse if you care about tracking that sort of thing. > > In case of some network issue, where a node repeatedly flaps, why would > you want to fill the directory with possibly thousands of files? > > > Ari > > > On 15/12/19 2:35pm, Alexander Shraer wrote: > >

Re: AW: Configuration management for zoo.cfg

. > > Is there a way to get Zookeeper to play better with not rewriting its > own config file for my use case? > > > Ari > > > On 12/12/19 5:53am, Alexander Shraer wrote: > > It will change, the number represents the version of the configuration, > and > > wil

Re: AW: Configuration management for zoo.cfg

It will change, the number represents the version of the configuration, and will be updated if you issue a reconfiguration command. Its basically the zxid of the command. Alex On Tue, Dec 10, 2019 at 11:25 PM Aristedes Maniatis wrote: > > On 11/12/19 6:21pm, arne.bachm...@dlr.de wrote: > >

Re: Re: a misunderstanding of ZAB

enough to form quorum to elect a new leader. > > Now partition healed, the commit of p on F1 and F2 should be dropped (in > ZK, this is what "TRUNC" sync is for). > > >> L2 become new leader, he should skip p1. > > If your L2 is F2 here, p1 will not be skipped,

Re: a misunderstanding of ZAB

In case2, it is possible that p1 is committed or dropped. It depends on whether L2 knows about p1. Note that L2 needs the support of a quorum to become leader, and in ZK since there is no state copy from followers to leader, the leader candidate needs to have the longest log. So, if L2's log

Re: About ZooKeeper Dynamic Reconfiguration

ed benefit is that we can also control which data center gets the > quorum > in case of a network outage between the two. > Regards > Chris > > On 21 August 2019 16:42:37 Alexander Shraer wrote: > > > Hi, > > > > Reconfiguration, as implemented, is not a

Re: About ZooKeeper Dynamic Reconfiguration

Hi, Reconfiguration, as implemented, is not automatic. In your case, when failures happen, this doesn't change the ensemble membership. When 2 of 5 fail, this is still a minority, so everything should work normally, you just won't be able to handle an additional failure. If you'd like to remove

Re: Apache Zookeeper Bugs

Thanks Xiaoqin! Would you be able to open a Jira for this and perhaps submit a PR ? https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute On Thu, Aug 1, 2019 at 8:23 AM Xiaoqin Fu wrote: > Dear developers: > I am a Ph.D. student at Washington State University. I applied

Re: How to commit last epoch proposal in zab

The commit is not actually written to the log. The log is updated before a server ACKs a proposal - and what's in the log is what matters for recovery. In your example, server1 sent a commit for p2, so it got at least one ACK from server2 or server3. Since in your example server2 has been elected,

Re: ZK 3.5.5 : SecureClientPort and Server Specs

I think that Fred is correct - secureClientPort and secureClientPortAddress were not made part of the dynamic configuration (yet ?), so unlike other parameters, they are static. Fred, perhaps you could open a Jira to ask for this feature ? Thanks, Alex On Mon, Jul 1, 2019 at 2:58 PM Andor Molnar

Great talk from Ben Reed about the origins of ZooKeeper

https://atscaleconference.com/videos/systems-scale-2019-welcome-keynote/

Re: majority of non-failing machines VS quorum

yep, for odd n that's right. On Thu, May 30, 2019 at 1:30 PM Joel Mestres wrote: > ok great so the minimum quorum always is F + 1 considering n as odd, right > ? > > On Thu, May 30, 2019 at 3:38 PM Alexander Shraer > wrote: > > > If you're using "majority quor

Re: majority of non-failing machines VS quorum

If you're using "majority quorums" (the default in ZK), F=floor((n-1)/2). A quorum is any set containing a majority (or more) of servers. The basic requirement is that any two quorums must intersect. On Wed, May 29, 2019 at 6:55 PM Patrick Hunt wrote: > 2n+1 = ensemble size required to survive

Re: Dynamic Config

Hi, 1. Right - only the configuration parameters that live in the dynamic file are controlled by dynamic reconfig. The dynamic files are kept in sync across all the ZK servers, whereas the static files may not be the same. There is a backward compatibility mode, where you start up a server

Re: Is it safe to reuse zookeeper replica ID when reprovisioning?

is a voter then B can recover from A and they can continue normally. On Mon, Apr 1, 2019 at 5:46 PM Alexander Shraer wrote: > Lets say you have nodes A, B, C. Only B and C have latest data. You're > trying to replace B. > You replace B with a new server but before its in sync, C fails. What

Re: Is it safe to reuse zookeeper replica ID when reprovisioning?

, since B isn't registered to be able to vote. If C never recovers, you can recover manually by updating config files. On Mon, Apr 1, 2019 at 5:10 PM David Anderson wrote: > On Mon, Apr 1, 2019 at 4:48 PM Alexander Shraer wrote: > > > Hi, > > > > I think

Re: Is it safe to reuse zookeeper replica ID when reprovisioning?

Hi, I think that one of the problems with the proposed method is that you may end-up having a majority of servers that don't have the latest state (imagine that there is a minority failure while your replaced node hasn't been brought up do date yet). Have you considered using dynamic

Re: Zookeeper syncing with Curator

> I have to make sure that a read always reflects *all previous writes* (which might be performed on another zookeeper server and has not reached all other instances). By doing a sync before reading, as you say, the read should indeed reflect all *completed* previous writes, i.e., writes that

Re: test zookeeper observer

Hi, look at that server’s log - it should say that it is observing. On Fri, Oct 26, 2018 at 5:21 AM lamriq wrote: > Hello > > I add a new server Zookeeper as observer, but I am not sure if it's work > well or not, how can I test if the observer send OBSERVERINFO and don't > vote. > > Regards >

Re: dynamic config file number

The way it was implemented, is that the version (which is printed in your log, like version=1f001cc8d5) is not stored in the dynamic config file, but is actually part of its file name. It corresponds to the zxid at which the configuration was committed. You should never change that manually, or

Re: Is the value of $MYID allowed to change across runs in an HA ZK deployment?

Hi Jay, Perhaps it also depends on the restart? if the restart is done gradually, for example a leader is in the middle of collecting votes when one of the voters gets a new id and votes twice instead of once ? If the restart is a barrier, where all servers are shut down and then restarted, this

Re: how zookeeper promise FIFO client order

Hi, Specific implementations of Raft may guarantee client program order, but I don't think that it directly follows from tcp order + state machine. It matters whether commands are committed to the log according to program order. For example, here's an implementation that seems to be doing this:

Re: Any way to get information about cluster in CLI mode

In 3.5 there is also the "config" CLI command described here: https://zookeeper.apache.org/doc/trunk/zookeeperReconfig.html#sc_reconfig_retrieving Alex On Tue, Nov 7, 2017 at 11:34 AM, Abraham Fine wrote: > Hi Pavel- > > The ZooKeeper CLI does not have a way to get

Re: Zookeeper 3.5.3 reconfig blocked by ACL

Hi, Please look for "sc_reconfig_access_control" Here: https://github.com/apache/zookeeper/blob/master/docs/zookeeperReconfig.html Thanks, Alex On Tue, Oct 17, 2017 at 3:18 AM, oo4load wrote: > I have a 3.5.3 cluster where I am trying out the reconfig command. I am >

Re: ZooKeeper Time Synchronization

The general idea is to use time for availability but not correctness. So a leader could be suspected as failed which may make the system unavailable until a new one is elected but consistency is not affected. Alex On Fri, Jul 21, 2017 at 1:56 PM Michael Han wrote: > One

Re: ZooKeeper Time Synchronization

As far as I understand: 1) no 2) yes 3) yes 4) yes 5) no, except for the sync command (there is a jira open for that) Others please correct me if I'm wrong Thanks Alex On Fri, Jul 21, 2017 at 11:52 AM Sandeep Singh wrote: > Adding the Amr question. > Few things which

Re: gracefully remove a node from the ensamble

Well, first of all you need to bootstrap a system - so all the nodes should know of each other. This hasn't changed in 3.5. When you add a new server, you also need to bootstrap its config file with something (there are a few suggestions in the manual) - it doesn't need to be the latest config but

Re: gracefully remove a node from the ensamble

.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun( > QuorumPeerMain.java:120) > at > org.apache.zookeeper.server.quorum.QuorumPeerMain.main( > QuorumPeerMain.java:79) > > What am I doing wrong? should the second server reach the first one, get > the list of the other serve

Re: gracefully remove a node from the ensamble

I'd suggest to use 3.5.3. ZK only officially supports a Java and C client as far as I know. I know these two support it, not sure if anyone ported it to other clients. Alex On Fri, Jul 14, 2017 at 11:04 AM, Luigi Tagliamonte < luigi.tagliamont...@gmail.com> wrote: > Hello again Alexander, > so

Re: gracefully remove a node from the ensamble

> > On Thu, Jul 13, 2017 at 6:19 PM, Alexander Shraer <shra...@gmail.com> > wrote: > > > Hi Luigi, > > > > In 3.5.X yes: https://zookeeper.apache.org/doc/trunk/zookeeperReconfig. > > html > > > > For previous releases (3.4 etc) you would nee

Re: gracefully remove a node from the ensamble

Hi Luigi, In 3.5.X yes: https://zookeeper.apache.org/doc/trunk/zookeeperReconfig.html For previous releases (3.4 etc) you would need to do a rolling restart, where for each server you change the config file to exclude that member and bounce the server. Preferably do this one server at a time,

Re: New to zookeeper

Just a small comment - 3.5.3 is in beta. The getConfig API returns a list of servers in the cluster, including their ports and roles in the ensemble. Alex On Wed, Jul 12, 2017 at 7:53 AM, Washko, Daniel wrote: > I speak strictly from my experience with Zookeeper and not

Re: New PMC Member: Michael Han

congrats Michael!! On Tue, Jun 27, 2017 at 6:04 PM, Gaurav Sharma wrote: > Congrats Michael! > > On Tue, Jun 27, 2017 at 09:48 Flavio Junqueira wrote: > > > I'm very happy to announce that the Apache ZooKeeper PMC has voted to > > invite Michael Han to

Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false

le changes to zoo.cfg > followed > > by restart of a single server are ignored. It is necessary to stop all > > servers and do the steps above to make any changes to the ensemble. > > > > Thanks, > > > > Alexander Shraer <shra...@gmail.com> w

Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false

ns when I > talked with them about what this feature could do. There are also some > discussions around this flag / why it's there in ZOOKEEPER-2014. > > [1] > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ReleaseManagement > > > On Thu, Jun 22, 2017 at 10:39 PM,

Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false

he static zoo.cfg intact. > > > On Thu, Jun 22, 2017 at 9:35 PM, Alexander Shraer <shra...@gmail.com> > wrote: > > > This sounds like a bug in the implementation of reconfigEnabled. > > Could you please open a JIRA with the description you provided ? > > > > Out

Re: How to add nodes to a Zookeeper 3.5.3-beta ensemble with reconfigEnabled=false

This sounds like a bug in the implementation of reconfigEnabled. Could you please open a JIRA with the description you provided ? Out of curiosity, why do you disable reconfig ? It is intended exactly to perform the changes you're trying to make, in a simple and correct way. Thanks, Alex On

Re: [ANNOUNCE] Apache ZooKeeper 3.5.3-beta

The issue Patrick was referring to is described here: https://zookeeper.apache.org/doc/trunk/zookeeperReconfig.html#ch_reconfig_upgrade On Thu, Apr 20, 2017 at 9:22 AM, Patrick Hunt wrote: > On Thu, Apr 20, 2017 at 9:12 AM, Michael Han wrote: > > > Just to

Re: Zookeeper Ensemble Automation

Since configuration info is stored in a znode, you could access it using a simple get operation. The getconfig operation is basically doing just that. So if you have a 3.5 server and a 3.4 client, the client should be able to read the list of servers and get notified when the list changes by

Re: november meetup at facebook (take 2)

+1 for me too, thanks! On Fri, Sep 30, 2016 at 3:18 PM, Ryan Zhang wrote: > +1. My coworkers in twitter would be interested. > > > On Sep 30, 2016, at 2:35 PM, Raúl Gutiérrez Segalés > wrote: > > > > +1 (probably bringing along some people from

Re: Error Start ZK 3.5.1 a second time

Sounds like definitely a bug :) could you please open a JIRA ? And if you can upload a patch this would be very appreciated. This code should be in QuorumPeerConfig.java String dynamicConfigFilePath = PathUtils.normalizeFileSystemPath Thanks, Alex On Thu, Jul 14, 2016 at 8:36 AM, Cantrell,

Re: how is zookeeper deploy at multi datacenter?

our recent paper may be relevant: https://www.usenix.org/conference/atc16/technical-sessions/presentation/lev-ari On Wed, Jun 29, 2016 at 10:04 PM, chen dongming wrote: > How many ways to deploy at multi datacenter for backup? > > From my point view: > > 1. use observer >

Re: read under transaction

But these writes can be conditional (on the version of the data), which could probably be used to achieve what you need. On Tue, Jun 28, 2016 at 11:33 AM, Patrick Hunt wrote: > Multi is more of an atomic operation than a "transaction" in the typical > sense. See

Re: observer changing to participant when there is no quorum

gt; > > Client could not connect to reestablished quorum: giving up after 30+ > seconds > > > > > > I am passing the original server configure string to zk's reconfig > method. > > > > > > > > Thanks > > > > > > Printing e-mails was

Re: Zookeeper 3.5.1 dynamic configuration fails with EOFException

even if you start 2 as follower, it may restart leader election and drop a connection since it learns about a more up to date configuration. We didn't optimize such restarts for simplicity. On Fri, Jun 10, 2016 at 9:16 PM, Alexander Shraer <shra...@gmail.com> wrote: > In this spec

Re: zookeeper deployment strategy for multi data centers

> Is there any settings to override the quorum rule? Would you know the rationale behind it? The rule comes from a theoretical impossibility saying that you must have n > 2f replicas to tolerate f failures, for any algorithm trying to solve consensus while being able to handle periods of

Re: sharing a post on ZAB architecture

And here's another explanation of Zab we wrote for the reconfiguration paper, which explains ZAB in more abstract terms (without various optimizations), and in a way that relates it to Paxos: Section 2 in https://www.usenix.org/system/files/conference/atc12/atc12-final74.pdf On Wed, Jun 1, 2016

Re: how to make a server be leader permanently

If you're interested to work on something like that, a good starting point could be implementing a leader handoff API: ZOOKEEPER-2076 On Mon, May 2, 2016 at 4:19 AM, Flavio P JUNQUEIRA wrote: > We don't have this kind of behavior enabled because it'd affect > availability. If

Re: Zookeeper with SSL release date

Hi Shawn, My proposal was in the following context - Flavio suggested to add new flag(s) to disable reconfig in order not to surprise users with new security vulnerabilities that arise from dynamic reconfiguration. My point was that we already have such a mechanism we could use - ACLs. But if we

Re: Zookeeper with SSL release date

concerns. Why limit reconfig to ACL users only? > > On Thu, Mar 31, 2016 at 11:37 PM, Alexander Shraer <shra...@gmail.com> > wrote: > > > Citing Patrick: > > > > > If you're running zk w/o security turned on and suddenly folks can do > > reconfig &

Re: automatic update of server set at the client on reconfig

Hi, Please see update_addrs() function of the C client, and the following link: https://zookeeper.apache.org/doc/trunk/zookeeperReconfig.html#ch_reconfig_rebalancing It could be automated further (e.g., ZOOKEEPER-2016 ) but there hasn't been

Re: Zookeeper with SSL release date

? What would you check precisely? > > -Flavio > > > On 24 Mar 2016, at 21:19, Alexander Shraer <shra...@gmail.com> wrote: > > > > I'm not so sure its orthogonal. The question is whether someone would > ever > > want to use reconfig without ACLs,

Re: Zookeeper with SSL release date

rthogonal requirement? > > On Thu, Mar 24, 2016 at 3:37 PM, Alexander Shraer <shra...@gmail.com> > wrote: > > > How about a simpler alternative to the proposed flag for reconfig: a > check > > in the code that requires ACLs to be set. > > If people want to use rec

Re: Zookeeper with SSL release date

would say if in doubt add a safety. (a config parameter to turn it > off). Cost is almost zero and worst case it will just give us peace of > mind. ;-) > > Patrick > > On Mon, Mar 21, 2016 at 9:41 PM, Alexander Shraer <shra...@gmail.com> > wrote: > > ok, thanks for the sug

Re: Zookeeper with SSL release date

system if defined) and has an option to filter that info and just return the server connection string (server and client port only). On Mon, Mar 21, 2016 at 9:32 PM, Patrick Hunt <ph...@apache.org> wrote: > On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <shra...@gmail.com> >

Re: Zookeeper with SSL release date

another thing - shouldn't things like setting quotas also be part of the admin API ? how does that work now ? Alex On Mon, Mar 21, 2016 at 9:14 PM, Alexander Shraer <shra...@gmail.com> wrote: > I don't think that getConfig should be an admin functionality. It is > essential for cl

Re: Zookeeper with SSL release date

Looking at the list of ~50 blocker and critical bugs in ZooKeeper, only 3-4 are related to reconfig. Given this, and the fact that it is run in production since 2012 in multiple companies, I don't think its more unstable than any other part of ZooKeeper. There are multiple reconfig-related bugs

Re: Zookeeper with SSL release date

one using 3.5 who uses the "experimental" API must > explicit > > turn on the switch and enable the calls. If they do it, they need to be > > aware that the API can change. > > > > > > I must say that I haven't really looked closely into doing it, and I'm >

Re: zookeeper client session write-read consistency

The server to which the client is connected will buffer the read until the write is executed and applied to its state, so the read will necessarily return a value at least as recent as the one written by the write in your example. ZK guarantees that async operations are executed in order of

Re: ZooKeeper transaction properties (partial read)

t;3. Read /foo as A' > >4. Read /bar as B > > > > #3 & #4 start after #2 completes entirely, right? And #3 & #4 are issued > > 100% sequentially? > > > > ~Jared > > > > On Mon, Feb 1, 2016 at 2:54 PM, Alexander Shraer <shra...@gmail.

Re: ZooKeeper transaction properties (partial read)

Reading the 965 JIRA what you're describing sounds like a bug. Alex On Mon, Feb 1, 2016 at 10:41 AM, Whitney Sorenson wrote: > Hi, > > In searching through the ZK documentation, this list, > https://issues.apache.org/jira/browse/ZOOKEEPER-965, and curator > documentation

Re: Apache ZooKeeper Meetup - Jan 27, Cloudera HQ

g? Contact support here: > > https://cloudera.webex.com/cloudera/mc > > IMPORTANT NOTICE: Please note that this WebEx service allows audio and > > other information sent during the session to be recorded, which may be > > discoverable in a legal matter. You should inform all meet

Re: Apache ZooKeeper Meetup - Jan 27, Cloudera HQ

Thanks for organizing! If possible, I'd like to give a short presentation (10 min ?) about Kfir's work on ZOOKEEPER-2024. I think its a very important improvement and we should get this in 3.5 Cheers, Alex On Thu, Jan 21, 2016 at 9:31 AM, Rakesh Radhakrishnan < rakeshr.apa...@gmail.com> wrote:

Re: Best-practice guides on coordination of operations in distributed systems (and some C client specific questions)

design end to > end. > > > > I do not see a need to do a total replay of past history etc when using > ZK like system because ZK is built on idea of serialized and replicated > log, hence if you are using ZK then your design should be much simpler i.e > fail and re-start f

Re: Best-practice guides on coordination of operations in distributed systems (and some C client specific questions)

aster may not recognize the operation as consistent with its > decisions (or previous decision path). > > Thoughts? Suggestions? > > > > On Sun, Jan 3, 2016 at 2:18 PM, Alexander Shraer <shra...@gmail.com> > wrote: > > regarding atomic multi-znode updates

Re: Best-practice guides on coordination of operations in distributed systems (and some C client specific questions)

regarding atomic multi-znode updates -- check out "multi" updates <http://tdunning.blogspot.com/2011/06/tour-of-multi-update-for-zookeeper.html> . On Sat, Jan 2, 2016 at 10:45 PM, Alexander Shraer <shra...@gmail.com> wrote: > for 1, see the chubby paper > <http

Re: Best-practice guides on coordination of operations in distributed systems (and some C client specific questions)

for 1, see the chubby paper , section 2.4. for 2, I'm not sure I fully understand the question, but essentially, ZK guarantees that even during failures consistency of updates is preserved. The user

Re: ZK + dynamic config + EC2

ZK has a byte range for values server IDs - > 255 in total, how could you allocate those to instances without having a > central registry for IDs? It could be based on IPs, but it seems there are > no other idempotent/reliable ways to get them assigned? > > On 21 November 201

Re: ZK + dynamic config + EC2

The only issue I see is that if the new server has the same id as the old one you're replacing, I think you should first remove the old one and then in a separate command add the new one. Intuitively this way you avoid having the newly joining server act as someone who knows the current state of

Re: Migrate Cluster

assuming you're using 3.4 release and doing reboots to add/remove servers, option 2 doesn't seem safe. For example, if you have servers A, B, C and you're adding D and E, note that its possible that C isn't fully up to date since A and B can make progress without C's acks (2 out of 3). When you

Re: Zab Failure scenario

A reconfiguration is treated similarly to other proposals for recovery purposes (of course commit is different in that it changes the configuration). You can see the paper for details on how recovery works in principle, and

Re: 3-server Zab cluster

Committing locally when sending an ACK at a server would lead to loss of consistency - it is possible that this is the only server that acks, e.g., this server is temporarily disconnected from the leader, the leader gets re-elected and the operation is truncated from logs at other servers. Its ok

Re: [ANNOUNCE] New committer: Chris Nauroth

Congrats Chris, and welcome! On Mon, Sep 28, 2015 at 9:52 AM, Rakesh Radhakrishnan < rakeshr.apa...@gmail.com> wrote: > Welcome Chris, thanks for all your great work and congrats! > > -Rakesh > > On Mon, Sep 28, 2015 at 8:11 PM, Flavio Junqueira wrote: > > > The Apache

Re: 3-server Zab cluster

ate with L (pervious leader now become a follower), as a result Zxid10 > commits by new quorum. Is that correct? > > What do you think? > > Ibrahim > > > > > > -Original Message- > From: Alexander Shraer [mailto:shra...@gmail.com] > Sent: M

Re: Uninvited ZK joins the cluster

Hi, There were some thoughts to send and check the database id (if I'm not mistaken its called dbid) when servers connect to each other, which should be different for different zookeepers. It shouldn't be difficult to add, if you'd like to work on it. Alex On Wed, Sep 9, 2015 at 11:04 AM,

Re: Doubts about libzookeeper

get very far. We have guarantees about how far out of sync a quorum member can be before it's booted, so I would think that there's some way to timebound this potentially to prevent it, a la your suggestion about 3X synctime. C On Tue, Aug 4, 2015 at 4:58 PM, Alexander Shraer shra...@gmail.com

Re: Doubts about libzookeeper

Hi Camille, if the client received a response for the delete then sure it shouldn't be able to connect to servers that didn't see it. But if it disconnected before seeing the response the example seems possible to me. I haven't checked the code to see when exactly the transaction number is

Re: Doubts about libzookeeper

when the xid increments. If that is the case, you can force a sync before the read of the path, to prevent reading stale data. So that would be the solve for that edge case although it's an expensive solve. C On Tue, Aug 4, 2015 at 12:52 PM, Alexander Shraer shra...@gmail.com wrote: Hi

Re: Doubts about libzookeeper

? On Tue, Aug 4, 2015 at 2:20 PM, Alexander Shraer shra...@gmail.com wrote: It seems that since the delete may be in-flight (between server 1 and leader, or still being proposed by the leader) when the client connects to server 2, doing a sync right a way may not help since the operation

Re: Doubts about libzookeeper

just do it again once reconnected right, the whole discussion is unnecessarily complex for a delete op :) On Tue, Aug 4, 2015 at 2:29 PM, Flavio Junqueira f...@apache.org wrote: Touché! -Flavio On 04 Aug 2015, at 22:21, Jordan Zimmerman jor...@jordanzimmerman.com wrote: If the

Re: starting a ZK cluster one node at a time

Hi, When you're adding a node its config file should contain the current set of servers + itself. This will allow it to boot and connect to the cluster (once it does, its config file is overwritten automatically with the latest config of the cluster, which doesn't include the new node). Then, you

Re: starting a ZK cluster one node at a time

Original message From: Alexander Shraer shra...@gmail.com Date:07/24/2015 4:04 PM (GMT-08:00) To: user@zookeeper.apache.org Subject: Re: starting a ZK cluster one node at a time Hi, When you're adding a node its config file should contain the current set of servers

Re: new paper on optimizing replication config

, Alexander Shraer shra...@gmail.com escreveu: Our paper http://www.cs.technion.ac.il/~shralex/p2309-shraer.pdf on optimizing the configuration of distributed storage was recently accepted to the International Conference on Very Large Databases (VLDB). It basically shows that reconfiguration

Re: locking/leader election and dealing with session loss

+1 to what Camille is saying suggestion to use generations On Wed, Jul 15, 2015 at 12:04 PM, Camille Fournier skami...@gmail.com wrote: If client a does a full gc immediately before sending a message that is long enough to lose the lock, it will send the message out of order. You cannot

Re: locking/leader election and dealing with session loss

might not. -Jordan On July 15, 2015 at 3:15:16 PM, Alexander Shraer (shra...@gmail.com) wrote: Jordan, as Camille suggested, please read Sec 2.4 in the Chubby paper: link http://static.googleusercontent.com/media/research.google.com/en//archive/chubby-osdi06.pdf it suggests 2 ways

Re: locking/leader election and dealing with session loss

:57:12 PM, Alexander Shraer (shra...@gmail.com) wrote: This property may hold if you make a lot of timing/synchrony assumptions -- agreeing on who holds the lock in an asynchronous distributed system with failures is impossible, this is the FLP impossibility. But even if it holds

Re: locking/leader election and dealing with session loss

Jordan, as Camille suggested, please read Sec 2.4 in the Chubby paper: link http://static.googleusercontent.com/media/research.google.com/en//archive/chubby-osdi06.pdf it suggests 2 ways in which the storage can support lock generations and proposes an alternative for the case where the storage

new paper on optimizing replication config

Our paper http://www.cs.technion.ac.il/~shralex/p2309-shraer.pdf on optimizing the configuration of distributed storage was recently accepted to the International Conference on Very Large Databases (VLDB). It basically shows that reconfiguration can be used to significantly improve latency. It

Re: Is myid actually limited to [1, 255]?

negative ids could break stuff, such as here: https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java#L321 On Mon, Jul 13, 2015 at 11:07 AM, Raúl Gutiérrez Segalés r...@itevenworks.net wrote: Hi, On 13 July 2015 at 10:43,

Re: ZooKeeper ensemble. Size and Impact ?

In 3.4 releases you can't connect an observer to a standalone zookeeper server, but in 3.5.0 if you set standaloneEnabled=false your server will run in a distributed mode even if its the only one and you'll be able to have observers or reconfigure adding more servers later if needed. On Mon, Jul

  1   2   3   >