Re: a misunderstanding of ZAB
+1 with what Alex has said. The commit case is easy to understand. For skip case, think this example: old quorum: F1 F2 F3 F4 F5, with F1 as L1. L1 has p on F1 and F2. new quorum: F1 F2 F3 F4 F5, with F3 as L2. It's possible, because although F1 and F2 has latest zxid, they could be partitioned away and F3 F4 F5 are enough to form quorum to elect a new leader. Now partition healed, the commit of p on F1 and F2 should be dropped (in ZK, this is what "TRUNC" sync is for). >> L2 become new leader, he should skip p1. If your L2 is F2 here, p1 will not be skipped, since p1 is available on F2 the new leader. On Tue, Sep 3, 2019 at 10:35 AM Alexander Shraer wrote: > In case2, it is possible that p1 is committed or dropped. It depends on > whether L2 knows about p1. > Note that L2 needs the support of a quorum to become leader, and in ZK > since there is no state copy from followers to leader, the leader candidate > needs to have the longest log. > So, if L2's log includes p1 it will be committed otherwise it will be > dropped. > > In case1 L2's log necessarily includes p1 since it is present at a quorum > and without having it in the log its not possible to have a log more > up-to-date than that of a quorum / get the support of a quorum to become > leader. > > Alex > > > On Tue, Sep 3, 2019 at 4:52 AM Norbert Kalmar > > wrote: > > > Hi, > > > > That's a good question. So if I understand correctly, you are asking what > > happens if there is a new Leader Election in ZooKeeper, what is the "last > > seen zxid". I checked the ZAB protocol, it is not entirely clear for me > as > > well, but my understanding is that the last seen zxid is the last > > transaction, which is read from txnlogs in case of a recovery. Honestly, > > there's nothing else this could be read from. So if it hasn't been > > committed to the datatree (and that exists in memory anyway, at least > until > > a snapshot is taken), it is still the last txn that is logged by one of > the > > followers, so he will win the Leader Election, and the followers will get > > this txn as well. > > Anyone agree/disagree? :) > > > > Regards, > > Norbert > > > > On Mon, Sep 2, 2019 at 4:50 AM 121476...@qq.com <121476...@qq.com> > wrote: > > > > > hi, i'm a new to zookeeper, and this problem confuses me for nearly two > > > months... > > > papers tell me that zab must satisfy: > > > A message delivered by one sever must be delivered on quorum. > > > A message skipped must always be skipped. > > > Then consider two cases below, L is short for leader, F is short for > > > follower, p is short for proposal. > > > Case1: > > > L send p1 to F2 F3 F4 F5. > > > F2 F3 ack p1, reach a quorum. > > > L1 is about to send commit but failed... > > > L2 become new leader, he should commit. > > > > > > Case2: > > > L1 send p1 to F2 F3 F4 F5. > > > Only F2 ack p1, not reach a quorum. > > > Then L1 failed... > > > L2 become new leader, he should skip p1. > > > > > > i think L2 should handle the cases in election phase. but how > L2 > > > can know the global state and decide if commit p1 or skip p1？ > > > if anyone helps, i will be much appreciate. > > > > > > > > > > > > 121476...@qq.com > > > > > >
Re: a misunderstanding of ZAB
In case2, it is possible that p1 is committed or dropped. It depends on whether L2 knows about p1. Note that L2 needs the support of a quorum to become leader, and in ZK since there is no state copy from followers to leader, the leader candidate needs to have the longest log. So, if L2's log includes p1 it will be committed otherwise it will be dropped. In case1 L2's log necessarily includes p1 since it is present at a quorum and without having it in the log its not possible to have a log more up-to-date than that of a quorum / get the support of a quorum to become leader. Alex On Tue, Sep 3, 2019 at 4:52 AM Norbert Kalmar wrote: > Hi, > > That's a good question. So if I understand correctly, you are asking what > happens if there is a new Leader Election in ZooKeeper, what is the "last > seen zxid". I checked the ZAB protocol, it is not entirely clear for me as > well, but my understanding is that the last seen zxid is the last > transaction, which is read from txnlogs in case of a recovery. Honestly, > there's nothing else this could be read from. So if it hasn't been > committed to the datatree (and that exists in memory anyway, at least until > a snapshot is taken), it is still the last txn that is logged by one of the > followers, so he will win the Leader Election, and the followers will get > this txn as well. > Anyone agree/disagree? :) > > Regards, > Norbert > > On Mon, Sep 2, 2019 at 4:50 AM 121476...@qq.com <121476...@qq.com> wrote: > > > hi, i'm a new to zookeeper, and this problem confuses me for nearly two > > months... > > papers tell me that zab must satisfy: > > A message delivered by one sever must be delivered on quorum. > > A message skipped must always be skipped. > > Then consider two cases below, L is short for leader, F is short for > > follower, p is short for proposal. > > Case1: > > L send p1 to F2 F3 F4 F5. > > F2 F3 ack p1, reach a quorum. > > L1 is about to send commit but failed... > > L2 become new leader, he should commit. > > > > Case2: > > L1 send p1 to F2 F3 F4 F5. > > Only F2 ack p1, not reach a quorum. > > Then L1 failed... > > L2 become new leader, he should skip p1. > > > > i think L2 should handle the cases in election phase. but how L2 > > can know the global state and decide if commit p1 or skip p1？ > > if anyone helps, i will be much appreciate. > > > > > > > > 121476...@qq.com > > >
Re: Migrate from 3.4.x to 3.5.5
Il mar 3 set 2019, 13:45 Norbert Kalmar ha scritto: > In my opinion we should not fallback to something else if the client wants > a CONTAINER node, which is a new feature in 3.5. > +1 Enrico > This makes ZooKeeper behave kind of unreliable. Client wants a CONTAINER > node, and gets a PERSISTENT one. How should client handle this? > > For me, throwing an error seems more reasonable. And this doesn't brake > backward compatibleness, as if you upgrade your client, you are still able > to do the things you did with previous version. Just not the new feature > stuff. > > Regards, > Norbert > > On Tue, Sep 3, 2019 at 1:22 PM Zili Chen wrote: > > > Well, we cannot make any reasonable fallback from a previous version, > > but isn't it regraded as a break changes? > > > > Best, > > tison. > > > > > > Zili Chen 于2019年9月3日周二 下午7:20写道： > > > > > Hi here, > > > > > > If I communicate quorums running zk 3.4 with a zk 3.5 client, when I am > > > trying to create node in CONTAINER mode the quorums will compliant > > > with KeeperException.UnimplementedException. > > > > > > That is to say, as a client application, if I upgrade zk to 3.5 and > then > > > make use of this new feature, we doesn't have a fallback when zk server > > > version is 3.4 to PERSISTENT or something like that, but just fail with > > > an exception. Is it expected? If so, it seems upgrade client side force > > > user to upgrade server side also. > > > > > > Best, > > > tison. > > > > > > > > > Zili Chen 于2019年8月26日周一 下午8:47写道： > > > > > >> Thanks for your insight Andor! > > >> > > >> I'll checkout the page :-) > > >> > > >> Best, > > >> tison. > > >> > > >> > > >> Andor Molnar 于2019年8月26日周一 下午8:41写道： > > >> > > >>> Hi Zili, > > >>> > > >>> There’s no migration guide available for 3.5, because it shouldn’t > > break > > >>> any existing functionality and no need to upgrade the database > either. > > >>> > > >>> I’ve created a wiki page to collect upgrade experiences from users > > which > > >>> could give you some hint if you’re facing problems: > > >>> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ > > >>> > > >>> You can always drop an email here too to get help. > > >>> > > >>> Andor > > >>> > > >>> > > >>> > > >>> > On 2019. Aug 26., at 14:12, Zili Chen > wrote: > > >>> > > > >>> > Detailedly, in Flink community we try to bump ZooKeeper version > from > > >>> 3.4.10 > > >>> > to 3.5.5 but without accurate idea about how it would break > existing > > >>> > systems. > > >>> > Mainly we make use of the "client" of ZooKeeper. > > >>> > > > >>> > > > >>> > Zili Chen 于2019年8月26日周一 下午8:02写道： > > >>> > > > >>> >> Hi, > > >>> >> > > >>> >> Is there any migration guide for potentially breaking changes and > > how > > >>> to > > >>> >> deal with them? > > >>> >> > > >>> >> Best, > > >>> >> tison. > > >>> >> > > >>> > > >>> > > >
Re: a misunderstanding of ZAB
Hi, That's a good question. So if I understand correctly, you are asking what happens if there is a new Leader Election in ZooKeeper, what is the "last seen zxid". I checked the ZAB protocol, it is not entirely clear for me as well, but my understanding is that the last seen zxid is the last transaction, which is read from txnlogs in case of a recovery. Honestly, there's nothing else this could be read from. So if it hasn't been committed to the datatree (and that exists in memory anyway, at least until a snapshot is taken), it is still the last txn that is logged by one of the followers, so he will win the Leader Election, and the followers will get this txn as well. Anyone agree/disagree? :) Regards, Norbert On Mon, Sep 2, 2019 at 4:50 AM 121476...@qq.com <121476...@qq.com> wrote: > hi, i'm a new to zookeeper, and this problem confuses me for nearly two > months... > papers tell me that zab must satisfy: > A message delivered by one sever must be delivered on quorum. > A message skipped must always be skipped. > Then consider two cases below, L is short for leader, F is short for > follower, p is short for proposal. > Case1: > L send p1 to F2 F3 F4 F5. > F2 F3 ack p1, reach a quorum. > L1 is about to send commit but failed... > L2 become new leader, he should commit. > > Case2: > L1 send p1 to F2 F3 F4 F5. > Only F2 ack p1, not reach a quorum. > Then L1 failed... > L2 become new leader, he should skip p1. > > i think L2 should handle the cases in election phase. but how L2 > can know the global state and decide if commit p1 or skip p1？ > if anyone helps, i will be much appreciate. > > > > 121476...@qq.com >
Re: Migrate from 3.4.x to 3.5.5
In my opinion we should not fallback to something else if the client wants a CONTAINER node, which is a new feature in 3.5. This makes ZooKeeper behave kind of unreliable. Client wants a CONTAINER node, and gets a PERSISTENT one. How should client handle this? For me, throwing an error seems more reasonable. And this doesn't brake backward compatibleness, as if you upgrade your client, you are still able to do the things you did with previous version. Just not the new feature stuff. Regards, Norbert On Tue, Sep 3, 2019 at 1:22 PM Zili Chen wrote: > Well, we cannot make any reasonable fallback from a previous version, > but isn't it regraded as a break changes? > > Best, > tison. > > > Zili Chen 于2019年9月3日周二 下午7:20写道： > > > Hi here, > > > > If I communicate quorums running zk 3.4 with a zk 3.5 client, when I am > > trying to create node in CONTAINER mode the quorums will compliant > > with KeeperException.UnimplementedException. > > > > That is to say, as a client application, if I upgrade zk to 3.5 and then > > make use of this new feature, we doesn't have a fallback when zk server > > version is 3.4 to PERSISTENT or something like that, but just fail with > > an exception. Is it expected? If so, it seems upgrade client side force > > user to upgrade server side also. > > > > Best, > > tison. > > > > > > Zili Chen 于2019年8月26日周一 下午8:47写道： > > > >> Thanks for your insight Andor! > >> > >> I'll checkout the page :-) > >> > >> Best, > >> tison. > >> > >> > >> Andor Molnar 于2019年8月26日周一 下午8:41写道： > >> > >>> Hi Zili, > >>> > >>> There’s no migration guide available for 3.5, because it shouldn’t > break > >>> any existing functionality and no need to upgrade the database either. > >>> > >>> I’ve created a wiki page to collect upgrade experiences from users > which > >>> could give you some hint if you’re facing problems: > >>> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ > >>> > >>> You can always drop an email here too to get help. > >>> > >>> Andor > >>> > >>> > >>> > >>> > On 2019. Aug 26., at 14:12, Zili Chen wrote: > >>> > > >>> > Detailedly, in Flink community we try to bump ZooKeeper version from > >>> 3.4.10 > >>> > to 3.5.5 but without accurate idea about how it would break existing > >>> > systems. > >>> > Mainly we make use of the "client" of ZooKeeper. > >>> > > >>> > > >>> > Zili Chen 于2019年8月26日周一 下午8:02写道： > >>> > > >>> >> Hi, > >>> >> > >>> >> Is there any migration guide for potentially breaking changes and > how > >>> to > >>> >> deal with them? > >>> >> > >>> >> Best, > >>> >> tison. > >>> >> > >>> > >>> >
Re: Migrate from 3.4.x to 3.5.5
Well, we cannot make any reasonable fallback from a previous version, but isn't it regraded as a break changes? Best, tison. Zili Chen 于2019年9月3日周二 下午7:20写道： > Hi here, > > If I communicate quorums running zk 3.4 with a zk 3.5 client, when I am > trying to create node in CONTAINER mode the quorums will compliant > with KeeperException.UnimplementedException. > > That is to say, as a client application, if I upgrade zk to 3.5 and then > make use of this new feature, we doesn't have a fallback when zk server > version is 3.4 to PERSISTENT or something like that, but just fail with > an exception. Is it expected? If so, it seems upgrade client side force > user to upgrade server side also. > > Best, > tison. > > > Zili Chen 于2019年8月26日周一 下午8:47写道： > >> Thanks for your insight Andor! >> >> I'll checkout the page :-) >> >> Best, >> tison. >> >> >> Andor Molnar 于2019年8月26日周一 下午8:41写道： >> >>> Hi Zili, >>> >>> There’s no migration guide available for 3.5, because it shouldn’t break >>> any existing functionality and no need to upgrade the database either. >>> >>> I’ve created a wiki page to collect upgrade experiences from users which >>> could give you some hint if you’re facing problems: >>> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ >>> >>> You can always drop an email here too to get help. >>> >>> Andor >>> >>> >>> >>> > On 2019. Aug 26., at 14:12, Zili Chen wrote: >>> > >>> > Detailedly, in Flink community we try to bump ZooKeeper version from >>> 3.4.10 >>> > to 3.5.5 but without accurate idea about how it would break existing >>> > systems. >>> > Mainly we make use of the "client" of ZooKeeper. >>> > >>> > >>> > Zili Chen 于2019年8月26日周一 下午8:02写道： >>> > >>> >> Hi, >>> >> >>> >> Is there any migration guide for potentially breaking changes and how >>> to >>> >> deal with them? >>> >> >>> >> Best, >>> >> tison. >>> >> >>> >>>
Re: Migrate from 3.4.x to 3.5.5
Hi here, If I communicate quorums running zk 3.4 with a zk 3.5 client, when I am trying to create node in CONTAINER mode the quorums will compliant with KeeperException.UnimplementedException. That is to say, as a client application, if I upgrade zk to 3.5 and then make use of this new feature, we doesn't have a fallback when zk server version is 3.4 to PERSISTENT or something like that, but just fail with an exception. Is it expected? If so, it seems upgrade client side force user to upgrade server side also. Best, tison. Zili Chen 于2019年8月26日周一 下午8:47写道： > Thanks for your insight Andor! > > I'll checkout the page :-) > > Best, > tison. > > > Andor Molnar 于2019年8月26日周一 下午8:41写道： > >> Hi Zili, >> >> There’s no migration guide available for 3.5, because it shouldn’t break >> any existing functionality and no need to upgrade the database either. >> >> I’ve created a wiki page to collect upgrade experiences from users which >> could give you some hint if you’re facing problems: >> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ >> >> You can always drop an email here too to get help. >> >> Andor >> >> >> >> > On 2019. Aug 26., at 14:12, Zili Chen wrote: >> > >> > Detailedly, in Flink community we try to bump ZooKeeper version from >> 3.4.10 >> > to 3.5.5 but without accurate idea about how it would break existing >> > systems. >> > Mainly we make use of the "client" of ZooKeeper. >> > >> > >> > Zili Chen 于2019年8月26日周一 下午8:02写道： >> > >> >> Hi, >> >> >> >> Is there any migration guide for potentially breaking changes and how >> to >> >> deal with them? >> >> >> >> Best, >> >> tison. >> >> >> >>
Re: ZooKeeper swag on RedBubble available
Super ! Enrico Il mar 3 set 2019, 10:57 Andor Molnar ha scritto: > Hi ZooKeeper fans, > > Get your favourite ZooKeeper swag today! :) > > > https://www.redbubble.com/people/comdev/works/40935715-apache-zookeeper?asc=u > > Best, > Andor > > >
ZooKeeper swag on RedBubble available
Hi ZooKeeper fans, Get your favourite ZooKeeper swag today! :) https://www.redbubble.com/people/comdev/works/40935715-apache-zookeeper?asc=u Best, Andor
