[ANNOUNCE] Apache ZooKeeper 3.7.1

[Mohammad Arshad]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.7.1

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.7.1 Release Notes are at:
https://zookeeper.apache.org/doc/r3.7.1/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,

The ZooKeeper Team

RE: [ANNOUNCE] Apache ZooKeeper 3.6.3 Release

[Mohammad arshad]

Thanks  Damien Diederen , Szalay-Bekő Máté and All ZK community members who 
contributed and supported during this release.
Special Thanks to Enrico Olivelli for being the single point of contact for all 
my queries and doubts during this release and encouragement to volunteer as RM.

Thanks & Regards
Arshad

-Original Message-
From: Damien Diederen [mailto:ddiede...@apache.org] 
Sent: Thursday, April 15, 2021 11:42 AM
To: Mohammad Arshad 
Cc: d...@zookeeper.apache.org; user@zookeeper.apache.org
Subject: Re: [ANNOUNCE] Apache ZooKeeper 3.6.3 Release


Thank you, Arshad!  The release looks very good, and I'm really thankful for 
your efforts on quality and stability.

Cheers, -D


Mohammad Arshad  writes:
> The Apache ZooKeeper team is proud to announce Apache ZooKeeper 
> version
> 3.6.3
>
> ZooKeeper is a high-performance coordination service for distributed 
> applications. It exposes common services - such as naming, 
> configuration management, synchronization, and group services - in a 
> simple interface so you don't have to write them from scratch. You can 
> use it off-the-shelf to implement consensus, group management, leader 
> election, and presence protocols. And you can build on it for your 
> own, specific needs.
>
> For ZooKeeper release details and downloads, visit:
> https://zookeeper.apache.org/releases.html
>
> ZooKeeper 3.6.3 Release Notes are at:
> https://zookeeper.apache.org/doc/r3.6.3/releasenotes.html
>
> We would like to thank the contributors that made the release possible.
>
> Regards,
>
> The ZooKeeper Team

[ANNOUNCE] Apache ZooKeeper 3.6.3 Release

[Mohammad Arshad]

The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.6.3

ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.

For ZooKeeper release details and downloads, visit:
https://zookeeper.apache.org/releases.html

ZooKeeper 3.6.3 Release Notes are at:
https://zookeeper.apache.org/doc/r3.6.3/releasenotes.html

We would like to thank the contributors that made the release possible.

Regards,

The ZooKeeper Team

RE: [ANNOUNCE] Enrico Olivelli new ZooKeeper PMC Member

[Mohammad arshad]

Congratulations Enrico!


-Original Message-
From: rammohan ganapavarapu [mailto:rammohanga...@gmail.com] 
Sent: Wednesday, January 22, 2020 3:45 AM
To: user@zookeeper.apache.org
Cc: DevZooKeeper 
Subject: Re: [ANNOUNCE] Enrico Olivelli new ZooKeeper PMC Member

Congratulations Enrico!!

On Tue, Jan 21, 2020 at 1:41 PM Flavio Junqueira  wrote:

> I'm pleased to announce that Enrico Olivelli recently became the 
> newest ZooKeeper PMC member. Enrico has contributed immensely to this 
> community; he became a ZooKeeper committer in May 2019 and now he joins the 
> PMC.
>
> Join me in congratulating him on the achievement. Congrats, Enrico!
>
> -Flavio on behalf of the Apache ZooKeeper PMC

RE: ZooKeeper clients does not handle new error codes properly

[Mohammad arshad]

Created ZOOKEEPER-2695 to handle this scenario by throwing 
KeeperException.SystemErrorException instead of IllegalArgumentException

Best Regards
Mohammad Arshad
HUAWEI TECHNOLOGIES CO.LTD.
Huawei Tecnologies India Pvt. Ltd.
Near EPIP Industrial Area, Kundalahalli Village
Whitefield, Bangalore-560066
www.huawei.com
-
This e-mail and its attachments contain confidential information from HUAWEI, 
which 
is intended only for the person or entity whose address is listed above. Any 
use of the 
information contained herein in any way (including, but not limited to, total 
or partial 
disclosure, reproduction, or dissemination) by persons other than the intended 
recipient(s) is prohibited. If you receive this e-mail in error, please notify 
the sender by 
phone or email immediately and delete it!

-Original Message-
From: Mohammad arshad 
Sent: 05 October 2016 19:34
To: d...@zookeeper.apache.org; UserZooKeeper
Subject: RE: ZooKeeper clients does not handle new error codes properly

Thanks Michael Han for the information.
It is the same place in my case as well.

-Original Message-
From: Michael Han [mailto:h...@cloudera.com]
Sent: 04 October 2016 23:08
To: UserZooKeeper
Cc: d...@zookeeper.apache.org
Subject: Re: ZooKeeper clients does not handle new error codes properly

Hi Arshad,

I am curious where the IllegalArgumentException was thrown in code in your use 
case - I am asking because I spotted that in KeeperException.create, we have a 
default fall through [1] that throws IllegalArgumentException, and if this is 
not the place where you observed the IllegalArgumentException in your upgrade 
use case, then we may also need think to change the default fall through here 
from throwing IllegalArgumentException to return something like 
SystemErrorException, because it seems possible that unknown / not mapped error 
codes can be passed to KeeperException.create that will lead to 
IllegalArgumentException that client can't handle well?

[1] https://github.com/apache/zookeeper/blob/master/src/
java/main/org/apache/zookeeper/KeeperException.java#L144

On Tue, Oct 4, 2016 at 8:21 AM, Mohammad arshad <mohammad.ars...@huawei.com>
wrote:

> Thanks Benjamin Reed,
> Client protocol version is 0, it is not changed. Also there is no 
> logic executed at the server side based on the client protocol version.
> Increasing the client protocol version and converting the errors at 
> server may be the ideal and more precise solution but to handle it in 
> a generic way can we convert the unknown error to 
> KeeperException.SystemErrorException
> at client side? Do you foresee any problem in this solution?
>
> Thanks
> -Arshad
>
>
> -Original Message-
> From: Benjamin Reed [mailto:br...@apache.org]
> Sent: 04 October 2016 10:27
> To: user@zookeeper.apache.org
> Cc: DevZooKeeper
> Subject: Re: ZooKeeper clients does not handle new error codes 
> properly
>
> did we bump the protocol version when we added the new errors? the 
> server could do the conversion when it responds to older clients.
>
> On Mon, Oct 3, 2016 at 3:05 AM, Flavio Junqueira <f...@apache.org> wrote:
>
> > Hi Arshad,
> >
> > It makes sense to me. What if we convert unknown server errors to 
> > KeeperException.SystemErrorException? This is a generic error and it 
> > extends KeeperException.
> >
> > I don't see it as a big issue to make this change, but others may 
> > feel differently. If we do it, then we will need a release note 
> > pointing out the change of behavior.
> >
> > -Flavio
> >
> > > On 03 Oct 2016, at 08:54, Mohammad arshad 
> > > <mohammad.ars...@huawei.com>
> > wrote:
> > >
> > > Hi All,
> > > In Zookeeper rolling upgrade scenario where server is new but 
> > > client is
> > old, when sever sends error code which is not understood by a 
> > client, client throws IllegalArgumentException. Generally 
> > IllegalArgumentException is not handled by any of the ZK applications.
> > It is too generic. How to handle this scenario in ZK applications?
> > > My understanding is instead of throwing IllegalArgumentException 
> > > we
> > should throw a subclass of KeeperException, for example 
> > InvalidErrorCodeException, so that zk apps can take more specific action.
> > > Any thoughts?
> > >
> > > Thanks
> > > -Arshad
> > >
> >
> >
>



--
Cheers
Michael.

Re: New 3.5.X release

[Mohammad Arshad]

Hi Jens
ZOOKEEPER-2464  if
fixed as part of ZOOKEEPER-2680 and the fix will be available in coming
release 3.5.3.
As of now, may be you can build the zookeeper from branch-3.5, use it and
give feedback on the fix.

Thanks
Arshad



On Sat, Feb 11, 2017 at 8:35 PM, Jens Rantil  wrote:

> Hello Zoo workers,
>
> We started hitting the NPE in [1] which lead to Zookeeper not cleaning up
> our znodes anymore. We've worked around the issue[2], but it's just a
> matter time before this happens again.
>
> A fix for [1] has been merge into the 1.5.X branch. I asked in the JIRA
> issue about when a new release with this fix could be out, but has so far
> not gotten an answer. Is there anyone on this mailing list who could answer
> this?
>
> [1] https://issues.apache.org/jira/browse/ZOOKEEPER-2464
> [2] Luckily we don't have any persistent data in our Zookeeper cluster so,
> bringing it down fully and removing all logs and snapshots fixed things for
> us.
>
> Thanks,
> Jens
>
> --
> Jens Rantil
> Backend engineer
> Tink AB
>
> Email: jens.ran...@tink.se
> Phone: +46 708 84 18 32
> Web: www.tink.se
>
> Facebook  Linkedin
>  companies_res_photo=VSRPsearchId%3A1057023381369207406670%
> 2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary>
>  Twitter 
>

RE: ZooKeeper clients does not handle new error codes properly

[Mohammad arshad]

Thanks Michael Han for the information.
It is the same place in my case as well.

-Original Message-
From: Michael Han [mailto:h...@cloudera.com] 
Sent: 04 October 2016 23:08
To: UserZooKeeper
Cc: d...@zookeeper.apache.org
Subject: Re: ZooKeeper clients does not handle new error codes properly

Hi Arshad,

I am curious where the IllegalArgumentException was thrown in code in your use 
case - I am asking because I spotted that in KeeperException.create, we have a 
default fall through [1] that throws IllegalArgumentException, and if this is 
not the place where you observed the IllegalArgumentException in your upgrade 
use case, then we may also need think to change the default fall through here 
from throwing IllegalArgumentException to return something like 
SystemErrorException, because it seems possible that unknown / not mapped error 
codes can be passed to KeeperException.create that will lead to 
IllegalArgumentException that client can't handle well?

[1] https://github.com/apache/zookeeper/blob/master/src/
java/main/org/apache/zookeeper/KeeperException.java#L144

On Tue, Oct 4, 2016 at 8:21 AM, Mohammad arshad <mohammad.ars...@huawei.com>
wrote:

> Thanks Benjamin Reed,
> Client protocol version is 0, it is not changed. Also there is no 
> logic executed at the server side based on the client protocol version.
> Increasing the client protocol version and converting the errors at 
> server may be the ideal and more precise solution but to handle it in 
> a generic way can we convert the unknown error to 
> KeeperException.SystemErrorException
> at client side? Do you foresee any problem in this solution?
>
> Thanks
> -Arshad
>
>
> -Original Message-
> From: Benjamin Reed [mailto:br...@apache.org]
> Sent: 04 October 2016 10:27
> To: user@zookeeper.apache.org
> Cc: DevZooKeeper
> Subject: Re: ZooKeeper clients does not handle new error codes 
> properly
>
> did we bump the protocol version when we added the new errors? the 
> server could do the conversion when it responds to older clients.
>
> On Mon, Oct 3, 2016 at 3:05 AM, Flavio Junqueira <f...@apache.org> wrote:
>
> > Hi Arshad,
> >
> > It makes sense to me. What if we convert unknown server errors to 
> > KeeperException.SystemErrorException? This is a generic error and it 
> > extends KeeperException.
> >
> > I don't see it as a big issue to make this change, but others may 
> > feel differently. If we do it, then we will need a release note 
> > pointing out the change of behavior.
> >
> > -Flavio
> >
> > > On 03 Oct 2016, at 08:54, Mohammad arshad 
> > > <mohammad.ars...@huawei.com>
> > wrote:
> > >
> > > Hi All,
> > > In Zookeeper rolling upgrade scenario where server is new but 
> > > client is
> > old, when sever sends error code which is not understood by a 
> > client, client throws IllegalArgumentException. Generally 
> > IllegalArgumentException is not handled by any of the ZK applications.
> > It is too generic. How to handle this scenario in ZK applications?
> > > My understanding is instead of throwing IllegalArgumentException 
> > > we
> > should throw a subclass of KeeperException, for example 
> > InvalidErrorCodeException, so that zk apps can take more specific action.
> > > Any thoughts?
> > >
> > > Thanks
> > > -Arshad
> > >
> >
> >
>



--
Cheers
Michael.

RE: ZooKeeper clients does not handle new error codes properly

[Mohammad arshad]

Thanks Benjamin Reed,
Client protocol version is 0, it is not changed. Also there is no logic 
executed at the server side based on the client protocol version. Increasing 
the client protocol version and converting the errors at server may be the 
ideal and more precise solution but to handle it in a generic way can we 
convert the unknown error to KeeperException.SystemErrorException at client 
side? Do you foresee any problem in this solution? 

Thanks
-Arshad


-Original Message-
From: Benjamin Reed [mailto:br...@apache.org] 
Sent: 04 October 2016 10:27
To: user@zookeeper.apache.org
Cc: DevZooKeeper
Subject: Re: ZooKeeper clients does not handle new error codes properly

did we bump the protocol version when we added the new errors? the server could 
do the conversion when it responds to older clients.

On Mon, Oct 3, 2016 at 3:05 AM, Flavio Junqueira <f...@apache.org> wrote:

> Hi Arshad,
>
> It makes sense to me. What if we convert unknown server errors to 
> KeeperException.SystemErrorException? This is a generic error and it 
> extends KeeperException.
>
> I don't see it as a big issue to make this change, but others may feel 
> differently. If we do it, then we will need a release note pointing 
> out the change of behavior.
>
> -Flavio
>
> > On 03 Oct 2016, at 08:54, Mohammad arshad 
> > <mohammad.ars...@huawei.com>
> wrote:
> >
> > Hi All,
> > In Zookeeper rolling upgrade scenario where server is new but client 
> > is
> old, when sever sends error code which is not understood by a client, 
> client throws IllegalArgumentException. Generally 
> IllegalArgumentException is not handled by any of the ZK applications. 
> It is too generic. How to handle this scenario in ZK applications?
> > My understanding is instead of throwing IllegalArgumentException we
> should throw a subclass of KeeperException, for example 
> InvalidErrorCodeException, so that zk apps can take more specific action.
> > Any thoughts?
> >
> > Thanks
> > -Arshad
> >
>
>

ZooKeeper ping requests Unnecessarily go though request processor chain

[Mohammad arshad]

Hi All
ZooKeeper clients send ping request(heartbeat) to ZooKeeper server to keep its 
session alive. These ping requests do nothing but touch its session on the 
server.

If client is connected to a follower then the ping request is processed in 
sequence of ServerCnxn --> ZooKeeperServer --> FollowerRequestProcessor --> 
CommitProcessor --> FinalRequestProcessor. The ping request will wait in 
CommitProcessor for previous request completion. This wait for ping request is 
unnecessary. I think it offers no benefit.

is ping request doing more than touching its session? I think it is only 
touching its session, not doing anything else.
If this is the case we should process the ping request differently from the 
other requests. It should be  treated as system request and should be processed 
with higher priority. May be we can process in the sequence of ServerCnxn --> 
ZooKeeperServer --> PingRequestProcessor

Any thought?

Thanks
-Arshad

ZooKeeper clients does not handle new error codes properly

[Mohammad arshad]

Hi All,
In Zookeeper rolling upgrade scenario where server is new but client is old, 
when sever sends error code which is not understood by a client,  client throws 
IllegalArgumentException. Generally IllegalArgumentException is not handled by 
any of the ZK applications. It is too generic. How to handle this scenario in 
ZK applications?
My understanding is instead of throwing IllegalArgumentException we should 
throw a subclass of KeeperException, for example InvalidErrorCodeException, so 
that zk apps can take more specific action.
Any thoughts?

Thanks
-Arshad

RE: ZooKeeper Sync API path parameter does not have any impact.

[Mohammad arshad]

Got it.
Thanks Flavio, Chris Nauroth.

-Original Message-
From: Chris Nauroth [mailto:cnaur...@hortonworks.com] 
Sent: 07 September 2016 22:24
To: d...@zookeeper.apache.org; user@zookeeper.apache.org
Subject: Re: ZooKeeper Sync API path parameter does not have any impact.

Yes, the sync operation really applies to the client’s entire view of ZooKeeper 
state, not a specific path or sub-tree, so I would not expect to see any 
significant difference in time for sync(/path1) vs. sync(/path2).  The presence 
of the path is only a client API convenience.

Expanding a bit on what Flavio already said, the reason a client calls sync is 
often because it wants to read up-to-date state for a specific path.  If an 
application is doing this kind of logic for multiple paths, then it can check 
the path in AsyncCallback.VoidCallback#processResult and know that this is a 
completed sync in preparation for reading /path1 vs. /path2.

I woudn’t expect any of this to influence execution time of the sync operation 
though.  If you do see a difference in execution time, then that’s likely just 
a coincidence.  (Maybe it’s just a coincidence that there was more write 
activity at the time of calling sync(/path1) vs. sync(/path2), and therefore 
one of those sync calls needed to do more work to catch up with all of the 
transactions.)

--Chris Nauroth

On 9/7/16, 8:47 AM, "Mohammad arshad" <mohammad.ars...@huawei.com> wrote:

So path is completely for ZooKeeper client. It has nothing to do with 
server. Therefore whether I execute sync /path1 or sync /path2, sync request 
will complete in the same time. right?

-Arshad
-Original Message-
From: Flavio Junqueira [mailto:f...@apache.org] 
Sent: 07 September 2016 16:45
To: user@zookeeper.apache.org
Cc: d...@zookeeper.apache.org
Subject: Re: ZooKeeper Sync API path parameter does not have any impact.

The path parameter helps the app determine the path that the response is 
referring to. A sync call is typically associated to a read request to a given 
path, so the path in the sync call helps the application make the association 
to that path.

-Flavio

> On 07 Sep 2016, at 11:35, Mohammad arshad <mohammad.ars...@huawei.com> 
wrote:
> 
> Hello Everyone
> ZooKeeper sync API takes a path argument.
> But it seems this path is no where used in the server code. Same path is 
returned from the server to client again in SyncResponse.
> Is the any other purpose of sync API path parameter.
> Will there be any different impact for below sync commands form zk Cli?
> sync /path1
> sync /path2
> 
> Thanks
> -Arshad

RE: ZooKeeper Sync API path parameter does not have any impact.

[Mohammad arshad]

So path is completely for ZooKeeper client. It has nothing to do with server. 
Therefore whether I execute sync /path1 or sync /path2, sync request will 
complete in the same time. right?

-Arshad
-Original Message-
From: Flavio Junqueira [mailto:f...@apache.org] 
Sent: 07 September 2016 16:45
To: user@zookeeper.apache.org
Cc: d...@zookeeper.apache.org
Subject: Re: ZooKeeper Sync API path parameter does not have any impact.

The path parameter helps the app determine the path that the response is 
referring to. A sync call is typically associated to a read request to a given 
path, so the path in the sync call helps the application make the association 
to that path.

-Flavio

> On 07 Sep 2016, at 11:35, Mohammad arshad <mohammad.ars...@huawei.com> wrote:
> 
> Hello Everyone
> ZooKeeper sync API takes a path argument.
> But it seems this path is no where used in the server code. Same path is 
> returned from the server to client again in SyncResponse.
> Is the any other purpose of sync API path parameter.
> Will there be any different impact for below sync commands form zk Cli?
> sync /path1
> sync /path2
> 
> Thanks
> -Arshad

subscribe user@zookeeper.apache.org

[Mohammad arshad]

ZooKeeper Sync API path parameter does not have any impact.

[Mohammad arshad]

Hello Everyone
ZooKeeper sync API takes a path argument.
But it seems this path is no where used in the server code. Same path is 
returned from the server to client again in SyncResponse.
Is the any other purpose of sync API path parameter.
Will there be any different impact for below sync commands form zk Cli?
sync /path1
sync /path2

Thanks
-Arshad

RE: How to extend ZooKeeper data structure without breaking the backward compatibility

[Mohammad arshad]

Loading snapshot file into memory(deserialization) does not involve any 
protocol version.
are you suggesting version and dbId from FileHeader?
Which field (version, dbId) would be more appropriate to decide which version 
,StatPersisted or StatPersistedV0, to be deserialized. I think it should be the 
dbId.

Best Regards
Mohammad Arshad
HUAWEI TECHNOLOGIES CO.LTD.
Huawei Tecnologies India Pvt. Ltd.
Near EPIP Industrial Area, Kundalahalli Village
Whitefield, Bangalore-560066
www.huawei.com
-
This e-mail and its attachments contain confidential information from HUAWEI, 
which 
is intended only for the person or entity whose address is listed above. Any 
use of the 
information contained herein in any way (including, but not limited to, total 
or partial 
disclosure, reproduction, or dissemination) by persons other than the intended 
recipient(s) is prohibited. If you receive this e-mail in error, please notify 
the sender by 
phone or email immediately and delete it!


-Original Message-
From: Flavio Junqueira [mailto:f...@apache.org] 
Sent: 11 May 2016 18:18
To: user@zookeeper.apache.org
Cc: d...@zookeeper.apache.org
Subject: Re: How to extend ZooKeeper data structure without breaking the 
backward compatibility

In this case, I guess we have to check the protocol version and depending on 
the version deserialize differently.

-Flavio

> On 11 May 2016, at 13:21, Mohammad arshad <mohammad.ars...@huawei.com> wrote:
> 
> Thanks Flavio,
> 
> I think CreateTxn kind of solution cannot be applied in this scenario 
> I can case of transaction the records are separable but in case of 
> snapshot records are continuous In this scenario, cuser 
> de-serialization eats into the next record which causes bigger problem
> 
> Here is exception I got after adding cuser and running the ZooKeeper 
> on old data
> java.io.IOException: Unreasonable length = 796553071
>   at 
> org.apache.jute.BinaryInputArchive.checkLength(BinaryInputArchive.java:127)
>   at 
> org.apache.jute.BinaryInputArchive.readString(BinaryInputArchive.java:81)
>   at 
> org.apache.zookeeper.data.StatPersisted.deserialize(StatPersisted.java:143)
>   at org.apache.zookeeper.server.DataNode.deserialize(DataNode.java:173)
>   at 
> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99)
>   at 
> org.apache.zookeeper.server.DataTree.deserialize(DataTree.java:1278)
> 
> 
> Best Regards
> Mohammad Arshad
> HUAWEI TECHNOLOGIES CO.LTD.
> Huawei Tecnologies India Pvt. Ltd.
> Near EPIP Industrial Area, Kundalahalli Village Whitefield, 
> Bangalore-560066 www.huawei.com
> --
> ---
> This e-mail and its attachments contain confidential information from 
> HUAWEI, which is intended only for the person or entity whose address 
> is listed above. Any use of the information contained herein in any 
> way (including, but not limited to, total or partial disclosure, 
> reproduction, or dissemination) by persons other than the intended
> recipient(s) is prohibited. If you receive this e-mail in error, 
> please notify the sender by phone or email immediately and delete it!
> 
> -Original Message-
> From: Flavio Junqueira [mailto:f...@apache.org]
> Sent: 11 May 2016 16:02
> To: user@zookeeper.apache.org
> Cc: Zookeeper
> Subject: Re: How to extend ZooKeeper data structure without breaking 
> the backward compatibility
> 
> In the past, we simply created two versions of the data structure:
> 
>class CreateTxnV0 {
>ustring path;
>buffer data;
>vector acl;
>boolean ephemeral;
>}
>class CreateTxn {
>ustring path;
>buffer data;
>vector acl;
>boolean ephemeral;
>    int parentCVersion;
>}
> 
> and deal with it in the code. It is not ideal and the serialization framework 
> is actually a pretty old one.
> 
> -Flavio
> 
>> On 11 May 2016, at 11:14, Mohammad arshad <mohammad.ars...@huawei.com> wrote:
>> 
>> Hello Everyone
>> 
>> is there any way to extend ZooKeeper data structure without breaking the 
>> backward compatibility.
>> Suppose I want to add cuser field in StatPersisted class 
>> StatPersisted {
>>   
>>   ustring cuser;//user who create the node
>>   }
>> This is fine with fresh ZooKeeper server installation, where 
>> ZooKeeper data is created freshly But this causes problem while 
>> de-serializing when server is new but points to old ZooKeeper data.
>> Here by ZooKeeper data I

RE: How to extend ZooKeeper data structure without breaking the backward compatibility

[Mohammad arshad]

Thanks Flavio,

I think CreateTxn kind of solution cannot be applied in this scenario
I can case of transaction the records are separable but in case of snapshot 
records are continuous
In this scenario, cuser de-serialization eats into the next record which causes 
bigger problem

Here is exception I got after adding cuser and running the ZooKeeper on old data
java.io.IOException: Unreasonable length = 796553071
at 
org.apache.jute.BinaryInputArchive.checkLength(BinaryInputArchive.java:127)
at 
org.apache.jute.BinaryInputArchive.readString(BinaryInputArchive.java:81)
at 
org.apache.zookeeper.data.StatPersisted.deserialize(StatPersisted.java:143)
at org.apache.zookeeper.server.DataNode.deserialize(DataNode.java:173)
at 
org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99)
at org.apache.zookeeper.server.DataTree.deserialize(DataTree.java:1278)


Best Regards
Mohammad Arshad
HUAWEI TECHNOLOGIES CO.LTD.
Huawei Tecnologies India Pvt. Ltd.
Near EPIP Industrial Area, Kundalahalli Village
Whitefield, Bangalore-560066
www.huawei.com
-
This e-mail and its attachments contain confidential information from HUAWEI, 
which 
is intended only for the person or entity whose address is listed above. Any 
use of the 
information contained herein in any way (including, but not limited to, total 
or partial 
disclosure, reproduction, or dissemination) by persons other than the intended 
recipient(s) is prohibited. If you receive this e-mail in error, please notify 
the sender by 
phone or email immediately and delete it!

-Original Message-
From: Flavio Junqueira [mailto:f...@apache.org] 
Sent: 11 May 2016 16:02
To: user@zookeeper.apache.org
Cc: Zookeeper
Subject: Re: How to extend ZooKeeper data structure without breaking the 
backward compatibility

In the past, we simply created two versions of the data structure:

class CreateTxnV0 {
ustring path;
buffer data;
vector acl;
boolean ephemeral;
}
class CreateTxn {
ustring path;
buffer data;
vector acl;
boolean ephemeral;
int parentCVersion;
}

and deal with it in the code. It is not ideal and the serialization framework 
is actually a pretty old one.

-Flavio

> On 11 May 2016, at 11:14, Mohammad arshad <mohammad.ars...@huawei.com> wrote:
> 
> Hello Everyone
> 
> is there any way to extend ZooKeeper data structure without breaking the 
> backward compatibility.
> Suppose I want to add cuser field in StatPersisted class StatPersisted 
> {
>
>ustring cuser;//user who create the node
>}
> This is fine with fresh ZooKeeper server installation, where ZooKeeper 
> data is created freshly But this causes problem while de-serializing when 
> server is new but points to old ZooKeeper data.
> Here by ZooKeeper data I mean snapshot and transaction log
> 
> In protocol buffers a field can be optional but in jute optional field is not 
> supported.
> is there any way to overcome this jute limitation? any work around?
> 
> Best Regards
> Mohammad Arshad
> HUAWEI TECHNOLOGIES CO.LTD.
> Huawei Tecnologies India Pvt. Ltd.
> Near EPIP Industrial Area, Kundalahalli Village Whitefield, 
> Bangalore-560066 www.huawei.com<http://www.huawei.com/>
> --
> ---
> This e-mail and its attachments contain confidential information from 
> HUAWEI, which is intended only for the person or entity whose address 
> is listed above. Any use of the information contained herein in any 
> way (including, but not limited to, total or partial disclosure, 
> reproduction, or dissemination) by persons other than the intended
> recipient(s) is prohibited. If you receive this e-mail in error, 
> please notify the sender by phone or email immediately and delete it!
> 

How to extend ZooKeeper data structure without breaking the backward compatibility

[Mohammad arshad]

Hello Everyone

is there any way to extend ZooKeeper data structure without breaking the 
backward compatibility.
Suppose I want to add cuser field in StatPersisted
class StatPersisted {

ustring cuser;//user who create the node
}
This is fine with fresh ZooKeeper server installation, where ZooKeeper data is 
created freshly
But this causes problem while de-serializing when server is new but points to 
old ZooKeeper data.
Here by ZooKeeper data I mean snapshot and transaction log

In protocol buffers a field can be optional but in jute optional field is not 
supported.
is there any way to overcome this jute limitation? any work around?

Best Regards
Mohammad Arshad
HUAWEI TECHNOLOGIES CO.LTD.
Huawei Tecnologies India Pvt. Ltd.
Near EPIP Industrial Area, Kundalahalli Village
Whitefield, Bangalore-560066
www.huawei.com<http://www.huawei.com/>
-
This e-mail and its attachments contain confidential information from HUAWEI, 
which
is intended only for the person or entity whose address is listed above. Any 
use of the
information contained herein in any way (including, but not limited to, total 
or partial
disclosure, reproduction, or dissemination) by persons other than the intended
recipient(s) is prohibited. If you receive this e-mail in error, please notify 
the sender by
phone or email immediately and delete it!

RE: How to handle zookeeper data inconsistency

[Mohammad arshad]

Thanks for Flavio Junqueira for your response.

assume C received the commit request but before committing it failed, When C 
will be synced? What event will at leader or follower will synch it up.

Here is another scenario we faced.
Node got deleted successfully in leader node B. But due to network issue in 
Master node, the delete could not sync up to follower A and C.  At this moment, 
Leader node also goes down as faulty. 

Now one of the A and C becomes leader but it has inconsistent data. ( delete is 
not executed here)

As I know, This behavior is fine as per current ZK design. But to solve above 
data inconsistency issue, any suggestions ? I thought to commit the delete not 
only in leader but to at least in N/2 nodes in the same client call and then 
only mark delete as successful

Any suggestions ?


Best Regards
Mohammad Arshad

-Original Message-
From: Flavio Junqueira [mailto:f...@apache.org] 
Sent: 21 January 2016 19:11
To: user@zookeeper.apache.org
Cc: dev
Subject: Re: How to handle zookeeper data inconsistency

Hi Mohammad,

A delete operation only needs to reach a quorum to complete and A B form a 
quorum in your 3-server ensemble. If the delete operation never gets propagated 
to C and other write operations that have been ordered later complete on C, 
then you have an issue. If C simply stops receiving updates, then you have a 
problem with your C server and it could be a problem with ZK or just the 
environment.

If there has been write operations ordered after the delete and server C has 
seen those but not the delete, then I'd recommend that you have a look the txn 
logs with the log formatter.

> shall I check exists from leader only? but even leader can have some 
> node undeleted in the above scenario

There is no such a requirement, but you need to be aware that server C could 
definitely make an update visible later compared to other servers. ZooKeeper 
doesn't guarantee that updates are visible to all clients as soon as they are 
acknowledged.

I'd also search for jiras, especially if you're deleting an ephemeral. 

-Flavio

> On 21 Jan 2016, at 13:24, Mohammad arshad <mohammad.ars...@huawei.com> wrote:
> 
> Hi All,
> I came across a scenario where zookeeper was left in inconsistent 
> state(but that is valid as per the zookeeper theory) and because of 
> that dependent application's behaved wrongly The scenario is as follow
> 
> 1) I have three server Zookeeper cluster, let's say servers are A, B 
> and C. B is the leader
> 2) In one successful delete operation, a znode znode1 was deleted from A and 
> B but somehow not deleted from C. The reason for not deleted from C can be 
> either proposal or commit failed.
> 3) Now for application, which is connected to C, ZooKeeper.exists  
> returns the znod1 and that is why application enters into node exists 
> flow which is wrong
> 
> shall I check exists from leader only? but even leader can have some 
> node undeleted in the above scenario Any guideline to handle the above said 
> valid data inconsistency ??
> 
> Any suggestion/help is highly appreciated.
> 
> Best Regards
> Mohammad Arshad
> HUAWEI TECHNOLOGIES CO.LTD.
> Huawei Tecnologies India Pvt. Ltd.
> Near EPIP Industrial Area, Kundalahalli Village Whitefield, 
> Bangalore-560066 www.huawei.com<http://www.huawei.com/>
> --
> ---
> This e-mail and its attachments contain confidential information from 
> HUAWEI, which is intended only for the person or entity whose address 
> is listed above. Any use of the information contained herein in any 
> way (including, but not limited to, total or partial disclosure, 
> reproduction, or dissemination) by persons other than the intended
> recipient(s) is prohibited. If you receive this e-mail in error, 
> please notify the sender by phone or email immediately and delete it!
> 

Unncessary Jline logs in zk client Consle

[Mohammad arshad]

Hi  All,
Every time I connect to zookeeper through console zkCli.sh, I get following logs

[INFO] Unable to bind key for unsupported operation: backward-delete-word
[INFO] Unable to bind key for unsupported operation: backward-delete-word
[INFO] Unable to bind key for unsupported operation: down-history
[INFO] Unable to bind key for unsupported operation: up-history
[INFO] Unable to bind key for unsupported operation: up-history
[INFO] Unable to bind key for unsupported operation: down-history
[INFO] Unable to bind key for unsupported operation: up-history
[INFO] Unable to bind key for unsupported operation: down-history
[INFO] Unable to bind key for unsupported operation: up-history
[INFO] Unable to bind key for unsupported operation: down-history
[INFO] Unable to bind key for unsupported operation: up-history
[INFO] Unable to bind key for unsupported operation: down-history


The log is coming from jline.jar third party jar.  JLine is a Java library for 
handling console input and here mainly used for command completion

This library is not using log4j or any other logging library. It is directly  
printing logs to console
Also it is not providing any configuration to control the logging level.
Log is being generated from jline.console.ConsoleKeys and finally printed by 
jline.internal.Log to console



Is there any way to filter above logs without modifying jline jar?



Thanks  Regards
Mohammad Arshad