Re: PR Validation on Github Actions is very flaky

[Justin Ling Mao]

Let me take a look. I'm filing the java flaky test happened recently. I will 
investigate the CPP test issue.Although I almost forget my C language, but you 
know I am almighty :

- Original Message -
From: Enrico Olivelli 
To: DevZooKeeper 
Subject: PR Validation on Github Actions is very flaky
Date: 2021-06-15 23:40

Hello,
I see that CI precommit validation often fails on GitHub actions,
especially the execution of CPP tests.
Is there anyone who can take a look, investigate and create JIRAs ?
Sending out a fix would be great but from my understanding we still
haven't tracked the failures in JIRA.
Cheers
Enrico

Re: Re: [ANNOUNCE] Apache ZooKeeper 3.7.0 released

[Justin Ling Mao]

Awesome!!!
Thanks Damien for working as RM, thank the contributors that made the release 
possible. 
Cheers!!!
- Original Message -
From: Szalay-Bekő Máté 
To: DevZooKeeper 
Cc: annou...@apache.org, UserZooKeeper 
Subject: Re: [ANNOUNCE] Apache ZooKeeper 3.7.0 released
Date: 2021-03-29 02:51

Thanks for all your work Damien, and also for all the contributors and for
the whole community.
It's great to see 3.7.0 out! :)
Cheers,
Mate
On Sun, Mar 28, 2021 at 9:39 AM Damien Diederen 
wrote:
>
> The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
> 3.7.0.
>
> ZooKeeper is a high-performance coordination service for distributed
> applications. It exposes common services - such as naming,
> configuration management, synchronization, and group services - in a
> simple interface so you don't have to write them from scratch. You can
> use it off-the-shelf to implement consensus, group management, leader
> election, and presence protocols. And you can build on it for your
> own, specific needs.
>
> For ZooKeeper release details and downloads, visit:
> https://zookeeper.apache.org/releases.html
>
> ZooKeeper 3.7.0 Release Notes are at:
> https://zookeeper.apache.org/doc/r3.7.0/releasenotes.html
>
> We would like to thank the contributors that made the release possible.
>
> Regards,
>
> The ZooKeeper Team
>

Re: Re: quota stats data consistency

[Justin Ling Mao]

Thanks Li Wang for your question. If you have more questions, please ask again 
here:)

- Original Message -
From: Li Wang 
To: dev 
Subject: Re: quota stats data consistency
Date: 2021-03-02 05:07

Some updates on this.
I looked more into this and found that quota usage is actually
re-calculated when deserializing the data tree from the snapshot,  so the
quota data will be consistent even in  the case zk server crashes.
Best,
Li
On Tue, Feb 23, 2021 at 10:13 AM Li Wang  wrote:
>
> Hi,
>
> The Quota stats are updated by calling updateQuotaStat() API in the
> FinalRequestProcessor. However, the updateQuotaStat() API only updates the
> in-memory data tree.
>
> In the scenario that a ZK server crashed right before the updateQuotaStat
> is called or a snapshot was taken, are we going to have the following data
> inconsistency issue?
>
> 1. the quota stats is  inaccurate in the crashed server as the
> write transaction has been logged by  the SyncRequestProcessor but the
> quota stats update was lost?
>
> 2. the quota stats will be inconsistent across different servers as the
> quota stats persisted in other servers via snapshot but not the crashed one
>
>
> updateQuotaStat() code snippet
> 
> synchronized (statNode) {
> updatedStat = new StatsTrack(statNode.data);
> updatedStat.setCount(updatedStat.getCount() + countDiff);
> updatedStat.setBytes(updatedStat.getBytes() + bytesDiff);
>
> statNode.data = updatedStat.getStatsBytes();
> }
>
> Any thoughts?
>
> Thanks,
>
> Li
>

Re: Re: Re: [Commit Accident Case Study] Commit 4faf507 broke the build

[Justin Ling Mao]

I have created the PR(https://github.com/apache/zookeeper/pull/1610) to add the 
committer checklist to commit script to avoid The Accident Case like: PR-1597. 
Please review it and provide more checklists if I have not covered.


- Original Message -
From: "Justin Ling Mao" 
To: "dev" 
Cc: "andor" 
Subject: Re: Re: [Commit Accident Case Study] Commit 4faf507 broke the build
Date: 2021-02-10 17:26

Haha, it scared me. Let me go through this accident.
The root cause is: I'm over-confident, frivolous and hasty. I flatter myself 
that it's just a typo and committing it could not have anything bad happens. 
And I also don't give this PR a buffer time for other people's review.
Accident is bad, but it's much more terrible if we can not reflect on it and 
think about how to avoid it next time.
For remedy:
I will add a new section: Commit Accident Case Study in [1] for the successor’s 
learning (Can anyone give me the permission to edit that wiki)?
I will sum up our commit rules and the checklists before committing one patch, 
and do some works to use the Github CI and commit script to protect/check these 
constraint.
For punishment:
I will frozen/forbid my committership permission for three months(02-10 ~ 
05-10). During this period, I must not commit anything. I wish I could reflect 
on my fault and have a better understanding on the wording: "With great power 
comes great responsibility"

Reference:[1] 
https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute
- Original Message -
From: Andor Molnar 
To: maoling199210...@sina.com
Cc: dev 
Subject: Re: Commit 4faf507 broke the build
Date: 2021-02-10 00:26

I’m sorry Justin. There’s no excuse for a mistake like this. We should not show 
mercy for anybody, otherwise it would erode the trust in our community. Your 
committership is now revoked.
Just kidding. Don’t worry at all. ;-)
I reverted the patch, so now please create a new PR with all the required 
changes included.
Also I second Enrico’s comment: if CI is in bad shape, we should fix it.
Regards,
Andor
> On 2021. Feb 9., at 13:45, Justin Ling Mao  wrote:
> 
> Oops, it's my blame. I'm very sorry for my mistakes. Since these days the CI 
> is in disorder and it's a typo, so I'm not waiting for CI check and forgot 
> that an UT has covered this change although I wrote these related codes. It's 
> all my mistake and I will summarize our submission process and this accident. 
> I will write another letter to discuss the commit rules and how to improve 
> our code review throughput
> 
> 
> - Original Message -
> From: Andor Molnar 
> To: DevZooKeeper 
> Subject: Commit 4faf507 broke the build
> Date: 2021-02-09 19:43
> 
> Hi,
> I noticed that the latest commit 4faf507 ZOOKEEPER-4007: A typo in the 
> ZKUtil#validateFileInput method broke the build, because the unit test has 
> not been amended.
> I reverted the commit to fix the build. Please create new PR with a proper 
> patch.
> Has the committer verified that the build is green before submitting it?
> Andor

Re: collecting and exposing quota related metrics to Prometheus

[Justin Ling Mao]

@Li Wang
Great addition, we really need this. Please help us to do this:)


- Original Message -
From: Li Wang 
To: dev@zookeeper.apache.org
Subject: collecting and exposing quota related metrics to Prometheus
Date: 2021-02-16 09:23

Hi,
In 3.7, Quota limit can be enforced and the quota related stats are
captured in the StatsTrack.  From the "*listquota*" CLI command, we can get
the info about the quota limit and usage.
From the monitoring and alert points of view, we would like to collect the
quota metrics/stats and expose them to Prometheus so alert can be created
based on the quota level (e.g. 90% used).  I wonder if this feature has
been supported or there is a plan to support it?
If not, I will open a JIRA and can start to look into it.
Thanks,
Li

Re: Re: [Commit Accident Case Study] Commit 4faf507 broke the build

[Justin Ling Mao]

Haha, it scared me. Let me go through this accident.
The root cause is: I'm over-confident, frivolous and hasty. I flatter myself 
that it's just a typo and committing it could not have anything bad happens. 
And I also don't give this PR a buffer time for other people's review.
Accident is bad, but it's much more terrible if we can not reflect on it and 
think about how to avoid it next time.
For remedy:
I will add a new section: Commit Accident Case Study in [1] for the successor’s 
learning (Can anyone give me the permission to edit that wiki)?
I will sum up our commit rules and the checklists before committing one patch, 
and do some works to use the Github CI and commit script to protect/check these 
constraint.
For punishment:
I will frozen/forbid my committership permission for three months(02-10 ~ 
05-10). During this period, I must not commit anything. I wish I could reflect 
on my fault and have a better understanding on the wording: "With great power 
comes great responsibility"

Reference:[1] 
https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute
- Original Message -
From: Andor Molnar 
To: maoling199210...@sina.com
Cc: dev 
Subject: Re: Commit 4faf507 broke the build
Date: 2021-02-10 00:26

I’m sorry Justin. There’s no excuse for a mistake like this. We should not show 
mercy for anybody, otherwise it would erode the trust in our community. Your 
committership is now revoked.
Just kidding. Don’t worry at all. ;-)
I reverted the patch, so now please create a new PR with all the required 
changes included.
Also I second Enrico’s comment: if CI is in bad shape, we should fix it.
Regards,
Andor
> On 2021. Feb 9., at 13:45, Justin Ling Mao  wrote:
> 
> Oops, it's my blame. I'm very sorry for my mistakes. Since these days the CI 
> is in disorder and it's a typo, so I'm not waiting for CI check and forgot 
> that an UT has covered this change although I wrote these related codes. It's 
> all my mistake and I will summarize our submission process and this accident. 
> I will write another letter to discuss the commit rules and how to improve 
> our code review throughput
> 
> 
> - Original Message -
> From: Andor Molnar 
> To: DevZooKeeper 
> Subject: Commit 4faf507 broke the build
> Date: 2021-02-09 19:43
> 
> Hi,
> I noticed that the latest commit 4faf507 ZOOKEEPER-4007: A typo in the 
> ZKUtil#validateFileInput method broke the build, because the unit test has 
> not been amended.
> I reverted the commit to fix the build. Please create new PR with a proper 
> patch.
> Has the committer verified that the build is green before submitting it?
> Andor

Re: Commit 4faf507 broke the build

[Justin Ling Mao]

Oops, it's my blame. I'm very sorry for my mistakes. Since these days the CI is 
in disorder and it's a typo, so I'm not waiting for CI check and forgot that an 
UT has covered this change although I wrote these related codes. It's all my 
mistake and I will summarize our submission process and this accident. I will 
write another letter to discuss the commit rules and how to improve our code 
review throughput

- Original Message -
From: Andor Molnar 
To: DevZooKeeper 
Subject: Commit 4faf507 broke the build
Date: 2021-02-09 19:43

Hi,
I noticed that the latest commit 4faf507 ZOOKEEPER-4007: A typo in the 
ZKUtil#validateFileInput method broke the build, because the unit test has not 
been amended.
I reverted the commit to fix the build. Please create new PR with a proper 
patch.
Has the committer verified that the build is green before submitting it?
Andor

Re: [ZOOKEEPER-3922] rebase and retest on github

[Justin Ling Mao]

Hi ben, for [PR-1444](https://github.com/apache/zookeeper/pull/1444)AFAIK, you 
can let chingchan1996 grant write access for you and you'll be able to push 
changes on this branch. Look at an example of 
[PR-984](https://github.com/apache/zookeeper/pull/984)or you can create another 
PR to replace/continue that work.

- Original Message -
From: Benjamin Reed 
To: DevZooKeeper 
Subject: rebase and retest on github
Date: 2021-01-30 10:32

i would really like to get ZOOKEEPER-3922 but it needs to be rebased
and retested. is there a nice way to do that on github? or does the
pull requestor need to do that?
happy new year,
ben

Re: Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 0

[Justin Ling Mao]

I create the ticket: ZOOKEEPER-4188: add a doc about whoami CLI for me to do. 
It's not a blocker. Let's go ahead:)

- Original Message -
From: Patrick Hunt 
To: DevZooKeeper 
Subject: Re: [VOTE] Apache ZooKeeper release 3.7.0 candidate 0
Date: 2021-01-23 05:09

+1 - xsum/sig validated. Compiles/runs fine on macos+jdk11. Verified some
larger ensemble sizes manually and it worked ok.
I looked at a few of the new features listed - they look great! I did
notice some changes without documentation though (whoami eg), would be good
for committers to ensure that docs get updated along the way...
Thanks Damien for acting as RM. Regards,
Patrick
On Tue, Jan 19, 2021 at 4:40 AM Damien Diederen 
wrote:
>
> Dear all,
>
> This is a first release candidate for ZooKeeper 3.7.0.
>
> It introduces a number of new features, notably:
>
>   * An API to start a ZooKeeper server from Java (ZOOKEEPER-3874);
>
>   * Quota enforcement (ZOOKEEPER-3301);
>
>   * Host name canonicalization in quorum SASL authentication
> (ZOOKEEPER-4030);
>
>   * Support for BCFKS key/trust store format (ZOOKEEPER-3950);
>
>   * A choice of mandatory authentication scheme(s) (ZOOKEEPER-3561);
>
>   * A "whoami" API and CLI command (ZOOKEEPER-3969);
>
>   * The possibility of disabling digest authentication (ZOOKEEPER-3979);
>
>   * Multiple SASL "superUsers" (ZOOKEEPER-3959);
>
>   * Fast-tracking of throttled requests (ZOOKEEPER-3683);
>
>   * Additional security metrics (ZOOKEEPER-3978);
>
>   * SASL support in the C and Perl clients (ZOOKEEPER-1112,
> ZOOKEEPER-3714);
>
>   * A new zkSnapshotComparer.sh tool (ZOOKEEPER-3427);
>
>   * Notes on how to benchmark ZooKeeper with the YCSB tool
> (ZOOKEEPER-3264).
>
> The release notes are available here:
>
>
> https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-0/website/releasenotes.html
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12346617
>
> *** Please download, test and vote by January 24th 2020, 23:59 UTC+0. ***
>
> Source files:
>
>   https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-0/
>
> Maven staging repo:
>
>
> https://repository.apache.org/content/repositories/orgapachezookeeper-1065/
>
> The release candidate tag in git to be voted upon: release-3.7.0-0
>
>   https://github.com/apache/zookeeper/tree/release-3.7.0-0
>
> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
>
>   https://www.apache.org/dist/zookeeper/KEYS
>
> The staging version of the website is:
>
>
> https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-0/website/
>
> Should we release this candidate?
>
> Damien Diederen
>
>

Re: Re: Rate Quota

[Justin Ling Mao]

Here is a initial roadmap about the quote 
feature(https://cwiki.apache.org/confluence/display/ZOOKEEPER/Quotas).
We will nudge this work continuously. Here is the FB's 
video.(https://www.facebook.com/zkmeetup/videos/2984998221531756/) 

- Original Message -
From: Huizhi Lu 
To: dev@zookeeper.apache.org
Subject: Re: Rate Quota
Date: 2021-01-23 10:26

Thanks, Nobert, for the kind reply :)
Li, here is the picture (I've uploaded it to github in case you can't
see attachment):
https://user-images.githubusercontent.com/5187721/105566039-cdca1480-5cde-11eb-8d8e-e28cb9c0c273.png
We can keep following up on this feature. I think it's quite useful.
On Fri, Jan 22, 2021 at 6:08 PM Li Wang  wrote:
>
> Hi Huizhi,
>
> I am also looking for this rate quota per client feature. Would you mind
> sending again the picture from the Facebook presentation, as I didn't see
> that attachment.
>
> Thanks,
>
> Li
>
> On Wed, Jan 20, 2021 at 2:41 AM Norbert Kalmar 
> wrote:
>
> > Sorry, I missed the point there.
> > Unfortunately I don't know the timeline for this feature. Our friends at
> > Facebook weren't active lately, but hope you'll get an answer (I see you
> > also asked on the jira).
> >
> > - Norbert
> >
> > On Tue, Jan 19, 2021 at 8:33 PM Huizhi Lu  wrote:
> >
> > > Hi Norbert,
> > >
> > > Thanks for the reply. I understand request throttling was added in the
> > > ticket ZOOKEEPER-3243. My understanding is that it's a global request
> > > throttling: total requests for all clients. But it is not a per client
> > > level throttling.
> > >
> > > Just realized that the facebook presentation listed the zk ticket:
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-3467. I think this is
> > > what I'm looking for: reads/s and writes/s for per client. The feature
> > > is not yet available. I wonder if there is a timeline for it. Thanks!
> > >
> > >
> > > -Huizhi
> > >
> > > On Tue, Jan 19, 2021 at 2:20 AM Norbert Kalmar
> > >  wrote:
> > > >
> > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3243
> > > >
> > > > On Tue, Jan 19, 2021 at 11:19 AM Norbert Kalmar 
> > > > wrote:
> > > >
> > > > > Hi Huizhi,
> > > > >
> > > > > You might be looking for ZOOKEEPER-3243, it is available from 3.6.0.
> > > > >
> > > > > Regards,
> > > > > Norbert
> > > > >
> > > > > On Tue, Jan 19, 2021 at 9:49 AM Huizhi Lu  wrote:
> > > > >
> > > > >> Hey ZK Experts,
> > > > >>
> > > > >> I am actively looking into the request throttling/quota feature. I
> > > > >> recalled that at the zookeeper meetup, Facebook presented a rate
> > quota
> > > > >> per client (see attached pic) feature. But I can't find it in any zk
> > > > >> doc or code. Is the feature available? If not, is there any reason?
> > > > >>
> > > > >>
> > > > >> Thanks!
> > > > >> -Huizhi
> > > > >>
> > > > >
> > >
> >

Re: Re: New committer: Justin Ling Mao

[Justin Ling Mao]

Thanks all for your congratulations. I wish I could do much more works for our 
community in the future.

- Original Message -
From: Szalay-Bekő Máté 
To: DevZooKeeper 
Subject: Re: New committer: Justin Mao Ling
Date: 2021-01-18 20:27

Congratulations Maoling!!! :))
Regards,
Mate
On Mon, Jan 18, 2021 at 11:48 AM Norbert Kalmar
 wrote:
> Congratulations Maoling! Well-deserved!
>
> - Norbert
>
> On Mon, Jan 18, 2021 at 11:43 AM Andor Molnar  wrote:
>
> > Congrats Maoling!
> >
> >
> >
> >
> > > On 2021. Jan 18., at 11:09, Enrico Olivelli 
> wrote:
> > >
> > > The Project Management Committee (PMC) for Apache ZooKeeper
> > >
> > > has invited Justin Mao Long to become a committer and we are pleased
> > >
> > > to announce that he has accepted.
> > >
> > >
> > > Justin has been following the Project for a long time,
> > >
> > > He is very active in the community with discussions and code reviews
> > > and he contributed
> > > many patches.
> > >
> > >
> > > Being a committer enables easier contribution to the
> > >
> > > project since there is no need to go via the patch
> > >
> > > submission process. This should enable better productivity.
> > >
> > > Being a PMC member enables assistance with the management
> > >
> > > and to guide the direction of the project.
> > >
> > >
> > >
> > > Congratulations Justin !
> > >
> > >
> > > Enrico
> >
> >
>

The YCSB benchmark tool for zookeeper is now available

[Justin Ling Mao]

Now users can benchmark your zookeeper ensemble with YCSB. Here is the 
ZK-PR(https://github.com/apache/zookeeper/pull/1558). Have fun for it:)

Re: Re: hacktoberfest

[Justin Ling Mao]

That's really cool. I saw the zookeeper had already joined the Hacktoberfest 
event(https://github.com/topics/hacktoberfest).
You can login in (https://hacktoberfest.digitalocean.com/login) with your 
github account and enjoy yourself
Note: you must register and make four valid pull requests (PRs) for projects 
which have a Hacktoberfest topic between October 1-31. Wish you can win that 
T-shirt :)



- Original Message -
From: Christopher 
To: dev@zookeeper.apache.org
Subject: Re: hacktoberfest
Date: 2020-10-08 05:03


On Wed, Oct 7, 2020 at 4:12 PM Enrico Olivelli  wrote:
>
> Il Mer 7 Ott 2020, 21:02 Christopher  ha scritto:
>
> > I created https://github.com/apache/zookeeper/pull/1489
> > (https://issues.apache.org/jira/browse/ZOOKEEPER-3962)
> >
> Thank you
The sooner this gets updated, the sooner Hacktoberfest participants
will get "credit" for the PRs they have created, and the easier it
will be for new contributors to find ZK's repo.
>
> This should add the hacktoberfest label to the github repo.
> >
> > On a related note: I've noticed that a lot of people just do pull
> > requests, but this project's committers are pretty strict about
> > requiring a JIRA issue for everything, even trivial changes. That
> > information is hard to find for new contributors. If you created a
> > CONTRIBUTING.md file, GitHub would automatically provide a link to
> > that for new contributors coming from GitHub. It would be easier to
> > find than looking on the wiki/confluence page where that information
> > is currently stored. (Alternatively, you could relax your constraints
> > and not require a JIRA for everything...
>
>
> We do not require a JIRA for very simple patches. But  JIRA is super useful
> in order to track release notes.
So, I've seen some patches I thought were very simple, and a committer
still commented on the PR asking them to create a JIRA, even though
they could have just merged it as-is. I'm not really sure there's a
clear demarcation point for users to know when something is considered
"very simple" or not when the committers themselves aren't consistent
on this. I think it's just confusing, and the requirement should go
away entirely. If JIRA is useful to track some issues, sure, use it,
but if somebody has already done the work, and it's self-contained in
a PR, regardless of whether it's simple or not, I think just accepting
it (if it's ready) is the most community-friendly thing to do, rather
than ask a new contributor to create a JIRA at all (especially since
they may not want to create a separate JIRA account, if they don't
already have one). If a JIRA is really needed, the committer
themselves can take on this responsibility instead of asking the
contributor to do it... as a way of lowering the bar to
contributing... especially if the user is a new contributor. If they
are a regular contributor, then they should just be invited to be a
committer. It's easier to hold committers to a higher standard of
responsibility for these things.
Personally, I don't find the generated JIRA "release notes" very
useful. We used to use that in Accumulo, and we stopped doing it,
because it was basically nothing more than a commit log. When the
"release notes" are basically a replay of the git history, they lose
their value. As a replacement, we started curating our release notes
with the most important information users needed to know, and
publishing that in a feed on our website alongside each release. We
refer people (with a convenient link) to the full changes in the git
history, if they are interested. Most of the git history does have
references to issue numbers (because when we merge from the GitHub UI,
it appends the PR number to the log message), if people want
additional context from any conversation on the issue.
I'm not suggesting ZK do this... it may not work for everybody. I'm
merely describing how we did it in Accumulo and why, as an alternative
perspective.
>
> Do you mind adding the CONTRIBUTING.md file into your PR? Just with a
> simple link to confluence
I'm not sure I want to do that with just a link to confluence. I don't
think it's a good idea to make users click through multiple links to
eventually get to the information. Personally, I think the information
should be entirely contained in CONTRIBUTING.md and the confluence
page should be decommissioned... and links to it should point to the
CONTRIBUTING.md page in the repo instead. I'm not a fan of spreading
documentation across multiple sites like that. I think everything
should be on the project website or in the project repo.
So, if I were to contribute a PR to update this, I would pursue it in
that direction. Otherwise, I think somebody else should add
CONTRIBUTING.md if they think it should be done differently.
>
> Enrico
>
> when Accumulo did this, it
> > became much more friendly to new contributors.)
> >
> > On Wed, Oct 7, 2020 at 11:38 AM Enrico Olivelli 
> > wrote:
> > >
> > > Mate,
> > >
> > > Il giorno mer 7 ott 

Re: Containerizing ZooKeeper with Twine: Powering container orchestration from within

[Justin Ling Mao]

Great blog post!!!
- Original Message -
From: Mohamed Jeelani 
To: "dev@zookeeper.apache.org" 
Cc: Christopher Bunn 
Subject: Containerizing ZooKeeper with Twine: Powering container orchestration 
from within
Date: 2020-09-01 03:43

Hi ZooKeepers!
Hello from your friends here at Facebook. Chris Bunn from our team just 
published a blog post this morning regarding containerizing ZooKeeper and how 
we run that containerization platform as well on top of ZooKeeper. You can read 
the blog post at https://engineering.fb.com/developer-tools/zookeeper-twine/
Chris will also be doing a talk this Wednesday at 11am PST followed by a live 
Q as part of our Systems @Scale virtual 
event. You can 
register for the event at https://systemsscalempk2020.splashthat.com/.
Send in your questions for Wednesday’s live Q to 
systemsatsc...@fb.com

Re: Re: Re: Sequential Consistency Guarantees

[Justin Ling Mao]

@ Stanislav Thanks for giving an example to help us to get your idea quickly.
Just as Ted said, If C read 0x1112209 at t4, we have a guarantee to read 
v1(0x1112208), not possible to read v0 (0x1112207) 
Since the write is linearizable, all the writes are ordered with a 
monotonically increasing zxid.If the client C read 0x1112209, which 
indicates the write(0x1112209) is committed, we guarantee that all the 
previous write which had been acknowledged by quorum must have been committed.
Even if the client C reconnects to another server due to some reasons during 
this process, the server will reject the client's requests to avoid stale read 
when its last zxid is less than the last zxid the client C has seen
- Original Message -
From: Ted Dunning 
To: dev@zookeeper.apache.org
Cc: maoling199210...@sina.com
Subject: Re: Re: Sequential Consistency Guarantees
Date: 2020-09-02 00:03

Stanislav,
A ZK client should only ever see state move forward in terms of ZXID.
This is true even if a client disconnects and reconnects.
If you have multiple connections to a ZK cluster, each connection will
satisfy this constraint with respect to itself, but one could be behind the
other. Sync can be used to force a stricter relationship.
On Tue, Sep 1, 2020 at 7:35 AM Stanislav Kozlovski 
wrote:
> ...
>
> ```
> t1: A creates /myPathA v0 (0x1112207)
> t2: A  setData /myPathA v1 (0x1112208)
> t3: B creates /myPathB v0 (0x1112209)
> t4: C reads /myPathB
> t5: C reads /myPathA
> ```
>
> Given that, at t4, C read 0x1112209, do we have a guarantee that at t5
> C read v1 of /myPathA (0x1112208)?
> Is it possible that it could read v0 (0x1112207) under some edge
> conditions?
>
>

Re: Re: May violate the ZAB agreement -- version 3.6.1

[Justin Ling Mao]

@李珣The situation you describe may have conceptual deviations about how the 
consensus protocol works:---> Since the data of the follower when the follower 
uses the DIFF method to synchronize with the leader is still in the memory, it 
has not had time to persist1. The write path is: write transaction log(WAL) 
firstly, after reaching a consensus, then apply to memory, other than the 
opposite.
---> but at this time, the latest zxid_n of the leader has not been supported 
by the quorum of the follower. At this time, if a client connects to the leader 
and sees zxid_n,2. If a write has not been supported by the quorum, it's not 
safe to apply to the state machine and the client is not able to see this write.
I guess that your question may be: how the system handles the uncommitted logs 
when leader changes?



- Original Message -
From: Ted Dunning 
To: dev@zookeeper.apache.org
Subject: Re: May violate the ZAB agreement -- version 3.6.1
Date: 2020-08-28 01:25

How is it that participant A would have a later zxid than the leader?
In particular, it seems to me that it should be impossible to have these
two facts be true:
1) a transaction has been committed with zxid = z_0. This implies that a
quorum of the cluster has accepted this transaction and it has been
committed.
2) a new leader election nominates a leader with latest zxid < z_0.
My reasoning is that any new leader election has to involve a quorum and at
least a sufficient number of that quorum must have accepted zxid >= z_0 and
therefore would refuse to be part of the quorum (this is a contradiction).
Thus, no leader could be elected with zxid < z_0 if fact (1) is true.
What you are describing seems to require both of these facts.
Perhaps I am missing something about your suggested scenario. Could you
describe what you are thinking in more detail?
On Thu, Aug 27, 2020 at 2:08 AM 李珣 <274952...@qq.com> wrote:
> version 3.6.1
> org.apache.zookeeper.server.quorum.Learner.java line:605
> Suppose there is a situation
> zxid_n is the largest zxid of Participant A (the leader has just resumed
> from downtime). Zxid_n has not been recognized by the quorum. Assuming
> Participant A is elected as the Leader, then if a follower appears to use
> DIFF to synchronize data with the Leader, Leader After sending the
> UPTODATE, the leader can already provide external access, but at this time,
> the latest zxid_n of the leader has not been supported by the quorum of the
> follower. At this time, if a client connects to the leader and sees zxid_n,
> then at this time both the leader and the follower are down. For some
> reason, the leader cannot be started, and the follower can start normally.
> At this time, a new leader can only be elected from the follower. Since the
> data of the follower when the follower uses the DIFF method to synchronize
> with the leader is still in the memory, it has not had time to persist,
> then this The newly elected leader does not have the data of zxid_n, but
> before zxid_n has been seen by the client on the old leader, there will be
> inconsistencies in the data view.
> Is the above situation possible?

Re: Re: Sequential Consistency Guarantees

[Justin Ling Mao]

--->  if a client creates a node with zxid 0x11122a, it should be able to 
see (read) all data changes at least up to that zxid.
You may have a misunderstanding on what sequential consistency guarantees. For 
your example:
Suppose that you have following operations:
create   /mypath v0 (0x1112207) 
setData /mypath v1 (0x1112208)
setData /mypath v2 (0x1112209)
Now at the time(t1), the client setDatas 0x11122a(v3) successfully. Just 
after the time(t1), the client still has the potential possibility to read the 
value(v2), otherwise v3. (e.g.,: the client may connect to the minority server 
which may receive or write the transaction log slower than the majority due to 
network or disk latency, so the write is not exposed to that client in time)

Generally speaking, sequential consistency guarantees that we don't have the 
potential risk of read skew. However it doesn't guarantee that you can always 
read the latest value. What you want may be Linearizability.
You may be interested in these posts which have more accurate definition on the 
consistency:https://aphyr.com/posts/313-strong-consistency-modelshttps://jepsen.io/consistency/models/sequentialhttps://jepsen.io/consistency/models/linearizable

BTW, I am working on "ZOOKEEPER-3600:support the complete linearizable read and 
multiply read consistency level", and I wish it will be coming soon.


- Original Message -
From: Ted Dunning 
To: dev@zookeeper.apache.org
Subject: Re: Sequential Consistency Guarantees
Date: 2020-08-28 02:06

Can you describe your evidence in more detail as you understand it?
On Thu, Aug 27, 2020 at 10:26 AM Stanislav Kozlovski 
wrote:
> Hello!
>
> We were investigating an issue with consistent reads in ZK which made us
> question our assumptions.
> We believe that ZK offers sequential consistency such that if a client
> creates a node with zxid 0x11122a, it should be able to see (read) all
> data changes at least up to that zxid.
>
> Could I ask if this is a correct understanding? We seem to be seeing
> evidence that's pointing towards the opposite direction.
>
> Looking at the documentation
> <
> https://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html#ch_zkGuarantees
> >,
> it does not seem to address this specific point. It says that "Updates from
> a client will be applied in the order that they were sent" and talks about
> "Consistent Cross-Client Views" but does not seem to address cross-client
> update order, unless I'm missing something.
>
> --
> Best,
> Stanislav
>

Re: Re: Re: Re: Re: Re: Re: Grafana dashboard, update?

[Justin Ling Mao]

@simon.karberg
For alerting documentation, please look at 
this(https://github.com/apache/zookeeper/pull/1425)

- Original Message -
From: "Justin Ling Mao" 
To: "dev" , "simon.karberg" 
,
Subject: Re: Re: Re: Re: Re: Re: Grafana dashboard, update?
Date: 2020-08-04 12:11

--->Are there any documentation on what to alert on, related to Zookeeper?
A good suggestion. I create 
ZOOKEEPER-3907(https://issues.apache.org/jira/browse/ZOOKEEPER-3907) for myself.
Recommend using the Prometheus AlertManager, rather than Grafana alert one.

- Original Message -
From: Simon Karberg 
To: maoling199210...@sina.com
Cc: dev 
Subject: Re: Re: Re: Re: Re: Grafana dashboard, update?
Date: 2020-08-03 16:17

Thanks, it looks amazing already.
Are there any documentation on what to alert on, related to Zookeeper?
On Mon, Aug 3, 2020 at 4:33 AM Justin Ling Mao 
wrote:
> *Dashboard template v4 (*https://grafana.com/grafana/dashboards/10465*)
> is now available. I suggest you use this new one for better experience by
> overriding the old one v3(copy the dashboard Id:10465**)*
>
>
> *Change Log is here:*
>
> [*v4*]
>
>-
>
>*Feature*:
>- Add the JVM related metrics
>   - Support to choose by datasource, job and instance
>-
>
>*Fix*:
>- Remove duplicated metrics: e.g.,
>   *write_batch_time_in_commit_processor*
>-
>
>*Improvement*:
>- Use *rate()* to react the fluctuation of data for total time summary
>   metric
>   - Have a better classification by checking the source codes to know
>   what every metric wants to do
>   - The metrics in the *CommitProcessor* are placed the same order
>   with the source code for a better observation for the workflow of
>   *CommitProcessor*
>   - Use an unit(e.g., ms/byte) for metric graph
>   - Prioritize the metrics in the same panel
>-
>
>*Compatibility Test*: - Tested with grafana-6.x(*6.2.2*); grafana-7.x(
>*7.0.5*)
>
>
>
> - Original Message -
> From: Simon Karberg 
> To: maoling199210...@sina.com
> Cc: dev 
> Subject: Re: Re: Re: Re: Grafana dashboard, update?
> Date: 2020-07-29 15:46
>
> Great, I will be ready to add it to our grafana on Monday then :)
>
> On Wed, Jul 29, 2020 at 9:45 AM Justin Ling Mao 
> wrote:
>
> @Simon Karberg
> What a shame for me to do the work of dashboard v3(a really bullshit)
> which is disordered, has stupid
> mistakes. I wish It's not gonna happen these again at v4 which will be
> available at 08-01(Saturday).
>
>
> - Original Message -
> From: Simon Karberg 
> To: maoling199210...@sina.com
> Cc: dev 
> Subject: Re: Re: Re: Grafana dashboard, update?
> Date: 2020-07-27 06:17
>
> Your welcome and thanks for the good work, i can imagine it takes time to
> create such dashboards.
> On Sat, Jul 25, 2020 at 7:57 AM Justin Ling Mao  >
> wrote:
> > 1. Yes, v3 doesn't have that option. I have done this function in the
> > dashboard v4(Now v4 also supports to choose by job, instance/host).
> > 2. v4 will be coming asap with new things/improvements(the plan is at the
> > end of this month)
> > 3. After v4 is uploaded, I will notify you with email. Thanks for your
> > great suggestions
> >
> >
> > ----- Original Message -
> > From: Simon Karberg 
> > To: maoling199210...@sina.com
> > Cc: dev 
> > Subject: Re: Re: Grafana dashboard, update?
> > Date: 2020-07-24 15:29
> >
> > Hi,
> > I don't have that option in the dashboard ( 10465 ) :)
> > On Fri, Jul 24, 2020 at 5:16 AM Justin Ling Mao <
> maoling199210...@sina.com
> > >
> > wrote:
> > >
> > > @Simon Karberg
> > > Is it what you want(Look at the email attachment)? which can help users
> > > choose/change different data sources in the same dashboard without
> > creating
> > > multiply dashboards?
> > >
> > >
> > >
> > > - Original Message -
> > > From: Simon Karberg 
> > > To: maoling199210...@sina.com
> > > Cc: dev 
> > > Subject: Re: Grafana dashboard, update?
> > > Date: 2020-07-22 17:02
> > >
> > > No, What i mean is all the panels throughout the dashboard doesn't have
> > > "datasource": "$DS_PROMETHEUS" so grafana just sticks to what
> datasource
> > > there is defined as default.
> > > I managed to go through the 9000 lines of json and add the datasource
> so
> > > all if them is connected to the prometheus datasource i've chosen.
> > >
> > > On Wed, Jul 22, 2020 at 10:33 AM Justin Ling Mao <
> 

Re: Re: Re: Re: Re: Re: Grafana dashboard, update?

[Justin Ling Mao]

--->Are there any documentation on what to alert on, related to Zookeeper?
A good suggestion. I create 
ZOOKEEPER-3907(https://issues.apache.org/jira/browse/ZOOKEEPER-3907) for myself.
Recommend using the Prometheus AlertManager, rather than Grafana alert one.

- Original Message -
From: Simon Karberg 
To: maoling199210...@sina.com
Cc: dev 
Subject: Re: Re: Re: Re: Re: Grafana dashboard, update?
Date: 2020-08-03 16:17

Thanks, it looks amazing already.
Are there any documentation on what to alert on, related to Zookeeper?
On Mon, Aug 3, 2020 at 4:33 AM Justin Ling Mao 
wrote:
> *Dashboard template v4 (*https://grafana.com/grafana/dashboards/10465*)
> is now available. I suggest you use this new one for better experience by
> overriding the old one v3(copy the dashboard Id:10465**)*
>
>
> *Change Log is here:*
>
> [*v4*]
>
>-
>
>*Feature*:
>- Add the JVM related metrics
>   - Support to choose by datasource, job and instance
>-
>
>*Fix*:
>- Remove duplicated metrics: e.g.,
>   *write_batch_time_in_commit_processor*
>-
>
>*Improvement*:
>- Use *rate()* to react the fluctuation of data for total time summary
>   metric
>   - Have a better classification by checking the source codes to know
>   what every metric wants to do
>   - The metrics in the *CommitProcessor* are placed the same order
>   with the source code for a better observation for the workflow of
>   *CommitProcessor*
>   - Use an unit(e.g., ms/byte) for metric graph
>   - Prioritize the metrics in the same panel
>-
>
>*Compatibility Test*: - Tested with grafana-6.x(*6.2.2*); grafana-7.x(
>*7.0.5*)
>
>
>
> - Original Message -
> From: Simon Karberg 
> To: maoling199210...@sina.com
> Cc: dev 
> Subject: Re: Re: Re: Re: Grafana dashboard, update?
> Date: 2020-07-29 15:46
>
> Great, I will be ready to add it to our grafana on Monday then :)
>
> On Wed, Jul 29, 2020 at 9:45 AM Justin Ling Mao 
> wrote:
>
> @Simon Karberg
> What a shame for me to do the work of dashboard v3(a really bullshit)
> which is disordered, has stupid
> mistakes. I wish It's not gonna happen these again at v4 which will be
> available at 08-01(Saturday).
>
>
> - Original Message -
> From: Simon Karberg 
> To: maoling199210...@sina.com
> Cc: dev 
> Subject: Re: Re: Re: Grafana dashboard, update?
> Date: 2020-07-27 06:17
>
> Your welcome and thanks for the good work, i can imagine it takes time to
> create such dashboards.
> On Sat, Jul 25, 2020 at 7:57 AM Justin Ling Mao  >
> wrote:
> > 1. Yes, v3 doesn't have that option. I have done this function in the
> > dashboard v4(Now v4 also supports to choose by job, instance/host).
> > 2. v4 will be coming asap with new things/improvements(the plan is at the
> > end of this month)
> > 3. After v4 is uploaded, I will notify you with email. Thanks for your
> > great suggestions
> >
> >
> > - Original Message -----
> > From: Simon Karberg 
> > To: maoling199210...@sina.com
> > Cc: dev 
> > Subject: Re: Re: Grafana dashboard, update?
> > Date: 2020-07-24 15:29
> >
> > Hi,
> > I don't have that option in the dashboard ( 10465 ) :)
> > On Fri, Jul 24, 2020 at 5:16 AM Justin Ling Mao <
> maoling199210...@sina.com
> > >
> > wrote:
> > >
> > > @Simon Karberg
> > > Is it what you want(Look at the email attachment)? which can help users
> > > choose/change different data sources in the same dashboard without
> > creating
> > > multiply dashboards?
> > >
> > >
> > >
> > > - Original Message -
> > > From: Simon Karberg 
> > > To: maoling199210...@sina.com
> > > Cc: dev 
> > > Subject: Re: Grafana dashboard, update?
> > > Date: 2020-07-22 17:02
> > >
> > > No, What i mean is all the panels throughout the dashboard doesn't have
> > > "datasource": "$DS_PROMETHEUS" so grafana just sticks to what
> datasource
> > > there is defined as default.
> > > I managed to go through the 9000 lines of json and add the datasource
> so
> > > all if them is connected to the prometheus datasource i've chosen.
> > >
> > > On Wed, Jul 22, 2020 at 10:33 AM Justin Ling Mao <
> > > maoling199210...@sina.com> wrote:
> > >
> > > Yes, I'm here. The only supported data source currently is
> *Prometheus*.
> > > Do you want us to support *Influxdb* as data source?
> > >
> > > - Original Message -
> > > From: Si

Re: Re: Re: Re: Re: Grafana dashboard, update?

[Justin Ling Mao]

Dashboard template v4 (https://grafana.com/grafana/dashboards/10465) is now 
available. I suggest you use this new one for better experience by overriding 
the old one v3(copy the dashboard Id:10465)

Change Log is here:
[v4]Feature:Add the JVM related metricsSupport to choose by datasource, job and 
instanceFix:Remove duplicated metrics: e.g., 
write_batch_time_in_commit_processorImprovement:Use rate() to react the 
fluctuation of data for total time summary metricHave a better classification 
by checking the source codes to know what every metric wants to doThe metrics 
in the CommitProcessor are placed the same order with the source code for a 
better observation for the workflow of CommitProcessorUse an unit(e.g., 
ms/byte) for metric graphPrioritize the metrics in the same panelCompatibility 
Test: - Tested with grafana-6.x(6.2.2); grafana-7.x(7.0.5)

- Original Message -
From: Simon Karberg 
To: maoling199210...@sina.com
Cc: dev 
Subject: Re: Re: Re: Re: Grafana dashboard, update?
Date: 2020-07-29 15:46

Great, I will be ready to add it to our grafana on Monday then :) 
On Wed, Jul 29, 2020 at 9:45 AM Justin Ling Mao  
wrote:
@Simon KarbergWhat a shame for me to do the work of dashboard v3(a really 
bullshit) which is disordered, has stupidmistakes. I wish It's not gonna happen 
these again at v4 which will be available at 08-01(Saturday).

- Original Message -
From: Simon Karberg 
To: maoling199210...@sina.com
Cc: dev 
Subject: Re: Re: Re: Grafana dashboard, update?
Date: 2020-07-27 06:17

Your welcome and thanks for the good work, i can imagine it takes time to
create such dashboards.
On Sat, Jul 25, 2020 at 7:57 AM Justin Ling Mao 
wrote:
> 1. Yes, v3 doesn't have that option. I have done this function in the
> dashboard v4(Now v4 also supports to choose by job, instance/host).
> 2. v4 will be coming asap with new things/improvements(the plan is at the
> end of this month)
> 3. After v4 is uploaded, I will notify you with email. Thanks for your
> great suggestions
>
>
> - Original Message -
> From: Simon Karberg 
> To: maoling199210...@sina.com
> Cc: dev 
> Subject: Re: Re: Grafana dashboard, update?
> Date: 2020-07-24 15:29
>
> Hi,
> I don't have that option in the dashboard ( 10465 ) :)
> On Fri, Jul 24, 2020 at 5:16 AM Justin Ling Mao  >
> wrote:
> >
> > @Simon Karberg
> > Is it what you want(Look at the email attachment)? which can help users
> > choose/change different data sources in the same dashboard without
> creating
> > multiply dashboards?
> >
> >
> >
> > - Original Message -
> > From: Simon Karberg 
> > To: maoling199210...@sina.com
> > Cc: dev 
> > Subject: Re: Grafana dashboard, update?
> > Date: 2020-07-22 17:02
> >
> > No, What i mean is all the panels throughout the dashboard doesn't have
> > "datasource": "$DS_PROMETHEUS" so grafana just sticks to what datasource
> > there is defined as default.
> > I managed to go through the 9000 lines of json and add the datasource so
> > all if them is connected to the prometheus datasource i've chosen.
> >
> > On Wed, Jul 22, 2020 at 10:33 AM Justin Ling Mao <
> > maoling199210...@sina.com> wrote:
> >
> > Yes, I'm here. The only supported data source currently is *Prometheus*.
> > Do you want us to support *Influxdb* as data source?
> >
> > - Original Message -
> > From: Simon Karberg 
> > To: dev@zookeeper.apache.org
> > Subject: Grafana dashboard, update?
> > Date: 2020-07-21 01:24
> >
> > Hi,
> > I have tried to import the dashboard 10465 (ZooKeeper by Prometheusby
> > maoling)
> > I found out that many of the graphs doesn't include the use of the
> > defined DS_PROMETHEUS so it just defaults to our default data source
> which
> > is influxdb.
> > I haven't been through the whole json file since it's almost 9000 lines,
> > but is there an updated version on it's way?
> > --
> > *Simon Karberg* / Operations Engineer
> > Mobile: +45 28901033
> > [image: Agillic logo]
> > www.agillic.com
> > Support Desk: +45 32 63 08 00
> > Copenhagen Office: +45 70 25 28 25
> > London Office: +44 7711 744892
> > [image: Facebook] <
> https://www.facebook.com/pages/Agillic/136444876415980>
> > [image: LinkedIn] <https://www.linkedin.com/company/agillic>
> >
> >
> >
> > --
> >
> > *Simon Karberg* / Operations Engineer
> > Mobile: +45 28901033
> >
> > [image: Agillic logo]
> >
> > www.agillic.com
> > Support Desk: +45 32 63 08 00
> > Copenhagen Office: +45 70 25 28 25
> > London Office: +44 7711 744892
> >
> > [i

Re: Re: Re: Re: Grafana dashboard, update?

[Justin Ling Mao]

@Simon KarbergWhat a shame for me to do the work of dashboard v3(a really 
bullshit) which is disordered, has stupidmistakes. I wish It's not gonna happen 
these again at v4 which will be available at 08-01(Saturday).

- Original Message -
From: Simon Karberg 
To: maoling199210...@sina.com
Cc: dev 
Subject: Re: Re: Re: Grafana dashboard, update?
Date: 2020-07-27 06:17

Your welcome and thanks for the good work, i can imagine it takes time to
create such dashboards.
On Sat, Jul 25, 2020 at 7:57 AM Justin Ling Mao 
wrote:
> 1. Yes, v3 doesn't have that option. I have done this function in the
> dashboard v4(Now v4 also supports to choose by job, instance/host).
> 2. v4 will be coming asap with new things/improvements(the plan is at the
> end of this month)
> 3. After v4 is uploaded, I will notify you with email. Thanks for your
> great suggestions
>
>
> - Original Message -
> From: Simon Karberg 
> To: maoling199210...@sina.com
> Cc: dev 
> Subject: Re: Re: Grafana dashboard, update?
> Date: 2020-07-24 15:29
>
> Hi,
> I don't have that option in the dashboard ( 10465 ) :)
> On Fri, Jul 24, 2020 at 5:16 AM Justin Ling Mao  >
> wrote:
> >
> > @Simon Karberg
> > Is it what you want(Look at the email attachment)? which can help users
> > choose/change different data sources in the same dashboard without
> creating
> > multiply dashboards?
> >
> >
> >
> > - Original Message -
> > From: Simon Karberg 
> > To: maoling199210...@sina.com
> > Cc: dev 
> > Subject: Re: Grafana dashboard, update?
> > Date: 2020-07-22 17:02
> >
> > No, What i mean is all the panels throughout the dashboard doesn't have
> > "datasource": "$DS_PROMETHEUS" so grafana just sticks to what datasource
> > there is defined as default.
> > I managed to go through the 9000 lines of json and add the datasource so
> > all if them is connected to the prometheus datasource i've chosen.
> >
> > On Wed, Jul 22, 2020 at 10:33 AM Justin Ling Mao <
> > maoling199210...@sina.com> wrote:
> >
> > Yes, I'm here. The only supported data source currently is *Prometheus*.
> > Do you want us to support *Influxdb* as data source?
> >
> > - Original Message -
> > From: Simon Karberg 
> > To: dev@zookeeper.apache.org
> > Subject: Grafana dashboard, update?
> > Date: 2020-07-21 01:24
> >
> > Hi,
> > I have tried to import the dashboard 10465 (ZooKeeper by Prometheusby
> > maoling)
> > I found out that many of the graphs doesn't include the use of the
> > defined DS_PROMETHEUS so it just defaults to our default data source
> which
> > is influxdb.
> > I haven't been through the whole json file since it's almost 9000 lines,
> > but is there an updated version on it's way?
> > --
> > *Simon Karberg* / Operations Engineer
> > Mobile: +45 28901033
> > [image: Agillic logo]
> > www.agillic.com
> > Support Desk: +45 32 63 08 00
> > Copenhagen Office: +45 70 25 28 25
> > London Office: +44 7711 744892
> > [image: Facebook] <
> https://www.facebook.com/pages/Agillic/136444876415980>
> > [image: LinkedIn] <https://www.linkedin.com/company/agillic>
> >
> >
> >
> > --
> >
> > *Simon Karberg* / Operations Engineer
> > Mobile: +45 28901033
> >
> > [image: Agillic logo]
> >
> > www.agillic.com
> > Support Desk: +45 32 63 08 00
> > Copenhagen Office: +45 70 25 28 25
> > London Office: +44 7711 744892
> >
> > [image: Facebook] <
> https://www.facebook.com/pages/Agillic/136444876415980>
> > [image: LinkedIn] <https://www.linkedin.com/company/agillic>
> >
> --
> *Simon Karberg* / Operations Engineer
> Mobile: +45 28901033
> [image: Agillic logo]
> www.agillic.com
> Support Desk: +45 32 63 08 00
> Copenhagen Office: +45 70 25 28 25
> London Office: +44 7711 744892
> [image: Facebook] <https://www.facebook.com/pages/Agillic/136444876415980>
> [image: LinkedIn] <https://www.linkedin.com/company/agillic>
>
-- 
*Simon Karberg* / Operations Engineer
Mobile: +45 28901033
[image: Agillic logo]
www.agillic.com
Support Desk: +45 32 63 08 00
Copenhagen Office: +45 70 25 28 25
London Office: +44 7711 744892
[image: Facebook] <https://www.facebook.com/pages/Agillic/136444876415980>
[image: LinkedIn] <https://www.linkedin.com/company/agillic>

Re: Re: Re: Grafana dashboard, update?

[Justin Ling Mao]

1. Yes, v3 doesn't have that option. I have done this function in the dashboard 
v4(Now v4 also supports to choose by job, instance/host).2. v4 will be coming 
asap with new things/improvements(the plan is at the end of this month) 3. 
After v4 is uploaded, I will notify you with email. Thanks for your great 
suggestions 
- Original Message -
From: Simon Karberg 
To: maoling199210...@sina.com
Cc: dev 
Subject: Re: Re: Grafana dashboard, update?
Date: 2020-07-24 15:29

Hi,
I don't have that option in the dashboard ( 10465 ) :)
On Fri, Jul 24, 2020 at 5:16 AM Justin Ling Mao 
wrote:
>
> @Simon Karberg
> Is it what you want(Look at the email attachment)? which can help users
> choose/change different data sources in the same dashboard without creating
> multiply dashboards?
>
>
>
> - Original Message -
> From: Simon Karberg 
> To: maoling199210...@sina.com
> Cc: dev 
> Subject: Re: Grafana dashboard, update?
> Date: 2020-07-22 17:02
>
> No, What i mean is all the panels throughout the dashboard doesn't have
> "datasource": "$DS_PROMETHEUS" so grafana just sticks to what datasource
> there is defined as default.
> I managed to go through the 9000 lines of json and add the datasource so
> all if them is connected to the prometheus datasource i've chosen.
>
> On Wed, Jul 22, 2020 at 10:33 AM Justin Ling Mao <
> maoling199210...@sina.com> wrote:
>
> Yes, I'm here. The only supported data source currently is *Prometheus*.
> Do you want us to support *Influxdb* as data source?
>
> - Original Message -
> From: Simon Karberg 
> To: dev@zookeeper.apache.org
> Subject: Grafana dashboard, update?
> Date: 2020-07-21 01:24
>
> Hi,
> I have tried to import the dashboard 10465 (ZooKeeper by Prometheusby
> maoling)
> I found out that many of the graphs doesn't include the use of the
> defined DS_PROMETHEUS so it just defaults to our default data source which
> is influxdb.
> I haven't been through the whole json file since it's almost 9000 lines,
> but is there an updated version on it's way?
> --
> *Simon Karberg* / Operations Engineer
> Mobile: +45 28901033
> [image: Agillic logo]
> www.agillic.com
> Support Desk: +45 32 63 08 00
> Copenhagen Office: +45 70 25 28 25
> London Office: +44 7711 744892
> [image: Facebook] <https://www.facebook.com/pages/Agillic/136444876415980>
> [image: LinkedIn] <https://www.linkedin.com/company/agillic>
>
>
>
> --
>
> *Simon Karberg* / Operations Engineer
> Mobile: +45 28901033
>
> [image: Agillic logo]
>
> www.agillic.com
> Support Desk: +45 32 63 08 00
> Copenhagen Office: +45 70 25 28 25
> London Office: +44 7711 744892
>
> [image: Facebook] <https://www.facebook.com/pages/Agillic/136444876415980>
> [image: LinkedIn] <https://www.linkedin.com/company/agillic>
>
-- 
*Simon Karberg* / Operations Engineer
Mobile: +45 28901033
[image: Agillic logo]
www.agillic.com
Support Desk: +45 32 63 08 00
Copenhagen Office: +45 70 25 28 25
London Office: +44 7711 744892
[image: Facebook] <https://www.facebook.com/pages/Agillic/136444876415980>
[image: LinkedIn] <https://www.linkedin.com/company/agillic>

Re: Grafana dashboard, update?

[Justin Ling Mao]

Yes, I'm here. The only supported data source currently is Prometheus. Do you 
want us to support Influxdb as data source?
- Original Message -
From: Simon Karberg 
To: dev@zookeeper.apache.org
Subject: Grafana dashboard, update?
Date: 2020-07-21 01:24

Hi,
I have tried to import the dashboard 10465 (ZooKeeper by Prometheusby
maoling)
I found out that many of the graphs doesn't include the use of the
defined DS_PROMETHEUS so it just defaults to our default data source which
is influxdb.
I haven't been through the whole json file since it's almost 9000 lines,
but is there an updated version on it's way?
-- 
*Simon Karberg* / Operations Engineer
Mobile: +45 28901033
[image: Agillic logo]
www.agillic.com
Support Desk: +45 32 63 08 00
Copenhagen Office: +45 70 25 28 25
London Office: +44 7711 744892
[image: Facebook] 
[image: LinkedIn] 

Re: Re: Cutting 3.6.1 HEADS UP !

[Justin Ling Mao]

I found ZOOKEEPER-3807. lots of bad formats in the website pages build due to 
bash marker. It's my blame.Do we need to apply it to 3.6.1 ?

- Original Message -
From: Christopher 
To: dev@zookeeper.apache.org
Subject: Re: Cutting 3.6.1 HEADS UP !
Date: 2020-04-17 13:27


Okay, I found the problem. The exec-maven-plugin seems to ignore the now-empty:
${mvngit.commit.id}
This is a bug in either exec-maven-plugin or maven itself when parsing
config, possibly related to:
https://github.com/mojohaus/exec-maven-plugin/issues/132
I provided a pull request to fix it in:
https://github.com/apache/zookeeper/pull/1321
On Tue, Apr 14, 2020 at 11:03 PM Christopher  wrote:
>
> On Sun, Apr 12, 2020 at 12:39 PM Enrico Olivelli  wrote:
> >
> > Christopher
> > Maybe with your commit a93ff0fe631d1c96ee056a79e3c16535ab33c794 we have
> > broken the source release tarball.
>
> Possibly. I patched 3.7/master. I wasn't involved in the backport to
> 3.6 and did not test it there.
>
> Speaking of which... it is *very* confusing that everything is
> cherry-picked from branch to branch... git commit ids are not
> consistent across branches, so you can't use things like `git branch
> --contains ` to find useful branches containing a commit. Also,
> it seems to me that unless every commit starts in master and is
> selectively back-ported, then cherry-picking could lead to a lot of
> regressions (a commit targeted for an older branch is forgotten to be
> added to newer branches).
>
> >
> > It looks like we are not passing the git sha to VerGen or something like
> > that
> >
> > Just download the source tarball from my staging area and try.
> >
> > Do you have time to help fixing this issue?
> > Otherwise I can try to fix it or simply git revert that commit as it is not
> > a blocker issue for the release
>
> How do you generate a source release tarball?
> What is broken?
> With more details, I can suggest a fix, but I don't see anything
> broken in the build.
>
> >
> > Cheers
> > Enrico
> >
> > Il giorno dom 12 apr 2020 alle ore 14:58 Enrico Olivelli <
> > eolive...@gmail.com> ha scritto:
> >
> > > Christopher
> > > This is my staging area
> > > https://people.apache.org/~eolivelli/zookeeper-3.6.1-candidate-0/
> > >
> > > this is not the VOTE thread, I will send the official VOTE email once I
> > > feel the staging area is valid
> > > this is the Maven Repository
> > > https://repository.apache.org/content/repositories/orgapachezookeeper-1054
> > >
> > > I will hopefully send the VOTE email tomorrow
> > >
> > > Please send me comments directly and not to the ML, in order not to create
> > > confusion
> > >
> > > Thank in advance
> > > Enrico
> > >
> > > Il giorno ven 10 apr 2020 alle ore 04:35 Christopher 
> > > ha scritto:
> > >
> > >> I don't anticipate any issues, but I can test Accumulo with the
> > >> release candidate when it's ready.
> > >> Do you already have the binary tarball built and uploaded somewhere?
> > >>
> > >> On Thu, Apr 9, 2020 at 2:43 PM Enrico Olivelli 
> > >> wrote:
> > >> >
> > >> > Il Gio 9 Apr 2020, 20:33 Norbert Kalmar 
> > >> ha
> > >> > scritto:
> > >> >
> > >> > > Hi Enrico,
> > >> > >
> > >> > > Thanks for driving this!
> > >> > >
> > >> > > I managed to build HBase with ZooKeeper 3.5.7 having cherry-picked 
> > >> > > the
> > >> > > getRevision() patch. I know it's not 3.6.x, but I found the problem
> > >> with
> > >> > > this 3.5.7 and fixed it according to this on 3.6 as well. So it
> > >> should be
> > >> > > fine now.
> > >> > >
> > >> >
> > >> > Thank you
> > >> >
> > >> > Enrico
> > >> >
> > >> > >
> > >> > > - Norbert
> > >> > >
> > >> > > On Thu, Apr 9, 2020 at 11:00 AM Enrico Olivelli 
> > >> > > wrote:
> > >> > >
> > >> > > > Hi,
> > >> > > > I am going to prepare a release candidate for ZooKeeper 3.6.1.
> > >> > > >
> > >> > > > There is no JIRA issue with fixVersion = 3.6.1 that is unresolved.
> > >> > > >
> > >> > > > I have tested a few projects that had compatibility issues and they
> > >> are
> > >> > > > resolved (like Apache BookKeeper and other non OS projects in my
> > >> > > company).
> > >> > > >
> > >> > > > I remember that Norbert pointed a problem with HBase client, that
> > >> uses
> > >> > > > getRevision() method, it would be super great to have some feedback
> > >> of
> > >> > > > compatibility of HBase with 3.6.1 client.
> > >> > > >
> > >> > > > I have deployed the snapshots to snapshots.apache.org, this way
> > >> you can
> > >> > > > easily test your project in CI, even on Travis.
> > >> > > >
> > >> > > > I have created the work branch for release
> > >> > > > https://github.com/apache/zookeeper/tree/release-3.6.1
> > >> > > >
> > >> > > > Please ping me if you have any questions or concerns or you need to
> > >> add
> > >> > > new
> > >> > > > items.
> > >> > > >
> > >> > > > I will start the release procedure once I have self validated the
> > >> status
> > >> > > of
> > >> > > > that branch
> > >> > > >
> > >> > > > Stay tuned
> > >> > > >
> > >> > > > 

Re: Re: Jepsen.io report

[Justin Ling Mao]

Actually, we have a doc about how to test zookeeper with 
jepsen(https://github.com/apache/zookeeper/blob/master/zookeeper-docs/src/main/resources/markdown/zookeeperTools.md#jepsen-test)
Jepsen had not found any bugs recently? We needs to redo that Jepsen test after 
we have a linearizability read api?
etcd had paid for Kyle by fund from CNCF :)
- Original Message -
From: Michael Han 
To: dev@zookeeper.apache.org
Subject: Re: Jepsen.io report
Date: 2020-02-18 14:02

I believe the original report for ZooKeeper
https://aphyr.com/posts/291-call-me-maybe-zookeeper was done by the author
of Jepsen. Some of the conclusions were not very accurate (see
https://github.com/jepsen-io/jepsen/issues/399 - "insumity" was an intern
in my team fyi). Would be nice if we can integrate Jepsen as part of apache
zookeeper CI.
On Sat, Feb 15, 2020 at 11:19 PM Enrico Olivelli 
wrote:
> Hi community,
> I see that Jepsen.io has a very old analysis of Zookeeper  (2013)
>
> https://jepsen.io/analyses
>
> Did they do by themselves or did any company pay for such analysis?
>
> Do you have experience in using such tools?
>
>
> Enrico
>

Re: Re: My Contributions

[Justin Ling Mao]

David, highly appreciated your contributions. Looking forward to more. Please 
go ahead!

- Original Message -
From: Norbert Kalmar 
To: DevZooKeeper 
Subject: Re: My Contributions
Date: 2020-02-03 18:34

Hi David,
Thanks for your contributions!
We consider the patch committed if it should be backported or not. In my
opinion if a contributor finds a bug/improvement to do, we can't expect
him/her to know what branches this should be committed to. Of course if the
author takes the time to dig through the different branches and suggests to
backport or not, than that's great, but just creating a patch against
master works just as well :) And of course sometimes (a great example is
security fixes) we just automatically backport the change to all (or just
some) branch still maintained.
And one more thing: anyone is in a position to drive big changes, ZooKeeper
is open source, contributors are welcomed :)
(Of course depending on how big a change, it might be worth starting on
this dev list to vote / give a heads up / ask for help).
Regards,
Norbert
On Mon, Feb 3, 2020 at 6:04 AM David Mollitor  wrote:
> Hey Team,
>
> I have very much appreciated all of the support and merging of my PRs.  I
> just wanted to clarify / get clarification of the ZK release process.
>
> I'm not in a position to drive any big changes or big big fixes into the
> product.  I'm just trying to keep the code clean, keeping down the lines of
> code. and to hopefully preventing others from making mistakes,... and maybe
> add a percentage point or two on performance.
>
> Unless I suggest otherwise, I intend all of my work to be pushed into
> master.  No need to back port it to existing released versions.  Typically,
> only big fixes are added to such releases.
>
> All the more reason to not back port my work to past releases, several big
> users are consuming changes from trunk anyway.  So my changes are being
> consumed pretty quickly and do not need to go out in previous releases to
> add value.
>
> Thanks,
> David
>

2019 year-end review

[Justin Ling Mao]

Hi zk guys:
Happy New Year, Good luck to your 2020.
IMHO it's a good time to look back the community work of the past year(2019) at 
the beginning of this new year.
Forgive my this letter, please let me share some my private and humble 
observations.
In retrospect 2019, these new things have happened[1]:
  Apache Software Foundation(ASF) 2019 "top 5" projects
  Three new releases: 3.5.6, 3.5.5, 3.4.14
  292 Commits. raise 97% comparing to 2018(148 Commits)
  79375 Lines added, 44392 Lines removed.
  One new PMC: Andor Molnar; Three new Committers: Enrico Olivelli, Norbert 
Kalmar, Brian Nixon
  60 contributors. Top 5 contributors[2]: Enrico Olivelli, maoling, Jie Huang, 
Brian Nixon, Andor Molnar

Correct me if I have something wrong above, supplement me if I forget something.
Please join this email thread to share your story. e.g: what great changes do 
you want zookeeper community make, what's your expectations of zookeeper 
community at this new year?
Notes:[1] The statistics are generated by GitStats, download mail attachment to 
explore more(view index.html)[2] Ranking by commits number

Re: Re: Pending issue for 3.6.0

[Justin Ling Mao]

---> "This is my point of view: Patch 1) introduces wire protocol changes (new 
RC), it is a big work and we are not ready to ship it, so I would defer to 
3.7.0."
Agree with this. Stability is much more import than including more 
features/improvements at the birth of this release.

- Original Message -
From: David Mollitor 
To: dev@zookeeper.apache.org
Subject: Re: Pending issue for 3.6.0
Date: 2019-12-24 04:20

Hello,
I would also like to request ZOOKEEPER-3342.  I just cleaned up the
latest PR.
On Mon, Dec 23, 2019 at 11:17 AM Enrico Olivelli 
wrote:
> Hi folks,
> I am keeping an eye on issues that the community tagged with fixVersion =
> 3.6.0
>
> This is the JIRA filter:
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZooKeeper%20%20and%20fixVersion%20%20%3D%203.6.0%20and%20resolution%20is%20EMPTY%20
>
> We have 3 pending works:
> 1) https://issues.apache.org/jira/browse/ZOOKEEPER-3301 Enforce Quota
> limit
> (by Mao Ling)
> 2) https://issues.apache.org/jira/browse/ZOOKEEPER-3512 Real time data
> integrity check during broadcast time (by Fangmin)
> 3) https://issues.apache.org/jira/browse/ZOOKEEPER-1112 Add support for C
> client for SASL authentication (by Damien Diederen)
>
> This is my point of view:
> Patch 1) introduces wire protocol changes (new RC), it is a big work and we
> are not ready to ship it, so I would defer to 3.7.0.
>
> Patch 2) Micheal Han and me have already reviewed it, but it is waiting for
> last ack from Andor , and it will need one last rebase (I am sorry Fangmin)
>
> Patch 3) is big, and it needs careful reviews from C guys.
>
> So my proposal is to:
> - move 1) and 3) to 3.7.0
> - wait for 2)
>
> Once 2) lands to branch-3.6 I will run the build.
>
> It would be super great If you have time to build branch-3.6 and test it,
> expecially manual tests about new features.
> This is the link to the draft of the release notes:
>
> https://github.com/apache/zookeeper/blob/branch-3.6/zookeeper-docs/src/main/resources/markdown/releasenotes.md
>
> Best regards
> Enrico
>

Re: Re: ZK makes apache 2019 "top 5" projects

[Justin Ling Mao]

https://www.cbronline.com/list/top-five-apache-software-projects-in-2019Chinese 
users can visit this due to GFW :)
- Original Message -
From: tison 
To: u...@zookeeper.apache.org
Cc: DevZooKeeper 
Subject: Re: ZK makes apache 2019 "top 5" projects
Date: 2019-12-12 19:53

Kudos!
Best,
tison.
Patrick Hunt  于2019年12月12日周四 上午11:32写道：
> This is really awesome, check it out:
> https://twitter.com/phunt/status/1204966326118141952
>
> Kudos ZooKeeper community on all the hard work and efforts!
>
> Patrick
>

Re: Re: Hadoop logo

[Justin Ling Mao]

+1, I have created a ticket for this: 
https://issues.apache.org/jira/browse/ZOOKEEPER-3648. If someone is interested 
in, please pick it ip.
BTW: the github project description(Mirror of Apache Hadoop ZooKeeper) may also 
needs a update.
- Original Message -
From: Michael Han 
To: dev@zookeeper.apache.org
Subject: Re: Hadoop logo
Date: 2019-12-13 09:31

+1
I do see a book keeper gif file in repo, should remove that one too.
On my wish list: redesign of the ZooKeeper logo :)
On Thu, Dec 12, 2019 at 7:44 AM Enrico Olivelli  wrote:
> +1
>
> Maybe we should also check if we have old pages about Bookkeeper project.
> It was a subproject of ZK but now it is a (great) top level independent
> project
>
> Enrico
>
> Il gio 12 dic 2019, 16:38 Flavio Junqueira  ha scritto:
>
> > ZooKeeper was a subproject of Hadoop in the early Apache days, and we
> > still carry that flag... ;-)
> >
> > -Flavio
> >
> > > On 12 Dec 2019, at 16:16, Norbert Kalmar  >
> > wrote:
> > >
> > > Oh, wow, I didn't even notice that until now.
> > > Makes sense, knowing a lot of the time ZK is used "standalone" (I mean
> > > outside of any hadoop ecosystem).
> > >
> > > Regards,
> > > Norbert
> > >
> > > On Thu, Dec 12, 2019 at 2:52 PM Flavio Junqueira 
> wrote:
> > >
> > >> Should we remove that Hadoop logo from the documentation? It has been
> a
> > >> while that we aren't a subproject of Hadoop any longer.
> > >>
> > >> -Flavio
> >
> >
>

Re: Re: Re: Re: Re: [VOTE] Apache ZooKeeper release 3.5.6 candidate 4

[Justin Ling Mao]

+1 (non-binding)A benchmark test for 3.5.6 candidate 4 with YCSB tool:
Machine Env:my local(8core,16G, 256SSD)
ZooKeeper Env: 3.5.6-c11b7e26bc554b8523dc929761dd28808913f091, built on 
10/08/2019 20:18 GMTstandalone mode(one zk server); 100 znodes;   data 
length(per znode):100;default configuration without any tuning.
Benchmark result:PS: workloada(read:50%,write:50%); 
workloadb(read:95%,write:5%); workloadc(read:100%)

- Original Message -
From: Zili Chen 
To: DevZooKeeper 
Subject: Re: [VOTE] Apache ZooKeeper release 3.5.6 candidate 4
Date: 2019-10-14 06:55

+1 (non-binding)
+ verify source tarball contains no binary files
+ verify binary tarball contains no source files
+ locally build by `mvn clean install -DskipTests -Pfull-build)` and verify
by `mvn verify`, tests pass
+ Verified basic zookeeper operation through Cli
Best,
tison.
Mohammad arshad  于2019年10月11日周五 下午2:38写道：
> +1 (non-binding)
> Verified checksums and signature
> Run UT on ubuntu 16.04 with jdk1.8.0_221 with non-root user. All 2554 UTs
> passed
> Verified basic zk operation through Cli
> Executed 4 letter word commands
> Verified admin server commands
> Ran rat
> All are OK :-)
>
>
> Thanks & Regards
> Mohammad Arshad
>
> -Original Message-
> From: sujith simon [mailto:sujithsimo...@gmail.com]
> Sent: Friday, October 11, 2019 11:32 AM
> To: dev@zookeeper.apache.org
> Subject: Re: [VOTE] Apache ZooKeeper release 3.5.6 candidate 4
>
> +1 (non-binding)
>
> - Verified Checksum
> - Verified Signature
> - Verfied that tags are correct
> - Verified build, installed ZooKeeper and tested basic commands from
> source tarball
> - Installed ZooKeeper from bin tarball on a 3 node cluster and tested
> basic ZooKeeper commands
> - Verified that Unit tests are passing
>
> Thanks :)
>
>
> On Thu, Oct 10, 2019 at 2:53 PM Norbert Kalmar
> 
> wrote:
>
> > +1 (non-binding)
> >
> > - unit tests pass
> > - built and started ZK + run few commands from source tarball
> > - checked bin tarball, license files, run ZK + few commands
> > - signature OK.
> > - git tag OK :)
> >
> > Thanks Enrico!
> >
> > Norbert
> >
> > On Wed, Oct 9, 2019 at 10:45 PM Andor Molnar  wrote:
> >
> > > +1
> > >
> > > Verified…
> > >
> > > - checksums, signature,
> > > - unit tests
> > > - 3-node cluster smoke tests.
> > >
> > > Andor
> > >
> > >
> > > > On 2019. Oct 9., at 22:40, Enrico Olivelli 
> > wrote:
> > > >
> > > > Il mer 9 ott 2019, 21:14 Patrick Hunt  ha scritto:
> > > >
> > > >> +1 checksums/sig validated. rat ran clean and I was able to build
> > > >> +and
> > > >> exercise the code just fine with java 8.
> > > >>
> > > >> Note dep check is failing again however:
> > > >>
> > > >> jackson-databind-2.9.10.jar
> > > >> (pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.9.10,
> > > >> cpe:2.3:a:fasterxml:jackson:2.9.10:*:*:*:*:*:*:*,
> > > >> cpe:2.3:a:fasterxml:jackson-databind:2.9.10:*:*:*:*:*:*:*) :
> > > >> CVE-2019-16942, CVE-2019-16943
> > > >>
> > > >> I looked at the issue and they seem very specific, given that and
> > > >> the status of databind these days I think we should get this one
> > > >> next time around vs re-re... spinning the rc. What do you think?
> > > >>
> > > >
> > > > Agreed.
> > > > And as we are doing a very limited use of Jackson we can look for
> > > > a replacement
> > > >
> > > > Enrico
> > > >
> > > >>
> > > >> Patrick
> > > >>
> > > >>
> > > >> On Tue, Oct 8, 2019 at 1:46 PM Enrico Olivelli
> > > >> 
> > > >> wrote:
> > > >>
> > > >>> This is a bugfix release candidate for 3.5.6.
> > > >>>
> > > >>> It fixes 29 issues, including upgrade of third party libraries,
> > > >>> TTL Node APIs for C API, support for PCKS12 Keystores, upgrade
> > > >>> of
> > > Netty 4
> > > >>> and better procedure for the upgrade of servers from 3.4 to 3.5.
> > > >>>
> > > >>> The full release notes is available at:
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310
> > 801=12345243
> > > >>>
> > > >>> *** Please download, test and vote by October 11th 2019, 23:59
> UTC+0.
> > > ***
> > > >>>
> > > >>> Source files:
> > > >>> https://people.apache.org/~eolivelli/zookeeper-3.5.6-candidate-4
> > > >>>
> > > >>> Maven staging repo:
> > > >>>
> > > >>
> > >
> > https://repository.apache.org/content/repositories/orgapachezookeeper-
> > 1044
> > > >>>
> > > >>> The release candidate tag in git to be voted upon:
> > > >>> release-3.5.6-rc4
> > > >>> https://github.com/apache/zookeeper/tree/release-3.5.6-rc4
> > > >>>
> > > >>> ZooKeeper's KEYS file containing PGP keys we use to sign the
> release:
> > > >>> https://www.apache.org/dist/zookeeper/KEYS
> > > >>>
> > > >>> Should we release this candidate?
> > > >>>
> > > >>> Enrico Olivelli
> > >
> > >
> >
>

Re: Re: Coding Style

[Justin Ling Mao]

1. AFAIK, the IDEA also supports  Eclipse Format Template(install Eclipse Code 
Formatter plugin for IDEA) , so no duplicated work.
2. Since we now use a customized code checkstyle, so we cannot take advantage 
of the ready-make sun/google formatter, we also need a customized formatter, 
just like 
this:https://github.com/apache/hbase/blob/master/dev-support/hbase_eclipse_formatter.xml
3. Overall, this work is not in a hurry because mvn cmd can tell us where is 
the violations
- Original Message -
From: Zili Chen 
To: Justin Ling Mao 
Cc: dev 
Subject: Re: Coding Style
Date: 2019-10-08 10:52

Hi Justin,
If Eclipse has a checkstyle plugin as that in IDEA I don't think we have 
tomaintain two versions of code style rules. Otherwise we have to sync onewith 
the other when we add/remove rules, which is unnecessarily overhead.
Best,tison.

Justin Ling Mao  于2019年10月8日周二 上午10:49写道：
1. Currently, our coding style checker(maven checkstyle plugin) doesn't abide 
by Sun's conventions completely

2. maybe we need a eclipse_formatter.xml to  the automatic formatting for the 
IDEs,  https://hbase.apache.org/book.html#eclipse.code.formatting is a good 
example.

- Original Message -

From: David Mollitor 

To: dev@zookeeper.apache.org

Subject: Coding Style

Date: 2019-10-08 03:10



Hello,

I am looking at this page for a coding style format / template.

https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute

There is a note about it with a link to: "Sun's conventions" (however, the

link is dead)

Can someone propose an Eclipse Format Template which has the required

coding standards required by this project and submit them into 'dev' folder

in GitHub?

Thanks!

Re: Coding Style

[Justin Ling Mao]

1. Currently, our coding style checker(maven checkstyle plugin) doesn't abide 
by Sun's conventions completely
2. maybe we need a eclipse_formatter.xml to  the automatic formatting for the 
IDEs,  https://hbase.apache.org/book.html#eclipse.code.formatting is a good 
example.
- Original Message -
From: David Mollitor 
To: dev@zookeeper.apache.org
Subject: Coding Style
Date: 2019-10-08 03:10

Hello,
I am looking at this page for a coding style format / template.
https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute
There is a note about it with a link to: "Sun's conventions" (however, the
link is dead)
Can someone propose an Eclipse Format Template which has the required
coding standards required by this project and submit them into 'dev' folder
in GitHub?
Thanks!

Re: Re: [ANNOUNCE] New ZooKeeper committer: Brian Nixon

[Justin Ling Mao]

congrats Brian, well-deserved.
- Original Message -
From: Enrico Olivelli 
To: DevZooKeeper 
Subject: Re: [ANNOUNCE] New ZooKeeper committer: Brian Nixon
Date: 2019-10-02 18:23

Yeah, congrats Brian, welcome aboard !
Enrico
Il giorno mer 2 ott 2019 alle ore 11:15 Zili Chen  ha
scritto:
> Congratulations Brian!
>
> Best,
> tison.
>
>
> Andor Molnar  于2019年10月2日周三 下午5:11写道：
>
> > The Apache ZooKeeper PMC recently extended committer karma to Brian and
> he
> > has accepted. Brian has made some great contributions and we are looking
> > forward to even more :)
> >
> > Congratulations and welcome aboard Brian!
> >
> > Andor
> >
> >
> >
>

Re: PoweredBy Zookeeper

[Justin Ling Mao]

1.Thanks Olivelli for this letter. ZOOKEEPER-3529 has now landed.   1.1 The 
references may be a good reading material and they're are public and free. 
welcome scrutiny.   1.2 The documentation has a declaration about intellectual 
property rights and privacy.2 If you have a use case, please do not hesitate, 
submit a pull request or write an email to **dev@zookeeper.apache.org**

- Original Message -
From: Enrico Olivelli 
To: dev@zookeeper.apache.org
Subject: PoweredBy Zookeeper
Date: 2019-09-14 20:22

Hi,
Maoling started this initiative to have a great Zookeeper use cases page:
https://github.com/apache/zookeeper/pull/1073
I think this is a good idea, but before 'approving' that change I feel we
must discuss more about it.
The major point for me is whether we should ask for consent for publishing
such information
Usually maintainers of projects actively adk for their project to be added,
with this page we are citing thid party productd and companies
Once we clear this point I will be happy to merge that change.
Thanks Maoling
Enrico

Re: Re: Cutting Zookeeper 3.5.6

[Justin Ling Mao]

awesome!!!. Looking forward to this release.
- Original Message -
From: Andor Molnar 
To: DevZooKeeper 
Subject: Re: Cutting Zookeeper 3.5.6
Date: 2019-09-02 00:28

Great +1
On Thu, Aug 29, 2019 at 4:37 PM Enrico Olivelli  wrote:
> Hello,
> I am going to cut 3.5.6, if you have patches to cherry pick or you are
> aware of blocker issues for such release please tell me as soon as
> possible.
>
> I will start the procedure tomorrow
>
>
> Enrico
>

Re: Re: Consistency Guarantees

[Justin Ling Mao]

@Karolos Thanks for this interesting discussion.Let me quote some words from 
the Book:《Designing Data-Intensive Applications》Martin Kleppmann. If you have 
more interested in this topic, have a fun to read the CHAPTER 9:Consistency and 
Consensus--->"While this procedure ensures linearizable writes, it doesn’t 
guarantee linearizable
reads—if you read from a store that is asynchronously updated from the log, it 
may
be stale. (To be precise, the procedure described here provides sequential 
consistency
[47, 64], sometimes also known as timeline consistency [65, 66], a slightly 
weaker
guarantee than linearizability.) To make reads linearizable, there are a few 
options:









You can sequence reads 
through the log by appending a message, reading the log,
and performing the actual read when the message is delivered back to you. The
message’s position in the log thus defines the point in time at which the read
happens. (Quorum reads in etcd work somewhat like this [16].)



If the log allows you 
to fetch the position of the latest log message in a linearizable way, you can 
query that position, wait for all entries up to that position to be
delivered to you, and then perform the read. (This is the idea behind Zoo‐
Keeper’s sync() operation [15].)



You can make your read 
from a replica that is synchronously updated on writes,
and is thus sure to be up to date. (This technique is used in chain replication
[63]; see also “Research on Replication” on page 155.) "
- Original Message -
From: Karolos Antoniadis 
To: dev@zookeeper.apache.org
Subject: Re: Consistency Guarantees
Date: 2019-08-19 03:14

Hi Jordan,
When Aphyr tested ZooKeeper, he did not seem to know that it is not
linearizable. See here: https://github.com/jepsen-io/jepsen/issues/399, where
I pointed-out that even with *sync + read*, ZooKeeper might return stale
data.
ZooKeeper can only be considered linearizable if we assume that specific
timing constraints apply. Naturally, in a real system, we cannot make such
assumptions if we want to be 100% safe.
For instance, if the time(TCP timeout) > (syncLimit * tickTime), ZooKeeper
provides linearizable reads. However, if this does not hold (e..g, skewed
clocks), then ZooKeeper might return stale data.
To conclude, I do not think we can argue that ZooKeeper is linearizable.
Cheers,
Karolos
On Sun, 18 Aug 2019 at 11:34, Jordan Zimmerman 
wrote:
> ZooKeeper _is_ linearizable. I’m pretty sure the ZAB paper talks about it.
> Aphyr does as well here: https://aphyr.com/posts/291-jepsen-zookeeper
>
> 
> Jordan Zimmerman
>
> > On Aug 18, 2019, at 1:23 PM, Karolos Antoniadis 
> wrote:
> >
> > Hello everyone,
> >
> > I was wondering on the exact consistency guarantees that ZooKeeper
> provides.
> > It seems that ZooKeeper does not provide strong consistency (i.e.,
> > linearizability) since reads could potentially return arbitrarily old
> > values.
> > On the other hand, ZooKeeper provides sequential consistency, since the
> > order of operations of a specific client is respected and all operations
> > appear to take place in some total order (
> > https://jepsen.io/consistency/models/sequential).
> > However, ZooKeeper provides linearizable writes, and therefore it
> provides
> > something stronger than sequential consistency, but still not as strong
> as
> > linearizability. In other words, ZooKeeper guarantees are somewhere
> between
> > sequential consistency and linearizability.
> > Is there a specific name for the specific consistency guarantees that
> > ZooKeeper provides?
> > What would the ZooKeeper community claim about the consistency guarantees
> > of ZooKeeper?
> >
> > Best Regards,
> > Karolos
>

Re: Upgrade FAQ

[Justin Ling Mao]

@andor
IMHO, we need a documentation about how to do the rolling-upgrade between 
different release in the code base.
Am I right?
- Original Message -
From: Andor Molnar 
To: DevZooKeeper 
Subject: Upgrade FAQ
Date: 2019-08-16 23:39


Hi,
I’ve created the following FAQ page on the wiki to collect all upgrade related 
issues and workarounds.
https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ
Haven’t found any existing doc on the topic, hence I’ve created this one 
quickly. We might want to move it to the documentation in the long run.
Please add anything that you’ve come across already.
Andor

Re: Re: complete quota system proposal

[Justin Ling Mao]

1. IMHO, before that work, we need to merge the 
ZOOKEEPER-3301(https://github.com/apache/zookeeper/pull/934) firstly(：D) which 
has a good expansibility to the quota storage and support the soft/hard quota 
limit and some good refactors and utils for the quota.
2.In the ZOOKEEPER-1383, Thawan Kooburat has implemented a straightforward 
throughput quota based on the time windows.   A tip is:   for the soft limit, 
slow down the w/r requests; for the hard limit, just throws the 
QuotaExceededException.
3.how to identify the client? cxid, hostname, ip or a unique name users 
specify? the client side supports the client group? 

- Original Message -
From: Mocheng Guo 
To: dev@zookeeper.apache.org
Subject: Re: complete quota system proposal
Date: 2019-07-21 06:53

Good point and I missed it. Yes client identity inside certificate can be
used if connection is secured, otherwise we could use existing zk auth
protocol to enable client sending its id to server.
0. client identification
1. quota configuration - metric key and value, format, storage
2. metrics collection and export - storage/rate/watch/connection
3. throttling implementation based on metrics inside server/client
On Thu, Jul 18, 2019 at 9:18 PM Michael Han  wrote:
> Sounds good. I think we also need a plan on how to identify clients. If
> auth is enabled, we might be able to reuse some of the auth information,
> but a generalized client-id based approach sounds better.
>
> On Thu, Jul 18, 2019 at 11:32 AM Mocheng Guo  wrote:
>
> > Hi Michael, thanks for your feedback and I will create an epic for this.
> I
> > will divide into following stories and please let me know if this makes
> > sense
> >
> > 1. quota configuration - metric key and value, format, storage
> > 2. metrics collection and export - storage/rate/watch/connection
> > 3. throttling implementation based on metrics inside server/client
> >
> > On Wed, Jul 17, 2019 at 8:02 PM Michael Han  wrote:
> >
> > > >> a more complete quota/throttling system would be valuable here
> > >
> > > Absolutely. This will improve the stability of a shared zookeeper
> > cluster,
> > > which is a very common use case. Because the lack of enforce quota (and
> > > other soft isolation mechanisms such as flow control / request
> limiting),
> > > the usual practice of dealing with such excessive client usages is to
> > > provision dedicated ZK cluster per customer which creates additional
> > > operation overheads and is waste of resource.
> > >
> > > >> We'd be happy to submit the work for review
> > >
> > > Sounds great. I would suggest we create an epic around this and then
> > break
> > > down the stories to multiple sub tasks and move existing issues that's
> > > relevant under the epic, so we can consolidate efforts and have a
> single
> > > place to track the progress.
> > >
> > >
> > > On Tue, Jul 16, 2019 at 12:46 PM Mocheng Guo 
> wrote:
> > >
> > > > Hi, currently Zookeeper has a storage quota feature which is
> > > informational
> > > > only and there are a few JIRAs to enforce throttling.
> > > >
> > > > ZOOKEEPER-451
> > > > ZOOKEEPER-2593
> > > > ZOOKEEPER-3301
> > > >
> > > > I am wondering if a more complete quota/throttling system covering
> > > metrics
> > > > such as storage/rate/watch/connection would be valuable here? We at
> FB
> > > have
> > > > been battling with excessive system usage from clients and did some
> > work
> > > in
> > > > this area. We'd be happy to submit the work for review and
> consolidate
> > > with
> > > > existing efforts if people feel this is a good feature to add.
> > > >
> > > > thanks
> > > > Mocheng
> > > >
> > >
> >
>

Re: Re: Re: Clean up the all the checkstyle violations in the zookeeper-server module

[Justin Ling Mao]

@Zili  don't worry about anything.I'am here.My first-shot patch will be coming 
at this weekend, then you can help me review it.
- Original Message -
From: Zili Chen 
To: DevZooKeeper 
Subject: Re: Re: Clean up the all the checkstyle violations in the 
zookeeper-server module
Date: 2019-07-19 06:04

Hi Enrico,
I see your concern. My question is, how does "a new execution of checkstyle"
implement? My experience with checkstyle plugin is one pass to check all
rules so I'm curiously whether we add another profile to execute or
in other ways.
Best,
tison.
Enrico Olivelli  于2019年7月18日周四 下午8:46写道：
> Hi Tison,
>
>
>
> Il gio 18 lug 2019, 10:40 Zili Chen  ha scritto:
>
> > Hi Enrico,
> >
> > I just noticed that you have mentioned
> >
> > We can keep current checkstyle invokation that checks for @author tags
> as a
> > separate 'execution' of the plugin with a specific checkstyle file (as
> you
> > already said)
> >
> > I'm curious how "a separated execution" achieves? I found
> > that checkstyle plugin can be only configured one checkstyle
> > conifg files and thus propose the current simple config applied
> > on all modules and a strict config covering the simple one
> > applied per package(server, prometheus, and jute as discussed).
> >
>
>
> We already have one checkstyle file and it must be checked against the
> whole codebase, ad it looks for @author tags.
>
> If we add new rules to the existing file they will be enabled for all of
> the modules.
> If you limit checkstyle to only some package we will lose current checks
>
> So my idea is to add a new execution of checkstyle with a new checkstyle
> configuration file, for this new execution of the plugin we will have:
> - strict  checks
> - selective per-package abilitation of checkstyle
>
>
>
> >
> > Best,
> > tison.
> >
> >
> > Enrico Olivelli  于2019年7月6日周六 下午8:20写道：
> >
> > > Justin,
> > > This is how we did it in Bookkeeper, we enabled checkstyle only for
> group
> > > of packages in the main module (the biggest one, bookkeeper-server)
> > >
> > > https://github.com/apache/bookkeeper/issues/230
> > >
> > > I suggest using that checkstyle config, I feel we won't have so many
> > > violations.
> > >
> > > We can keep current checkstyle invokation that checks for @author tags
> > as a
> > > separate 'execution' of the plugin with a specific checkstyle file (as
> > you
> > > already said)
> > >
> > > I am happy to help, thank you for driving this effort
> > >
> > > Enrico
> > >
> > >
> > > Il sab 6 lug 2019, 11:33 Justin Ling Mao 
> ha
> > > scritto:
> > >
> > > > - 1.It seems that we had reached a consensus to work on this.- 2.I
> also
> > > > agree on the way: fix one package at a time, then another.- 3.Now Let
> > us
> > > > discuss some details:- 3.1 how to make the checkstyle only check
> > the
> > > > package we specify? I found this:  URL:
> > > >
> > >
> >
> https://stackoverflow.com/questions/26455174/only-enable-some-checks-for-certain-inner-package
> > > > @Olivelli Could you give me more your insight?- 3.2 What
> rules
> > > will
> > > > we init in the checkstyle.xml?   3.2.1 - I also think the rules
> > from
> > > > the hbase is too strict which will cause too many,many violations.
> > > >  3.2.2 - apply the "Google's Java Style Checkstyle Coverage" is a
> good
> > > > option? which seems to be more simplify and more suitable for us?
> > > >  URL:https://checkstyle.sourceforge.io/google_style.html
> > > >
> > > >
> > > >
> > > > - Original Message -
> > > > From: Andor Molnar 
> > > > To: DevZooKeeper 
> > > > Subject: Re: Clean up the all the checkstyle violations in the
> > > > zookeeper-server module
> > > > Date: 2019-07-02 13:22
> > > >
> > > > Yes. That way we only need to fix one package at a time.
> > > > Andor
> > > > On Mon, Jul 1, 2019 at 4:10 PM Zili Chen 
> wrote:
> > > > > Hi Andor,
> > > > >
> > > > > To be exact, "iterations" means we define the original rules
> > > > > in checkstyle configuration at once and turn them on one package
> > > > > after another, so iterations. Is it correct?
> > > > >
> > > &g

Re: Re: Re: Re: Re: Clean up the all the checkstyle violations in the zookeeper-server module

[Justin Ling Mao]

-1. @Zili Chen   I had linked ZOOKEEPER-3434(closed),ZOOKEEPER-3446 to the 
ZOOKEEPER-3431 which now is a Umbrella JIRA (Type:Task).   I will also take 
your advice about the subtasks.-2 --->"Please add jute and Prometheus module"   
@Olivelli.That's OK.

- Original Message -
From: Zili Chen 
To: DevZooKeeper 
Cc: maoling199210...@sina.com
Subject: Re: Re: Re: Re: Clean up the all the checkstyle violations in the 
zookeeper-server module
Date: 2019-07-16 15:18

The main concern here is that we have already toomany issues on enable specific 
rules on zookeeper,including ZOOKEEPER-3434 and ZOOKEEPER-3446,and it would be 
quite noisy to enable per rule(asbeen described and reached a consensus).
Best,tison.

Zili Chen  于2019年7月16日周二 上午9:17写道：
Hi Justin,
Thanks for driving this thread. Please go ahead!
One thing I'd like to pick up is that ZOOKEEPER-3431has a specific description 
and I'm afraid it could notbe an umbrella issue.
How about close all checkstyle related issues and starta new issues structure as
Umbrella: Enable Google checkstyle configuration  Subtask-1: Add silent Google 
checkstyle configuration  Subtask-2: Enable Google checkstyle configuration on 
zookeeper-server  Subtask-3: Enable Google checkstyle configuration on 
zookeeper-jute
  Subtask-4: Enable Google checkstyle configuration on zookeeper-prometheus
  ...
Best,tison.

Enrico Olivelli  于2019年7月16日周二 上午12:06写道：
Il lun 15 lug 2019, 09:14 Justin Ling Mao  ha

scritto:



> - any advance for the discussion???- any objections about these two

> things: 1.only clean the main-module:zookeeper-server;





Please add jute and Prometheus module



2.using the google's checkstyle_style?-





Works for me



> who will head it up?  how about me？

>



Sure! Go for it. Thanks



Enrico



>

>

> - Original Message -

> From: "Justin Ling Mao" 

> To: "dev" 

> Subject: Re: Re: Re: Clean up the all the checkstyle violations in the

> zookeeper-server module

> Date: 2019-07-07 15:56

>

> 1.--->“we'd better first create an umbrella issue named "Enable checkstyle

> rules" or sth”I had created ZOOKEEPER-3431 previously, and we can create a

> series of sub-tasks under it.

> 2.I think we still have two things which should be discussed:  2.1

> Currently, we only need to enforce the checkstyle violations check in the

> main-module:zookeeper-server, not included other modules?  IMO, because

> the zookeeper-contrib, zookeeper-recipes are now not well-maintained.

> and some violations in the zookeeper-jute are auto-generated. so focusing

> on zookeeper-server is enough?

>   2.2 What checkstyle template we will pick up? Now we have three

> options:  A:[google_style](

> https://checkstyle.sourceforge.io/google_style.html)

> B:[bookkeeper_style] (

> https://github.com/apache/bookkeeper/blob/master/buildtools/src/main/resources/bookkeeper/checkstyle.xml)

> C:[hbase_style](

> https://github.com/apache/hbase/blob/master/hbase-checkstyle/src/main/resources/hbase/checkstyle.xml)

> Which one will we choose?

>

>

> - Original Message -

> From: Enrico Olivelli 

> To: dev@zookeeper.apache.org

> Cc: maoling199210...@sina.com

> Subject: Re: Re: Clean up the all the checkstyle violations in the

> zookeeper-server module

> Date: 2019-07-07 15:13

>

> Il dom 7 lug 2019, 01:29 Zili Chen  ha scritto:

> > Justin & Enrico,

> >

> > Receiving no opposition on this proposal, we could regard it as

> > a consensus. According to bookkeeper#230 we'd better first create

> > an umbrella issue named "Enable checkstyle rules" or sth. Under

> > there we can finally decide the checkstyle configuration and

> > start sub-tasks enabling per package.

> >

> > For keeping current checkstyle, I'd like to pick up that it's

> > possible that we remain the current simple config for all pkgs,

> > adding a config said copied from bookkeeper named

> > "strict-checkstyle.xml", enabling per pkg, which contains @author

> > tags and rules in simple config. Once we enabling the strict one

> > for all pkgs. We can merge two configs into one.

> >

> +1 please go ahead

> Enrico

> > Best,

> > tison.

> >

> >

> > Enrico Olivelli  于2019年7月6日周六 下午8:20写道：

> >

> > > Justin,

> > > This is how we did it in Bookkeeper, we enabled checkstyle only for

> group

> > > of packages in the main module (the biggest one, bookkeeper-server)

> > >

> > > https://github.com/apache/bookkeeper/issues/230

> > >

> > > I suggest using that checkstyle config, I feel we won't have so many

> > > violations.

> > >

> > > We can keep c

Re: Re: Re: Time to think about a 3.6.0 release?

[Justin Ling Mao]

- 1. @Andor Thanks for reformatting my email. I must be careful next time.








- 2. I absolutely agree with what Olivelli said.  --> “I guess that if we 
start now we can have a 3.6 in September”.   maybe not need so fast. But 
having a planning, especially about what features will be included, what 
approximate time(A time range) will cut 3.6.0 is a good idea  drawing from the 
previous lessons- 3. I have a question: "After 3.6.0 has landed, the 3.5.* will 
not be applied any new feature and only fix the bugs and prominent 
improvements" Is it right?



- Original Message -
From: Enrico Olivelli 
To: DevZooKeeper , maoling199210...@sina.com
Subject: Re: Re: Time to think about a 3.6.0 release?
Date: 2019-07-15 21:55

Justin,
I think that current master has already plenty of new features and it is
worth to start thinking to a release.
The more with add code the more we add risk.
The point of this thread is more about 'stop adding new stuff, complete
ongoing work and start a rampdown phase', it is not "cut a release right
now"
As far as I know current master is very different from branch 3.5,
expecially on server side code, lots of feature came in and stalled on
master branch for months or even years,
so feedback from 3.5 is useful but not so blocker
We should make current master stable
IMHO the recipe for a great release is:
1) enough stuff committed to the release branch sothat it is worth to cut a
release
2) code in good shape: tests are passing, automatic checks are passing
(spotbugs, rat...)
3) licensing stuff is okay
4)  upgrade instructions and changelog about breaking changes/new
beheaviours are complete
5) CI is doing well
6) consensus of the community about the release
Hopefully now that we got out of the 3.5-BETA  problem and we are stable we
can think about a time based release schedule, if a feature can't be
delivered on a release it won't pass so much time for a new release, say
3-4 months
I don't know how many companies are using "current master" (or something
like that) in production, I feel that running 3.5 does not tell very much
about the stability of current 3.6
So my plan would be:
- merge pending pull requests that are ready
- stabilize the codebase  (no more BLOCKER issues for the release)
- start release process
I guess that if we start now we can have a 3.6 in September
Enrico
Il lun 15 lug 2019, 11:55 Justin Ling Mao  ha
scritto:
> - 1.Since the 3.5.5 has just released in May. we still need some time to
> collect the users' feedback.we cannot make sure the release time of 3.6.0?
> Giving the experience from the previous release history:)- 2.please Let me
> share some my thoughts, and the work in progress will be arriving into
> 3.6.0. Plz correct me if I got something wrong.
>  
> --P0
>- Support the backend store engine:LMDB. this work needs a very detailed
> proposal which I will send to the community for being discussed fully.
>  - Add a complete backup mechanism for zookeeper internal(PR-917) which I
> will sharp it this week. - A very powerful benchmark tool(PR-1011)
> which will be available within these two week. - improve the
> performance of read/write to have the distinct advantages compared to etcd
> v3.4 which will be released soon. - To strengthen the quota
> feature(PR-934,PR-936,PR-938) and implement the throughout quota. - To
> strengthen the implements of TTL node(PR-1010) - Add some new very
> useful CLIs: quorumInfo, watch .etc - Observe and strengthen the new
> metric system continuously.
>  
> --P1
>- strength the docs, especially about the c client, local session,
> security(TLS),ZAB protocol .etc - introduce some chaos, fuzzy tests and
> tools to hit and check the zk. - Clean up the all the checkstyle
> violations in the zookeeper-server module(ZOOKEEPER-3431)
>  -
> P2-- -
> Debug mode feature. Look at an example of redis - the tracing
> feature(PR-994). if having another time, integrating with opentracing
> sounds a very good idea. - replace jute with thrift or PB may be put
> into the 4.0.0 when wanting to break the backward compatibility? And at the
> 4.0.0, implementing the restful api is also a  very good idea.
>
>
>
> - Original Message -
> From: Fangmin Lv 
> To: dev@zookeeper.apache.org
> Subject: Re: Time to think about a 3.6.0 release?
> Date: 2019-06-26 07:33
>
> It's great to have a 3.6.0 release, currently all the FB contributed
> features has been running inside FB for more th

Re: Re: Time to think about a 3.6.0 release?

[Justin Ling Mao]

- 1.Since the 3.5.5 has just released in May. we still need some time to 
collect the users' feedback.we cannot make sure the release time of 3.6.0? 
Giving the experience from the previous release history:)- 2.please Let me 
share some my thoughts, and the work in progress will be arriving into 3.6.0. 
Plz correct me if I got something wrong. 
--P0
 - Support the backend store engine:LMDB. this work needs a very detailed 
proposal which I will send to the community for being discussed fully. - 
Add a complete backup mechanism for zookeeper internal(PR-917) which I will 
sharp it this week. - A very powerful benchmark tool(PR-1011) which will be 
available within these two week. - improve the performance of read/write to 
have the distinct advantages compared to etcd v3.4 which will be released soon. 
- To strengthen the quota feature(PR-934,PR-936,PR-938) and implement the 
throughout quota. - To strengthen the implements of TTL node(PR-1010) - 
Add some new very useful CLIs: quorumInfo, watch .etc - Observe and 
strengthen the new metric system continuously. 
--P1
 - strength the docs, especially about the c client, local session, 
security(TLS),ZAB protocol .etc - introduce some chaos, fuzzy tests and 
tools to hit and check the zk. - Clean up the all the checkstyle violations 
in the zookeeper-server module(ZOOKEEPER-3431) 
- 
P2-- - 
Debug mode feature. Look at an example of redis - the tracing 
feature(PR-994). if having another time, integrating with opentracing sounds a 
very good idea. - replace jute with thrift or PB may be put into the 4.0.0 
when wanting to break the backward compatibility? And at the 4.0.0, 
implementing the restful api is also a  very good idea.



- Original Message -
From: Fangmin Lv 
To: dev@zookeeper.apache.org
Subject: Re: Time to think about a 3.6.0 release?
Date: 2019-06-26 07:33

It's great to have a 3.6.0 release, currently all the FB contributed
features has been running inside FB for more than a month, so it
should be stable enough for community to use.
Also I agreed with Patrick's point to review all flags and consider to turn
on by default.
For the pending PRs, the following might be higher priority and would be
nice to include in the 3.6.0 release:
* ZOOKEEPER-3356: Implement advanced Netty flow control based on feedback
from ZK to avoid OOM issue
* ZOOKEEPER-3145: Avoid watch missing issue due to stale pzxid when
replaying CloseSession txn with fuzzy snapshot
* ZOOKEEPER-3240: Close socket on Learner shutdown to avoid dangling socket
Thanks,
Fangmin
On Sat, Jun 15, 2019 at 9:21 AM Patrick Hunt  wrote:
> Good idea. Agree on including anything we've postponed to a new cycle - the
> patch from mapr is an obvious one to consider.
>
> We should also look at things we've disabled by default and consider
> whether we can turn them on by default. If not why not, and what can we do
> to fix this in a subsequent release.
>
> Have we deprecated anything that we should now remove?
>
> Also a good time to review the state of Java versions and make changes wrt
> supported versions and so forth.
>
> There was a proposal to remove contribs, or at least consider the ones that
> are still valuable vs moving some out. We should do that as well.
>
> Regards,
>
> Patrick
>
> On Sat, Jun 15, 2019 at 9:02 AM Jordan Zimmerman <
> jor...@jordanzimmerman.com>
> wrote:
>
> > On Persistent/Recursive watches: I’m willing to rebase, etc if there’s
> > confidence it will be merged.
> >
> > 
> > Jordan Zimmerman
> >
> > > On Jun 15, 2019, at 10:59 AM, Andor Molnar  >
> > wrote:
> > >
> > > Hi Enrico!
> > >
> > > Very good point, I entirely support the idea.
> > >
> > > Question to Friends@Facebook and Twitter contributors: how many
> > outstanding
> > > Jiras/PRs do you have which you would like to see in 3.6?
> > >
> > > I'd also like to highlight the long outstanding PR from Mapr:
> > > https://github.com/apache/zookeeper/pull/730
> > >
> > > And some great new features which are still looking for to be merged:
> > > - Persistent recursive watchers:
> > > https://github.com/apache/zookeeper/pull/136
> > > - Enforce client auth: https://github.com/apache/zookeeper/pull/118
> > > - Slow operation log
> > > - Jetty port unification
> > >
> > > Regards,
> > > Andor
> > >
> > >
> > >
> > >
> > >
> > >> On Sat, Jun 15, 2019 at 1:31 PM Enrico Olivelli 
> > wrote:
> > >>
> > >> Hi Zookeepers !
> > >> I checked on JIRA and it seems that master in good shape, no real
> > blockers
> > >> that mine the stability of the code.
> > >>
> > >> We have plenty of cool pull requests almost ready to be merged 

Re: Re: Re: Re: Clean up the all the checkstyle violations in the zookeeper-server module

[Justin Ling Mao]

- any advance for the discussion???- any objections about these two things: 
1.only clean the main-module:zookeeper-server;  2.using the google's 
checkstyle_style?- who will head it up?  how about me？


- Original Message -
From: "Justin Ling Mao" 
To: "dev" 
Subject: Re: Re: Re: Clean up the all the checkstyle violations in the 
zookeeper-server module
Date: 2019-07-07 15:56

1.--->“we'd better first create an umbrella issue named "Enable checkstyle 
rules" or sth”I had created ZOOKEEPER-3431 previously, and we can create a 
series of sub-tasks under it.
2.I think we still have two things which should be discussed:  2.1 Currently, 
we only need to enforce the checkstyle violations check in the 
main-module:zookeeper-server, not included other modules?  IMO, because the 
zookeeper-contrib, zookeeper-recipes are now not well-maintained.  and some 
violations in the zookeeper-jute are auto-generated. so focusing on 
zookeeper-server is enough?
  2.2 What checkstyle template we will pick up? Now we have three options:  
A:[google_style](https://checkstyle.sourceforge.io/google_style.html)  
B:[bookkeeper_style] 
(https://github.com/apache/bookkeeper/blob/master/buildtools/src/main/resources/bookkeeper/checkstyle.xml)
  
C:[hbase_style](https://github.com/apache/hbase/blob/master/hbase-checkstyle/src/main/resources/hbase/checkstyle.xml)
  Which one will we choose?


- Original Message -
From: Enrico Olivelli 
To: dev@zookeeper.apache.org
Cc: maoling199210...@sina.com
Subject: Re: Re: Clean up the all the checkstyle violations in the 
zookeeper-server module
Date: 2019-07-07 15:13

Il dom 7 lug 2019, 01:29 Zili Chen  ha scritto:
> Justin & Enrico,
>
> Receiving no opposition on this proposal, we could regard it as
> a consensus. According to bookkeeper#230 we'd better first create
> an umbrella issue named "Enable checkstyle rules" or sth. Under
> there we can finally decide the checkstyle configuration and
> start sub-tasks enabling per package.
>
> For keeping current checkstyle, I'd like to pick up that it's
> possible that we remain the current simple config for all pkgs,
> adding a config said copied from bookkeeper named
> "strict-checkstyle.xml", enabling per pkg, which contains @author
> tags and rules in simple config. Once we enabling the strict one
> for all pkgs. We can merge two configs into one.
>
+1 please go ahead
Enrico
> Best,
> tison.
>
>
> Enrico Olivelli  于2019年7月6日周六 下午8:20写道：
>
> > Justin,
> > This is how we did it in Bookkeeper, we enabled checkstyle only for group
> > of packages in the main module (the biggest one, bookkeeper-server)
> >
> > https://github.com/apache/bookkeeper/issues/230
> >
> > I suggest using that checkstyle config, I feel we won't have so many
> > violations.
> >
> > We can keep current checkstyle invokation that checks for @author tags
> as a
> > separate 'execution' of the plugin with a specific checkstyle file (as
> you
> > already said)
> >
> > I am happy to help, thank you for driving this effort
> >
> > Enrico
> >
> >
> > Il sab 6 lug 2019, 11:33 Justin Ling Mao  ha
> > scritto:
> >
> > > - 1.It seems that we had reached a consensus to work on this.- 2.I also
> > > agree on the way: fix one package at a time, then another.- 3.Now Let
> us
> > > discuss some details:- 3.1 how to make the checkstyle only check
> the
> > > package we specify? I found this:  URL:
> > >
> >
> https://stackoverflow.com/questions/26455174/only-enable-some-checks-for-certain-inner-package
> > > @Olivelli Could you give me more your insight?- 3.2 What rules
> > will
> > > we init in the checkstyle.xml?   3.2.1 - I also think the rules
> from
> > > the hbase is too strict which will cause too many,many violations.
> > >  3.2.2 - apply the "Google's Java Style Checkstyle Coverage" is a good
> > > option? which seems to be more simplify and more suitable for us?
> > >  URL:https://checkstyle.sourceforge.io/google_style.html
> > >
> > >
> > >
> > > - Original Message -
> > > From: Andor Molnar 
> > > To: DevZooKeeper 
> > > Subject: Re: Clean up the all the checkstyle violations in the
> > > zookeeper-server module
> > > Date: 2019-07-02 13:22
> > >
> > > Yes. That way we only need to fix one package at a time.
> > > Andor
> > > On Mon, Jul 1, 2019 at 4:10 PM Zili Chen  wrote:
> > > > Hi Andor,
> > > >
> > > > To be exact, "iterations" means we define the original rules

The ZooKeeper Grafana dashboard is now available

[Justin Ling Mao]

- Due to the work from 
ZOOKEEPER-3447(https://github.com/apache/zookeeper/pull/1008), the ZooKeeper 
Grafana dashboard is now available(https://grafana.com/dashboards/10465) which 
I had included some screenshots for the preview.
- Users can ask for this Grafana dashboard account if having any good 
improvements by writing a email to dev@zookeeper.apache.org.

Re: [patch] use zookeeper-client-c has some link problems

[Justin Ling Mao]

1.--->"I fixed it in a simple way and it looks fine."  1.1 how did you 
solve this issue?  1.2 It‘s a issue about your demo(may be a link or 
dependencies error)? not about the zookeeper c client?
2. Could you please tell me how to compile and build the zookeeper c client in 
the release-3.5.5 ? :)

- Original Message -
From: 孙权 
To: dev@zookeeper.apache.org
Subject: [patch] use zookeeper-client-c has some link problems
Date: 2019-07-09 02:58

Sorry for disturbance!
Description:use cmake generate zookeeper.sln, build generate cli.pdb
hashtable.lib
hashtable.pdb
zookeeper.lib
zookeeper.pdbI write a demo linked “zookeeper.lib” and “hashtable.lib”occur 
errors:4> Creating library D:/A_src/ACenter/build/Debug/testzuv.lib and object 
D:/A_src/ACenter/build/Debug/testzuv.exp
4>zookeeper.lib(zookeeper.obj) : error LNK2019: Unresolved external symbol 
_strtok_r, the symbol is referenced in function _resolve_hosts
4>zookeeper.lib(zk_log.obj) : error LNK2019: Unresolved external symbol 
_localtime_r, the symbol is referenced in function _time_now
4>D:\A_src\ACenter\build\Debug\testzuv.dll : fatal error LNK1120: 2 external 
commandsmoudle:  zookeeper-client/zookeeper-client-cversion:  release-3.5.5   
git commit id : 390fe37eplatform: Windows 10 compile: Visual studio 2017 
I fixed it in a simple way and it looks fine.

Re: Re: Re: Clean up the all the checkstyle violations in the zookeeper-server module

[Justin Ling Mao]

1.--->“we'd better first create an umbrella issue named "Enable checkstyle 
rules" or sth”I had created ZOOKEEPER-3431 previously, and we can create a 
series of sub-tasks under it.
2.I think we still have two things which should be discussed:  2.1 Currently, 
we only need to enforce the checkstyle violations check in the 
main-module:zookeeper-server, not included other modules?  IMO, because the 
zookeeper-contrib, zookeeper-recipes are now not well-maintained.  and some 
violations in the zookeeper-jute are auto-generated. so focusing on 
zookeeper-server is enough?
  2.2 What checkstyle template we will pick up? Now we have three options:  
A:[google_style](https://checkstyle.sourceforge.io/google_style.html)  
B:[bookkeeper_style] 
(https://github.com/apache/bookkeeper/blob/master/buildtools/src/main/resources/bookkeeper/checkstyle.xml)
  
C:[hbase_style](https://github.com/apache/hbase/blob/master/hbase-checkstyle/src/main/resources/hbase/checkstyle.xml)
  Which one will we choose?


- Original Message -
From: Enrico Olivelli 
To: dev@zookeeper.apache.org
Cc: maoling199210...@sina.com
Subject: Re: Re: Clean up the all the checkstyle violations in the 
zookeeper-server module
Date: 2019-07-07 15:13

Il dom 7 lug 2019, 01:29 Zili Chen  ha scritto:
> Justin & Enrico,
>
> Receiving no opposition on this proposal, we could regard it as
> a consensus. According to bookkeeper#230 we'd better first create
> an umbrella issue named "Enable checkstyle rules" or sth. Under
> there we can finally decide the checkstyle configuration and
> start sub-tasks enabling per package.
>
> For keeping current checkstyle, I'd like to pick up that it's
> possible that we remain the current simple config for all pkgs,
> adding a config said copied from bookkeeper named
> "strict-checkstyle.xml", enabling per pkg, which contains @author
> tags and rules in simple config. Once we enabling the strict one
> for all pkgs. We can merge two configs into one.
>
+1 please go ahead
Enrico
> Best,
> tison.
>
>
> Enrico Olivelli  于2019年7月6日周六 下午8:20写道：
>
> > Justin,
> > This is how we did it in Bookkeeper, we enabled checkstyle only for group
> > of packages in the main module (the biggest one, bookkeeper-server)
> >
> > https://github.com/apache/bookkeeper/issues/230
> >
> > I suggest using that checkstyle config, I feel we won't have so many
> > violations.
> >
> > We can keep current checkstyle invokation that checks for @author tags
> as a
> > separate 'execution' of the plugin with a specific checkstyle file (as
> you
> > already said)
> >
> > I am happy to help, thank you for driving this effort
> >
> > Enrico
> >
> >
> > Il sab 6 lug 2019, 11:33 Justin Ling Mao  ha
> > scritto:
> >
> > > - 1.It seems that we had reached a consensus to work on this.- 2.I also
> > > agree on the way: fix one package at a time, then another.- 3.Now Let
> us
> > > discuss some details:- 3.1 how to make the checkstyle only check
> the
> > > package we specify? I found this:  URL:
> > >
> >
> https://stackoverflow.com/questions/26455174/only-enable-some-checks-for-certain-inner-package
> > > @Olivelli Could you give me more your insight?- 3.2 What rules
> > will
> > > we init in the checkstyle.xml?   3.2.1 - I also think the rules
> from
> > > the hbase is too strict which will cause too many,many violations.
> > >  3.2.2 - apply the "Google's Java Style Checkstyle Coverage" is a good
> > > option? which seems to be more simplify and more suitable for us?
> > >  URL:https://checkstyle.sourceforge.io/google_style.html
> > >
> > >
> > >
> > > - Original Message -
> > > From: Andor Molnar 
> > > To: DevZooKeeper 
> > > Subject: Re: Clean up the all the checkstyle violations in the
> > > zookeeper-server module
> > > Date: 2019-07-02 13:22
> > >
> > > Yes. That way we only need to fix one package at a time.
> > > Andor
> > > On Mon, Jul 1, 2019 at 4:10 PM Zili Chen  wrote:
> > > > Hi Andor,
> > > >
> > > > To be exact, "iterations" means we define the original rules
> > > > in checkstyle configuration at once and turn them on one package
> > > > after another, so iterations. Is it correct?
> > > >
> > > > Best,
> > > > tison.
> > > >
> > > >
> > > > Andor Molnar  于2019年7月1日周一 下午9:09写道：
> > > >
> > > > > I like the idea of doing this in iterations.
> 

Re: Re: Clean up the all the checkstyle violations in the zookeeper-server module

[Justin Ling Mao]

First of all, sorry for the late reply (I thought I already
> answered
> > >>>> this,
> > >>>>> I remember reading it and drawing up an answer. Oh well)
> > >>>>>
> > >>>>> Some big patches are already reviewed, I think we should commit as
> > >> much
> > >>>> as
> > >>>>> possible before doing this refactor. (I'll also try to rev up my
> code
> > >>>>> review/commit thread)
> > >>>>>
> > >>>>> As for waiting for 3.6.0 - I don't see the reason we should. Unless
> > >> of
> > >>>>> course this would delay the release too much...
> > >>>>>
> > >>>>> I haven't checked HBase checkstyle against our code, I don't think
> we
> > >>>>> should introduce anything "fancy". What Enrico listed up sounds
> like
> > >> a
> > >>>> good
> > >>>>> starting point.
> > >>>>>
> > >>>>> +1 on introducing and forcing checkstyle.
> > >>>>>
> > >>>>> Regards,
> > >>>>> Norbert
> > >>>>>
> > >>>>>
> > >>>>> On Sun, Jun 23, 2019 at 7:27 PM Enrico Olivelli <
> eolive...@gmail.com
> > >>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Justin,
> > >>>>>> Thank you so much for your help in this.
> > >>>>>>
> > >>>>>> I would suggest to apply all of the rules in one pass, splitting
> > >> the
> > >>>> work
> > >>>>>> per package.
> > >>>>>> This way reviews will be easier, we will limit the number of
> > >> commits
> > >>>> and
> > >>>>> we
> > >>>>>> won't annoy too much the contributors , asking for hard rebases
> > >>>>>>
> > >>>>>> This is how we did it on Apache Bookkeeper
> > >>>>>> https://github.com/apache/bookkeeper/issues/230
> > >>>>>>
> > >>>>>> I will help review and commit all of your patches,  it will be
> > >>> mostly a
> > >>>>>> matter of code reformat without any behavior change.
> > >>>>>> Currently I am doing the same kind of work on others projects of
> my
> > >>>>>> company, so I perfectly know how the work will go.
> > >>>>>>
> > >>>>>> Before starting we must ensure that:
> > >>>>>> 1) community is willing to do this work (we will force a rebase on
> > >>>> mostly
> > >>>>>> every pending PR)
> > >>>>>> 2) the proposed configuration is accepted by the community
> > >>>>>> 3) it is the good time to do it, or should we wait for 3.6.0 to be
> > >>> out
> > >>>>>>
> > >>>>>>
> > >>>>>> I see you are referring to hbase checkstyle file,  did  you
> already
> > >>>>> checked
> > >>>>>> how much different it is from current project style?
> > >>>>>> Will we only need to remove trailing spaces, reorder members, fix
> > >>>>> imports,
> > >>>>>> cut long lines ?
> > >>>>>>
> > >>>>>>
> > >>>>>> Cheers
> > >>>>>> Enrico
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Il dom 23 giu 2019, 15:11 Justin Ling Mao <
> > >> maoling199210...@sina.com
> > >>>>
> > >>>> ha
> > >>>>>> scritto:
> > >>>>>>
> > >>>>>>> Background:zookeeper-server is the main-module of the zk
> > >> codebase.
> > >>>>>>> Unfortunately, there were many checkstyle violations in it. To
> > >>>> improve
> > >>>>>> the
> > >>>>>>> code quality and code standards,
> > >>>>>>> IMHO, it's time to clean up the all the checkstyle
> > >> violations(turn
> > >>> on
> > >>>>> the
> > >>>>>>> true). we can learn from the
> > >>> hbase
> > >>>>>> whose
> > >>>>>>> checkstyle(almost 40+ rules) is very strict and ensures a very
> > >>>> unified
> > >>>>>> code
> > >>>>>>> style.(
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> https://github.com/apache/hbase/blob/master/hbase-checkstyle/src/main/resources/hbase/checkstyle.xml
> > >>>>>>> )
> > >>>>>>> My planing is: clean up the all the checkstyle one rule by
> > >> another
> > >>> to
> > >>>>>>> avoid too much code changes for review.
> > >>>>>>> Everything's hard in the beginning, I have fired my first shot(
> > >>>>>>> https://github.com/apache/zookeeper/pull/992).
> > >>>>>>> If this draft has accepted by the community, I will create the
> > >>>>>>> corresponding sub-tasks for more people joining this work.
> > >>>>>>>
> > >>>>>>> Cited the comment from Enrico Olivelli in the
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> ZOOKEEPER-3431:--we
> > >>>>>>> have to discuss this topic with the community.I have experience
> > >> on
> > >>>>>>> BookKeeper, we had to clean up groups of packages.This is the
> > >> kind
> > >>> of
> > >>>>>> stuff
> > >>>>>>> that invalidates all of the pending pull requests.I have already
> > >>> sett
> > >>>>> up
> > >>>>>> a
> > >>>>>>> basic checkstyle configuration file and it is already active but
> > >> it
> > >>>> is
> > >>>>>>> performing only very basic checks (like no 'author' tags).I will
> > >>>>>> appreciate
> > >>>>>>> very much if you want to drive this effort, personally I would
> > >>> start
> > >>>>> this
> > >>>>>>> stuff after 3.6.0 release, once we consolidate current master and
> > >>> the
> > >>>>>> maven
> > >>>>>>> build. I would have sent an email on the dev@ list soon.We also
> > >>> have
> > >>>>> to
> > >>>>>>> agree on the checkstyle configuration, it is not trivial, I would
> > >>>> take
> > >>>>>> the
> > >>>>>>> file from HBase, BookKeeper or other ASF projects on the Haddoop
> > >>>>>> ecosystem.
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>

Re: Zookeeper code optimization

[Justin Ling Mao]

A good code optimizationCould you plz help us improve this issue?The 
contributor guideline is 
[here](https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute])

- Original Message -
From: 18779116352 <18779116...@163.com>
To: dev 
Subject: Zookeeper code optimization
Date: 2019-06-25 01:21

Dear developer:
I think the following code in line 623 of the QuorumCnxManager class can be 
optimized:
ArrayBlockingQueue bq = new ArrayBlockingQueue(
SEND_CAPACITY);
ArrayBlockingQueue oldq = queueSendMap.putIfAbsent(sid, bq);
if (oldq != null) {
addToSendQueue(oldq, b);
} else {
addToSendQueue(bq, b);
}
The optimization is as follows:
ArrayBlockingQueue bq = queueSendMap.computeIfAbsent(sid, 
serverId -> new ArrayBlockingQueue<>(SEND_CAPACITY));
addToSendQueue(bq, b);
Thanks

| |
18779116352
邮箱：18779116...@163.com
|
Signature is customized by Netease Mail Master

Re: Re: Great talk from Ben Reed about the origins of ZooKeeper

[Justin Ling Mao]

A great video,Thanks for the share.

- Original Message -
From: Brian Nixon 
To: dev@zookeeper.apache.org
Cc: UserZooKeeper 
Subject: Re: Great talk from Ben Reed about the origins of ZooKeeper
Date: 2019-06-30 02:11

It's great to hear the mindset that the ZooKeeper team was applying in the
early days. Some good advice there.
On Wed, Jun 26, 2019 at 8:42 PM Alexander Shraer  wrote:
> https://atscaleconference.com/videos/systems-scale-2019-welcome-keynote/
>

Re: Discuss on enabling checkstyle rules

[Justin Ling Mao]

I also got no reply from the email, but when checking the email 
archive(https://lists.apache.org/list.html?dev@zookeeper.apache.org)  
I found the reply from the community.It's very strange!!!
@Zili Chen Let us discuss in that thread?
After reading all the replies, I will give my new opinion and plan 
https://lists.apache.org/list.html?dev@zookeeper.apache.org
- Original Message -
From: Zili Chen 
To: DevZooKeeper 
Subject: Discuss on enabling checkstyle rules
Date: 2019-06-29 08:55

Hi zookeepers,
Recently I proceeded ZOOKEEPER-3446[1] and told that there is an ongoing
discussion on dev@zookeeper.apache.org about the approach to take in
applying checkstyle rules.
Subscribed dev-list today, I checked out mails archived these three months
but cannot find it. Could you please pick it up to dev-list again or show
me the link?
Best,
tison.
[1] https://github.com/apache/zookeeper/pull/1006

Clean up the all the checkstyle violations in the zookeeper-server module

[Justin Ling Mao]

Background:zookeeper-server is the main-module of the zk codebase. 
Unfortunately, there were many checkstyle violations in it. To improve the code 
quality and code standards,
IMHO, it's time to clean up the all the checkstyle violations(turn on the 
true). we can learn from the hbase whose 
checkstyle(almost 40+ rules) is very strict and ensures a very unified code 
style.(https://github.com/apache/hbase/blob/master/hbase-checkstyle/src/main/resources/hbase/checkstyle.xml)
My planing is: clean up the all the checkstyle one rule by another to avoid too 
much code changes for review.
Everything's hard in the beginning, I have fired my first 
shot(https://github.com/apache/zookeeper/pull/992).
If this draft has accepted by the community, I will create the corresponding 
sub-tasks for more people joining this work.

Cited the comment from Enrico Olivelli in the 
ZOOKEEPER-3431:--we 
have to discuss this topic with the community.I have experience on BookKeeper, 
we had to clean up groups of packages.This is the kind of stuff that 
invalidates all of the pending pull requests.I have already sett up a basic 
checkstyle configuration file and it is already active but it is performing 
only very basic checks (like no 'author' tags).I will appreciate very much if 
you want to drive this effort, personally I would start this stuff after 3.6.0 
release, once we consolidate current master and the maven build. I would have 
sent an email on the dev@ list soon.We also have to agree on the checkstyle 
configuration, it is not trivial, I would take the file from HBase, BookKeeper 
or other ASF projects on the Haddoop ecosystem.

Re: [Suggestion] Use Co-authored-by in commit messages

[Justin Ling Mao]

+1,A very good Suggestion.Thanks Norbert.I also suggest about the sign-off of 
the Reviewers' name.For the incentive, if someone participate in the review of 
PR, no matter whether he/she is a committer, we all need include his/her name?

- Original Message -
From: Norbert Kalmar 
To: dev@zookeeper.apache.org
Subject: [Suggestion] Use Co-authored-by in commit messages
Date: 2019-05-08 17:36

Hi Devs,
I've got this idea from HBase.
So: when there is a patch that is abandoned by its original author for any
reason, and it can no longer be merged, someone comes by, and asks to
continue to work on it. Usually the reply is to use the change freely or no
reply at all. Either way, what people end up doing is a new pull request,
and (correct me if I'm wrong) we do not have a standardized method how to
indicate, or even to indicate at all the original author.
My proposal is to use github's feature of Co-author, which is a way of
attributing multiple authors of a given commit. See more details here:
https://help.github.com/en/articles/creating-a-commit-with-multiple-authors
I wouldn't think this needs to be forced or anything on future PRs, but
it's a nice thing to have. And if someone sees an old patch, this could
give more incentive to continue to work on it, knowing there's a guideline
in the HowToContribute guide to credit him/her.
I can update the guide at
https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute if
the reception is positive.
Regards,
Norbert

Re: Re: Request for review & contribution of Apache Training

[Justin Ling Mao]

@Lars.Thanks for 
this:https://drive.google.com/file/d/1STmz-nTBjGeFuu8ycjrFDSL-KdahAQiG/view
- Original Message -
From: Flavio Junqueira 
To: dev@zookeeper.apache.org
Cc: Patrick Hunt 
Subject: Re: Request for review & contribution of Apache Training
Date: 2019-04-28 03:10


This is great, Lars. Thanks for putting this together. I have some additional 
comments:
- I'd rather not characterize zookeeper as a key-value store, one reason being 
that to access a znode we give a "path" and not a "key". I don't want to open 
this up to a discussion on what a key-value store actually is, but I typically 
say that it is a hierarchy of znodes and explain what a znode is (Slide 2). If 
it is early to talk about znodes, then say hierarchy of simple data files.
- It would be nice to have a slide illustrating how to get going with 
zookeeper, like starting a server with `bin/zkServer.sh start` and using the 
`bin/zkCli.sh` to issue some commands.
- In Slide 8, the statement saying that "in an ensemble we always have a single 
leader" isn't accurate. I think you want to say something along the lines of 
"in an ensemble there is at most on leader server supported by a majority of 
followers".
Thanks again for the initiative.
-Flavio
> On 27 Apr 2019, at 00:55, Lars Francke  wrote:
> 
> Thank you Patrick, that's great!
> 
> I've incorporated the feedback (and a bit more) and have uploaded a new
> version.
> 
> This is just a first start and I hope that over time the community will
> expand on it (e.g. slides on use cases/patterns/recipes, API usage, the Zab
> protocol, etc.). This is also an invitation to the ZooKeeper community to
> add anything at any point if you feel it's worthwhile to teach to potential
> learners. If you don't want to fiddle around with slides that's fine as
> well, just filing an issue in Jira in the TRAINING project will already
> help.
> 
> Cheers,
> Lars
> 
> On Fri, Apr 26, 2019 at 10:49 PM Patrick Hunt  wrote:
> 
>> Wow, this is great, thanks Lars. I reviewed, here are my comments:
>> 
>> 3: you might highlight that it's used outside hadoop (e.g. solr and many
>> others inside and outside apache). Also heavily using in the tech industry;
>> facebook, twitter, linkedin, etc...
>> 
>> You might also reference Curator as it's a good way to get started from a
>> client perspective.
>> 
>> Regards,
>> 
>> Patrick
>> 
>> On Fri, Apr 26, 2019 at 7:20 AM Lars Francke 
>> wrote:
>> 
>>> Hi ZooKeeper devs,
>>> 
>>> we started the Apache Training (incubating) project back in February.
>>> Our aim is to develop training material (slides, labs, tests etc.) for all
>>> kinds of projects (mostly focusing on Apache projects but not
>>> exclusively).
>>> 
>>> We're still getting off the ground with homepage and so. We're also
>>> working
>>> on the very first content donation which is a super simple four slide
>>> ZooKeeper introduction.
>>> 
>>> I chose this because it's small and we can test our processes.
>>> 
>>> I still wanted to reach out and see if anyone here is interested in
>>> contributing anything and/or reviewing the existing material. It can be
>>> found in Jira: 
>>> 
>>> Thank you very much!
>>> 
>>> Cheers,
>>> Lars
>>> 
>> 

Re: Prometheus.io endpoint vs admin server

[Justin Ling Mao]

@Enrico1.  --->"bind '/metrics' to the current "Admin server"" What do you mean 
is the JettyAdminServer？ which is enable and binding to default port:8080?
2.  Is the following a good 
option?e.g:http://:8080/commands/metricsor:http://:8080/metrics

- Original Message -
From: Enrico Olivelli 
To: DevZooKeeper 
Subject: Prometheus.io endpoint vs admin server
Date: 2019-04-24 07:27


Hi,
I am working on Prometheus endpoint.
Prometheus client needs to expose an HTTP service, usually it is
mapped at /metrics, the Prometheus serve polls that endpoint
periodically in order to store metrics inside its local database.
We have two ways:
1) bind '/metrics' to the current "Admin server"
2) let the MetricsProvider open another http endpoint
I see value in both of the two approaches.
Approach 1): No new ports, so more "secure" ? more user-friendly ?
Approach 2): If I am not an user of the HTTP Admin Server I would need
to fully enable it if I want to integrate with Prometheus, so a
separate endpoint would be better for me.
In Apache BookKeeper we have all of the two options available (bind to
standard http admin endpoint or start a separate http endpoint)
I would like to see the option of the community.
I am open to any of the two and even to having both of them.
Regards
Enrico

Re: Raise the Quality of ZooKeeper Docs with Google Season of Docs Project

[Justin Ling Mao]

Thanks Fangmin for bringing up this interesting topic.maybe next year, maybe no 
time,maybe nobody· :D

- Original Message -
From: Fangmin Lv 
To: dev@zookeeper.apache.org
Subject: Raise the Quality of ZooKeeper Docs with Google Season of Docs Project
Date: 2019-04-24 06:23

Hi guy,
Raise here to see if we have interest to participant in the Google Season
of Docs or not.
(The deadline is today, so we probably won't attend this season, but raise
earlier to collect the interest here and prepare for next year)
[Background]
Google started the project of season of docs (
https://developers.google.com/season-of-docs/) this march, the goal is to
bring open source and technical writer communities together, and raise the
awareness of open source and quality of docs from technical writer.
[Benefit to ZooKeeper Community]
High quality doc will make an open source project more success and easier
to involve. ZooKeeper has nice docs, but there are still rooms to improve,
like make it more readable, add more technique details, and make it easier
to maintain. This is a good opportunity for us to improve the docs with the
help from technical writer.
[What needed to involve]
To involve in this project, at least two people from the ZooKeeper
community need to be registered with Season of Docs team, and being the
mentors to discuss the project ideas with technical writer.
This is non-trivial effort and we need to be planned ahead, given the
deadline for this season is today, I don't think we're able to ride this
first season. But we should keep an eye on this, if it turns out to be a
success thing for other open source projects, then we could prepare and
attend next year.
Thanks,
Fangmin

Re: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 3

[Justin Ling Mao]

+1, no bindingLook at my previous jepsen, benchmark and zkCli test, 3.5.5 works 
well.
- Original Message -
From: Andor Molnar 
To: DevZooKeeper 
Subject: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 3
Date: 2019-04-12 19:58

Hi team,
Just to wrap up the votes so far:
We got +1 from
- Enrico
- Norbert
Do we have +1 from Maoling by any chance?
ZooKeeper PMCs,
We need you one more time here to release stable 3.5!
Please vote!
Thanks,
Andor
> On 2019. Apr 9., at 21:13, Andor Molnár  > wrote:
> 
> Thanks Enrico for testing.
> 
> I think both checks are valid and should be added to the release verification 
> procedure. None of the issues are showstopper for the RC, but let's fix them 
> to get a clean sheet.
> 
> - Spotbugs issue can be ignored - null-check is valid when the version has 
> qualifier: e.g. -SNAPSHOT, which is not the case for releases.
> 
> - Rat check has found 27 files with missing license headers for me 
> (attached). Let's add them to the exclusion list.
> 
> 
> 
> Regards,
> 
> Andor
> 
> 
> 
> 
> 
> On 4/9/19 16:49, Enrico Olivelli wrote:
>> I am testing this RC, before casting my vote I would like to share
>> these problems, I am not sure they are blockers:
>> 
>> 1) spotbugs is not passing (tested both with jdk11 and jdk8)
>> 
>> mvn clean install -DskipTests spotbugs:check
>> 
>> [INFO] --- spotbugs-maven-plugin:3.1.9:check (default-cli) @ zookeeper ---
>> [INFO] BugInstance size is 1
>> [INFO] Error size is 0
>> [INFO] Total bugs: 1
>> [ERROR] Redundant nullcheck of
>> org.apache.zookeeper.version.Info.QUALIFIER which is known to be null
>> in org.apache.zookeeper.Version.getVersion()
>> [org.apache.zookeeper.Version] Redundant null check at
>> Version.java:[line 44] RCN_REDUNDANT_NULLCHECK_OF_NULL_VALUE
>> [INFO]
>> 
>> 2) Apache RAT check is not passing
>> mvn clean apache-rat:check
>> 
>> Those checks need to be added to the release procedure and in CI
>> checks (I have filed an issue yesterday to improve CI checks
>> https://issues.apache.org/jira/browse/ZOOKEEPER-3351 
>> )
>> 
>> 
>> Enrico
>> 
>> Il giorno mar 9 apr 2019 alle ore 11:01 Andor Molnar
>>   ha scritto:
>>> This is the first stable release of 3.5 branch: 3.5.5. It resolves 117 
>>> issues, including Maven migration, Quorum TLS, TTL nodes and lots of other 
>>> performance and stability improvements.
>>> 
>>> The full release notes is available at:
>>> 
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12343268
>>>  
>>> 
>>> 
>>> *** Please download, test and vote by April 16th 2019, 23:59 UTC+0. ***
>>> 
>>> Source files:
>>> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.5.5-rc3/ 
>>> 
>>> 
>>> Maven staging repos:
>>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/parent/3.5.5/
>>>  
>>> 
>>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper-jute/3.5.5/
>>>  
>>> 
>>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.5/
>>>  
>>> 
>>> 
>>> The release candidate tag in git to be voted upon: release-3.5.5-rc3
>>> 
>>> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
>>> http://www.apache.org/dist/zookeeper/KEYS 
>>> 
>>> 
>>> Should we release this candidate?
>>> 
>>> 
> 

Re: ZK Smoke Screen Test

[Justin Ling Mao]

Good job,Thanks for 
this:https://github.com/BELUGABEHR/zk-smoketest#running-under-dockerYou can 
give a PR to phunt's repo,let more people benefit from it.Look at 
ZOOKEEPER-3264 which is aimed to implement a better benchmark tool for 
zookeeper internal.Stay tuned! :D
- Original Message -
From: David Mollitor 
To: dev@zookeeper.apache.org
Subject: ZK Smoke Screen Test
Date: 2019-04-13 06:25


Hello,
I am not sure if this is novel or perhaps old news, but I forked Patrick
Hunt's ZK smoketest project and updated the docker components.
I am able to pretty easily start a quorum (3x) using the latest ZK image
and run the smoke test against it.
https://github.com/BELUGABEHR/zk-smoketest
I've put a little bit about it at the bottom of the README.
Thanks.
 novel  ['nɒv(ə)l]  详细X  基本翻译n. 小说adj. 新奇的；异常的n. 
(Novel)人名；(法、西、英)诺韦尔  网络释义romance novel: 爱情小说
novel design: 款式新颖
saga novel: 长篇小说   

Re: Re: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 2

[Justin Ling Mao]

---> "That's cool, thanks Justin. I'd be interested to know - did you use the 
legacy/old c client binaries from the github repo or did you rebuild them? What 
about zkpython itself? I could see benefits or either/both."
@PatrickTo use that(your) benchmark tool, I did something like following:cd 
zookeeper-3.5.5/src/c./configuremake make install# bulid zkpython from source 
code:wget 
https://files.pythonhosted.org/packages/14/38/a761465ab0a154405c11f7e5d6e81edf6e84584e114e152fddd340f7d6d3/zkpython-0.4.2.tar.gzpython
 setup.py installexport LD_LIBRARY_PATH=/usr/local/lib/then I cd the 
zk-smoketest dir,exec the following:./zk-latencies.py --servers 
"localhost:2181" --root_znode="/zk-latencies_666" --znode_count=1 
--znode_size=100 --synchronous
- Original Message -
From: Patrick Hunt 
To: DevZooKeeper , maoling199210...@sina.com
Subject: Re: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 2
Date: 2019-04-08 12:24



On Sat, Apr 6, 2019 at 11:47 PM Justin Ling Mao  
wrote:
Test for the 3.5.5 candidate 2:
-[0]: Jepsen consistency check had passed. +1.   Look at the attachment: 
jepsen_test_report.txt.   how to use it: 
https://github.com/apache/zookeeper/pull/883
-[1]: zk smoke and latencies test passed, comparing to 3.5.4-beta, no any 
notable perfermance improvement .+1.  The tool:  
https://github.com/phunt/zk-smoketest   Env:   8 cores + 16GB Memory + 
256GB SSD(my local Mac Pro 2015)   java version:"1.8.0_191"   Look at 
the benchmark report: 
https://docs.google.com/spreadsheets/d/1QnADbG9bCGMyTorEi67jQXWQSUVXQ1yQADgLV4LJHcg/edit#gid=0
That's cool, thanks Justin. I'd be interested to know - did you use the 
legacy/old c client binaries from the github repo or did you rebuild them? What 
about zkpython itself? I could see benefits or either/both.
Regards,
Patrick -[2]: zkCli test passed.(base operation:create,set,get,delete ...etc)   
 +1.

- Original Message -
From: Enrico Olivelli 
To: DevZooKeeper 
Subject: Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 2
Date: 2019-04-06 17:49

Tamas,
Il sab 6 apr 2019, 10:47 Tamas Penzes  ha
scritto:
> Hi Enrico,
>
> Just my 2cents.
>
> Having the sources jars in the binary package can be useful if you want to
> debug without downloading the source tarball or checking out the whole
> project from github. In the same time it is not needed on the classpath.
>
> The docs html is generated from md files now, but the design didn't get
> updated. I wanted to do it (to have the same design as the website), but
> never had time for that. It doesn't look a blocker for me, can be done in
> any release later.
>
I agree, not all my points are blockers.
The most critical point is that the source tarball does not reflect the tag
and, important files are missing.
Cheers
Enrico
> Regards, Tamaas
>
> On Fri, Apr 5, 2019, 17:37 Enrico Olivelli  wrote:
>
> > -1 (non binding)
> >
> > Good news:
> > I have build the sources with jdk-8 and all tests passed
> > checksums and signatures are ok
> > I run a few smoke tests running on JDK11, all is ok
> >
> > Bad news:
> > I see the following problems in the "binary package", not all blockers:
> > - we have a lib/cobertura directory, not useful
> > - we have zookeeper-3.5.5-sources.jar not needed in the "binary"
> > package on the classpath
> > - we also have all of the 'recipes' not needed in the "binary" package
> > on the classpath
> > - inside the "docs" directory I see a webside with a very old look and
> feel
> >
> > The source package does not reflect the repository, for instance
> > build.xml file, owaspSuppressions.xml,
> > excludeFindBugsFilter.xml.are missing.
> > I think we have to release in a "source package" a clean copy of ASF
> > repository at the released "git tag"
> >
> >
> > Enrico
> >
> > Il giorno ven 5 apr 2019 alle ore 14:35 Andor Molnar
> >  ha scritto:
> > >
> > > This is the first stable release of 3.5 branch: 3.5.5. It resolves 117
> > issues, including Maven migration, Quorum TLS, TTL nodes and lots of
> other
> > performance and stability improvements.
> > >
> > > The full release notes is available at:
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12343268
> > >
> > > *** Please download, test and vote by April 12th 2019, 23:59 UTC+0. ***
> > >
> > > Source files:
> > > https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.5.5-rc2/
> > >
> > > Maven staging repos:
> > >
> >
> https://repository.apache.org/content/groups/staging/

Re: [VOTE] Apache ZooKeeper release 3.5.5 candidate 1

[Justin Ling Mao]

A good news!!!  A long time to have a stable version.
I volunteer to do the following test this weekend.
1.a jepsen test to check the consistency.
2.a namazu fuzzy test to check the stablibity.(which will doc this test 
tool in ZOOKEEPER-3343.).
3.a test for the zkCli.sh
- Original Message -
From: Andor Molnar 
To: DevZooKeeper 
Subject: [VOTE] Apache ZooKeeper release 3.5.5 candidate 1
Date: 2019-04-03 20:49


This is the first stable release of 3.5 branch: 3.5.5. It resolves 116 issues, 
including Maven migration, Quorum TLS, TTL nodes and lots of other performance 
and stability improvements.
The full release notes is available at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801=12343268
*** Please download, test and vote by April 10th 2019, 23:59 UTC+0. ***
Source files:
https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.5.5-rc1/
Maven staging repos:
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/parent/3.5.5/
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper-jute/3.5.5/
https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.5/
The release candidate tag in git to be voted upon: release-3.5.5-rc1
ZooKeeper's KEYS file containing PGP keys we use to sign the release:
http://www.apache.org/dist/zookeeper/KEYS
Should we release this candidate?

Re: [ANNOUNCE] Apache ZooKeeper 3.4.14

[Justin Ling Mao]

Awesome!!!
- Original Message -
From: Andor Molnar 
To: DevZooKeeper , u...@zookeeper.apache.org
Subject: [ANNOUNCE] Apache ZooKeeper 3.4.14
Date: 2019-04-02 18:32


The Apache ZooKeeper team is proud to announce Apache ZooKeeper version 3.4.14
ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming,
configuration management, synchronization, and group services - in a
simple interface so you don't have to write them from scratch. You can
use it off-the-shelf to implement consensus, group management, leader
election, and presence protocols. And you can build on it for your
own, specific needs.
For ZooKeeper release details and downloads, visit:
http://zookeeper.apache.org/releases.html
ZooKeeper 3.4.14 Release Notes are at:
http://zookeeper.apache.org/doc/r3.4.14/releasenotes.html
We would like to thank the contributors that made the release possible.
Regards,
The ZooKeeper Team

Re: Re: Re: Jute buffer size increasing.

[Justin Ling Mao]

- I also saw that you posted your question to the hbase community, Are you 
happy with the answer from Wellington Chevreuil? are you creating too many zk 
child nodes? - "excessive regions server" is the root cause? what you should do 
is calming this symptoms of hbase,other than increasing the jute.maxbuffer to a 
large value(128MB)?--->"Upon increase the jute buffer we are saying significant 
stability" Your hbase may be stable temporary, but zk will be panic.the logic 
is here:when write/get too large data to/from zk,it will give too much pressure 
on the server, sometimes cause the GC stop in the leader, follwers cannot ping 
the leader within the bounded time, then start to re-election.  I guess,haha:D
- Original Message -
From: Asim Zafir 
To: maoling199210...@sina.com
Cc: dev 
Subject: Re: Re: Jute buffer size increasing.
Date: 2019-03-11 21:14

Thanks for the response. Actually it was otherwise - when he had the buffer
size set to 32 mb, we were getting Len ERRORS reported by ZK after which ZK
was tearing down the connection. We saw excessive regions server exist and
at times complete HBase crash. It was very painful. Upon increase the jute
buffer we are saying significant stability. Can you explain he logic behind
jute buffer and instability that you were seeing, I was not able to find a
good resources to relate that level of detail so it will be extremely
helpful if you can explain.
On Sun, Mar 10, 2019 at 7:26 PM Justin Ling Mao 
wrote:
> - What's your user case using zk for hbase? Why increase jute.maxbuffer to
> 128MB? incredible! really need this change?
> - if the jute.maxbuffer is too large, it will have a side effect on the
> stability and throughout of the cluster,the response time between server
> and client.
> - for the "jmap -histo:live" you had provided, you have created almost
> 103842 zk-nodes,and their memory usage is less than 10 MB, so don't worry
> about this.
>
> - Original Message -
> From: Asim Zafir 
> To: Jordan Zimmerman 
> Cc: Zookeeper 
> Subject: Re: Jute buffer size increasing.
> Date: 2019-03-09 10:11
>
> thanks and very much appreciate your feedback.
> here the system that is using zookeeper is HBASE and you are right its a
> manual setting. we had it 1mb, then increase to 20mb, 32mb and now 128 mb..
> what is the impact of having higher jute buffer size and more (higher count
> of org.apache.zookeeper.data.StatPersisted object).
> On Fri, Mar 8, 2019 at 5:24 PM Jordan Zimmerman <
> jor...@jordanzimmerman.com>
> wrote:
> > The Jute buffer size is a manual setting, so it wouldn't increase by
> > itself. If you find that you have to keep increasing it's due to a few
> > possibilities: ZNodes with lots of children (with long names). Any 1
> > ZooKeeper API call is limited by jute max buffer. So, if a call to
> > getChildren() where there's a lot of children or lots of children with
> long
> > names (or both) can bust jute max buffer. Another possibility is ZNodes
> > with large payloads.
> >
> > -JZ
> >
> > On Mar 8, 2019, at 7:21 PM, Asim Zafir  wrote:
> >
> >
> > + ZK dev community. Please assist.
> > On Fri, Mar 8, 2019 at 4:10 PM Asim Zafir  wrote:
> >
> >> Hi Jordon,
> >>
> >> We are seeing constance increase in jute buffer size on our zookeeper
> >> instance. right now it is set to 128. We are primarily using zookeeper
> for
> >> HBase cluster. I want to see what is contributing to the increase of
> jute
> >> buffer size but so for after investigating the code and studying the
> >> protocol itself it appear it is a function of number of watches that
> gets
> >> set on the znodes. to see how many zookeeper watch objects are on
> zookeeper
> >> jvm /instance I did a jmap history:live on zookeeper pid and I got the
> >> following output (please see below). I am not sure what is [C, [B here
> and
> >> it doesn't appear its refers to any class - I don't see this on dev
> >> instance of zookeeper. due to suspect memory leak or another issue?
> Please
> >> guide me through this as I can't find a resource who can go that far to
> >> give me any hint as to what may be happening on my end. Also is it safe
> for
> >> ZK sizes to increase that much? I will greatly appreciate your feedback
> and
> >> help on this.
> >>
> >> num #instances #bytes class name
> >> --
> >> 1: 220810 140582448 [C
> >> 2: 109370 34857168 [B
> >> 3: 103842 7476624
> >> org.apache.zookeeper.data.StatPersisted
> >> 4: 220703 5296872 java.lang.String
> >> 5: 28682 3783712 
> >> 6: 2

Re: Re: datalength 0

[Justin Ling Mao]

the dataLength of the node "/hbase/tables" only shows the dataLength of the 
current node, not includes the child nodes.
- Original Message -
From: Asim Zafir 
To: maoling199210...@sina.com
Subject: Re: datalength 0
Date: 2019-03-11 21:18

Justin, Dont quite understand it .. my child znodes under /hbase/tables are  
all have data. they are multiple tables under neath..  despite that datalength 
0. Don't quite understand your explanation. 
Can you  please elaborate?Thanks,Asim



On Sun, Mar 10, 2019 at 7:58 PM Justin Ling Mao  
wrote:
--->"I see data length of 0 on my znodes?"  becasue the data of this node 
"/hbase/table" is empty check this: [zk: localhost:2181(CONNECTED) 2] get 
/hbase/table--->"timestamps are also not updating" set the data,the mtime will 
be updated. e.g [zk: localhost:2181(CONNECTED) 5] set /hbase/table 
mydata--->"how can I find the names of children under a znode" e.g [zk: 
localhost:2181(CONNECTED) 8] ls /hbase/table[emp, hbase:meta, hbase:namespace, 
t1, t8, test]BTW, googling a question like this, will give you enough you want 
:D
- Original Message -
From: Asim Zafir 
To: Zookeeper 
Subject: datalength 0
Date: 2019-03-09 10:34

HI All,
I see data length of 0 on my znodes? timestamps are also not updating. is
this a problem? - what does it mean? how can I find the names of children
under a znode - trying to understanding what is impacting the jute.buffer
to reach its max value. Will greatly appreciate if you can guide me on
this.
for example
[zk: localhost:2181(CONNECTED) 16] stat  /hbase/table
cZxid = 0x10050
ctime = Tue Jun 19 22:23:13 UTC 2018
mZxid = 0x10050
mtime = Tue Jun 19 22:23:13 UTC 2018
pZxid = 0x1003b396cc
cversion = 2231
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 41
[zk: localhost:21

Re: Re: Apache ZooKeeper meetup in Palo Alto March 14th

[Justin Ling Mao]

If your guys have speechs on some interesting topics, Plz video it, then post 
to the youtube,share with us. :D
- Original Message -From: Jie Huang 
To: dev@zookeeper.apache.org
Subject: Re: Apache ZooKeeper meetup in Palo Alto March 14th
Date: 2019-03-12 04:48

Hi, Patric
Are we still on for the meetup? I'm interested and planning to be there.
Thanks,
Jie
On Fri, Mar 1, 2019 at 1:46 PM Patrick Hunt  wrote:
> Hi folks. Andor is visiting the USA and he asked if I could facilitate a
> meetup the night of March 14th - 530-8pm. This would be in Palo Alto at
> Cloudera's HQ - 395 Page Mill Rd, Palo Alto. I will reserve one of our
> meeting rooms and order pizza and beer. I plan to keep it low key, a bit of
> an "unconference" if you will. A chance for people to catch up face to
> face, discuss interesting topics and generally hang out. If you do have
> something you'd like to present please let me know.
>
> Regards,
>
> Patrick
>

Re: datalength 0

[Justin Ling Mao]

--->"I see data length of 0 on my znodes?"  becasue the data of this node 
"/hbase/table" is empty check this: [zk: localhost:2181(CONNECTED) 2] get 
/hbase/table--->"timestamps are also not updating" set the data,the mtime will 
be updated. e.g [zk: localhost:2181(CONNECTED) 5] set /hbase/table 
mydata--->"how can I find the names of children under a znode" e.g [zk: 
localhost:2181(CONNECTED) 8] ls /hbase/table[emp, hbase:meta, hbase:namespace, 
t1, t8, test]BTW, googling a question like this, will give you enough you want 
:D
- Original Message -
From: Asim Zafir 
To: Zookeeper 
Subject: datalength 0
Date: 2019-03-09 10:34

HI All,
I see data length of 0 on my znodes? timestamps are also not updating. is
this a problem? - what does it mean? how can I find the names of children
under a znode - trying to understanding what is impacting the jute.buffer
to reach its max value. Will greatly appreciate if you can guide me on
this.
for example
[zk: localhost:2181(CONNECTED) 16] stat  /hbase/table
cZxid = 0x10050
ctime = Tue Jun 19 22:23:13 UTC 2018
mZxid = 0x10050
mtime = Tue Jun 19 22:23:13 UTC 2018
pZxid = 0x1003b396cc
cversion = 2231
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 41
[zk: localhost:21

Re: Re: Jute buffer size increasing.

[Justin Ling Mao]

- What's your user case using zk for hbase? Why increase jute.maxbuffer to 
128MB? incredible! really need this change?- if the jute.maxbuffer is too 
large, it will have a side effect on the stability and throughout of the 
cluster,the response time between server and client.- for the "jmap 
-histo:live" you had provided, you have created almost 103842 zk-nodes,and 
their memory usage is less than 10 MB, so don't worry about this.
- Original Message -
From: Asim Zafir 
To: Jordan Zimmerman 
Cc: Zookeeper 
Subject: Re: Jute buffer size increasing.
Date: 2019-03-09 10:11

thanks and very much appreciate your feedback.
here the system that is using zookeeper is HBASE and you are right its a
manual setting. we had it 1mb, then increase to 20mb, 32mb and now 128 mb..
what is the impact of having higher jute buffer size and more (higher count
of org.apache.zookeeper.data.StatPersisted object).
On Fri, Mar 8, 2019 at 5:24 PM Jordan Zimmerman 
wrote:
> The Jute buffer size is a manual setting, so it wouldn't increase by
> itself. If you find that you have to keep increasing it's due to a few
> possibilities: ZNodes with lots of children (with long names). Any 1
> ZooKeeper API call is limited by jute max buffer. So, if a call to
> getChildren() where there's a lot of children or lots of children with long
> names (or both) can bust jute max buffer. Another possibility is ZNodes
> with large payloads.
>
> -JZ
>
> On Mar 8, 2019, at 7:21 PM, Asim Zafir  wrote:
>
>
> + ZK dev community. Please assist.
> On Fri, Mar 8, 2019 at 4:10 PM Asim Zafir  wrote:
>
>> Hi Jordon,
>>
>> We are seeing constance increase in jute buffer size on our zookeeper
>> instance. right now it is set to 128. We are primarily using zookeeper for
>> HBase cluster. I want to see what is contributing to the increase of jute
>> buffer size but so for after investigating the code and studying the
>> protocol itself it appear it is a function of number of watches that gets
>> set on the znodes. to see how many zookeeper watch objects are on zookeeper
>> jvm /instance I did a jmap history:live on zookeeper pid and I got the
>> following output (please see below). I am not sure what is [C, [B here and
>> it doesn't appear its refers to any class - I don't see this on dev
>> instance of zookeeper. due to suspect memory leak or another issue? Please
>> guide me through this as I can't find a resource who can go that far to
>> give me any hint as to what may be happening on my end. Also is it safe for
>> ZK sizes to increase that much? I will greatly appreciate your feedback and
>> help on this.
>>
>>  num #instances #bytes  class name
>> --
>>1:220810  140582448  [C
>>2:109370   34857168  [B
>>3:1038427476624
>> org.apache.zookeeper.data.StatPersisted
>>4:2207035296872  java.lang.String
>>5: 286823783712  
>>6: 286823681168  
>>7:1110003552000  java.util.HashMap$Entry
>>8:1075693442208
>> java.util.concurrent.ConcurrentHashMap$HashEntry
>>9:1038423322944  org.apache.zookeeper.server.DataNode
>>   10:  26553179640  
>>   11:  23132017056  
>>   12:  26551842456  
>>   13:   3181241568
>> [Ljava.util.concurrent.ConcurrentHashMap$HashEntry;
>>   14:  75261221504  [Ljava.util.HashMap$Entry;
>>   15:  1820 812976  
>>   16:  8228 394944  java.util.HashMap
>>   17:  2903 348432  java.lang.Class
>>   18:  4077 229688  [S
>>   19:  4138 221848  [[I
>>   20:   231 125664  
>>   21:  7796 124736  java.util.HashSet
>>   22:  6771 108336  java.util.HashMap$KeySet
>>   23:  1263  62968  [Ljava.lang.Object;
>>   24:   746  59680  java.lang.reflect.Method
>>   25:  3570  57120  java.lang.Object
>>   26:   502  36144  org.apache.zookeeper.server.Request
>>   27:   649  25960  java.lang.ref.SoftReference
>>   28:   501  24048  org.apache.zookeeper.txn.TxnHeader
>>   29:   188  21704  [I
>>   30:   861  20664  java.lang.Long
>>   31:   276  19872  java.lang.reflect.Constructor
>>   32:   559  17888
>> java.util.concurrent.locks.ReentrantLock$NonfairSync
>>   33:   422  16880  java.util.LinkedHashMap$Entry
>>   34:   502  16064
>> org.apache.zookeeper.server.quorum.QuorumPacket
>>   35:   455  14560  java.util.Hashtable$Entry
>>   36:   495  14368  [Ljava.lang.String;
>>   37:   318  12720
>> java.util.concurrent.ConcurrentHashMap$Segment
>>   38: 3  12336  [Ljava.nio.ByteBuffer;

Re: Re: Question on ZK commit/patch policy.

[Justin Ling Mao]

agree with this from Brian Nixon.--->"For trivial changes like spelling, 
whitespace, pruning of import, does itmake sense to have one super/umbrella 
ticket with multiple PRs attached"
- Original Message -From: Brian Nixon 
To: dev@zookeeper.apache.org
Subject: Re: Question on ZK commit/patch policy.
Date: 2019-03-05 05:54

I like having JIRAs for all changes because it allows one to track all the
changes to given components through the JIRA web interface and it forces
the contributor to spend some time upfront making sure their change is a
single coherent unit.
For trivial changes like spelling, whitespace, pruning of import, does it
make sense to have one super/umbrella ticket with multiple PRs attached?
-Brian
On Wed, Feb 27, 2019 at 1:04 PM Enrico Olivelli  wrote:
> I think that having a JIRA makes it simpler to create release notes and
> track bugfixes/new features.
> Trivial changes, like typos are not worth a JIRA.
>
> My 2 cents
> Enrico
>
> Il mer 27 feb 2019, 17:57 Patrick Hunt  ha scritto:
>
> > Yea, the commit I just did was a single missing space so no big deal.
> > Jordan's link is to curator current policy which seems very similar to
> > ours.
> >
> > I know what current state is. My question though is what do people think?
> > Stay with the current mechanism or move to something else? Staying put is
> > fine, I just wanted to review given it's been a while (10+ years!) since
> we
> > last considered this and with github/gitbox and time baselines have
> changed
> > considerably over that time.
> >
> > Patrick
> >
> >
> > On Wed, Feb 27, 2019 at 8:44 AM Andor Molnar  >
> > wrote:
> >
> > > There were a few typo/language/cosmetic related patches which were so
> > small
> > > that we've decided it's probably not worth the effort to create a Jira
> > for
> > > every one of them.
> > > Similarly, I haven't created Jiras for issues that were found in
> release
> > > candidates.
> > >
> > > Other than this we generally still don't accept patches without Jira
> > ticket
> > > and properly formatted title / commit message.
> > >
> > > Andor
> > >
> > >
> > >
> > > On Wed, Feb 27, 2019 at 5:38 PM Patrick Hunt  wrote:
> > >
> > > > Historically we've only committed changes that have an associated
> JIRA.
> > > Now
> > > > with the move to gitbox we are seeing increased submissions (PRs)
> that
> > > > don't include a JIRA - I just committed one and then realized that it
> > > > didn't include a JIRA (sorry about that!). Given github and the
> recent
> > > move
> > > > to gitbox significantly streamlines the contribution process I'm
> > > wondering
> > > > if we should reconsider our process. Any thoughts? Anyone work on
> > another
> > > > Apache project that does things differently and has pro/con to share?
> > > >
> > > > Regards,
> > > >
> > > > Patrick
> > > >
> > >
> >
>

回复：Travis is Up an running

[Justin Ling Mao]

Awesome work,Thanks Enrico!
- 原始邮件 -
发件人：Enrico Olivelli 
收件人：DevZooKeeper 
主题：Travis is Up an running
日期：2019年02月22日 23点42分


Hi folks,
Andor merged the patch about Travis configuration, let me recap the
status quo about Pull Requests.
1) We are using Travis to test the build on JDK8 and JDK11 + spotbugs
(which has differents results on different JDK versions), using Maven
- in order to make Travis restart you have to login to Travis and
click the 'Restart button'
2) We are using ASF Jenkins in order to run tests on JDK8, we have two jobs:
- One job for Ant and JDK8  -> use 'retest ant build' to restart
- One job for Maven and JDK8  -> use 'retest maven build' to restart
I will update the wiki page
In the short term I would like to put up a separate job for the
C-client and eventually for integration tests/backward compatiblity
(maybe using docker)
Cheers
Enrico

回复：回复：Re: A big refactor for the documetations

[Justin Ling Mao]

Thanks for your ACK @Norbert!
--->"I think the formatting is broken, it was a bit hard to read through." 
  Wuuu,it's ok in my side.You can also read this Google Doc:
  
https://docs.google.com/document/d/1N_WS37E83LtoYFm9AnuAnnE3axpe-ypogJ65LCqqZMQ/edit?usp=sharing
--->"I'm not sure we want to change the formatting of the docs though"
  the new catalog use the origin doc skeleton,and almost only add some new 
navigation bar.
--->"It's been similar for a long time, and I guess people got used to it :)"
  the fact is that people complain about that docs for a long time.:D.
Our doc is inferior to other projects (e.g. redis,hbase,etcd).it is the 
time to bring forth the new through the old.
- 原始邮件 -
发件人：Norbert Kalmar 
收件人：dev@zookeeper.apache.org, maoling199210...@sina.com
主题：Re: A big refactor for the documetations
日期：2019年02月13日 22点26分

Hi,
I think the formatting is broken, it was a bit hard to read through.I think 
it's a good idea, especially adding new stuff like 3rd party tools Curator and 
examples. I'm not sure we want to change the formatting of the docs though. 
It's been similar for a long time, and I guess people got used to it :) But I'm 
not against it either.
So definitely +1 on adding new stuff and example, and I'm neutral on the 
changing the doc format part. It has its pros and cons. 
Regards,Norbert
On Wed, Feb 13, 2019 at 2:48 PM Justin Ling Mao  
wrote:
ping @anmolnar and others, any insights or suggestions ?

- 原始邮件 -发件人："Justin Ling Mao" 

收件人："dev" 

主题：A big refactor for the documetations

日期：2019年02月12日 11点50分



Hi gugs:I'am working on doing a big refactor for the documetations.it aims to  
- 1.make a better reading experiences and help users know more about zookeeper 
quickly,as good as other projects' doc(e.g redis,hbase). - 2.have less changes 
to diff with the original docs as far as possible. - 3.solve the problem when 
we have some new features or improvements,but cannot find a good place to doc 
it.

The new catalog may looks kile this:* is new one added. ** is the one 
to keep unchanged as far as possible. *** is the one 
modified.--|---Overview
|---Welcome ** [1.1]|---Overview ** [1.2]|---Getting Started ** 
[1.3]|---Release Notes ** [1.4]|---Developer|---API *** [2.1]
|---Programmer's Guide ** [2.2]|---Recipes *** [2.3]|---Clients * [2.4] 
   |---Use Cases * [2.5]|---Admin & Ops|---Administrator's Guide ** [3.1]   
 |---Quota Guide ** [3.2]|---JMX ** [3.3]|---Observers Guide ** [3.4]   
 |---Dynamic Reconfiguration ** [3.5]|---Zookeeper CLI * [3.6]|---Shell 
* [3.7]|---Configuration flags * [3.8]|---Troubleshooting & Tuning  * 
[3.9]|---Contributor Guidelines|---General Guidelines * [4.1]
|---ZooKeeper Internals ** [4.2]|---Miscellaneous|---Wiki ** [5.1]
|---Mailing Lists ** 
[5.2]--

The Roadmap is:1.(I pick up it : D)  1.1 write API[2.1], which includes the：
 1.1.1  original API Docs which is a Auto-generated java doc,just give a link.  
  1.1.2. Restful-api (the apis under the 
/zookeeper-contrib-rest/src/main/java/org/apache/zookeeper/server/jersey/resources)
  1.2 write Clients[2.4], which includes the:   1.2.1 C client   1.2.2 
zk-python, kazoo  1.2.3 Curator etc...  look at an example from: 
https://redis.io/clients



2. write Recipes[2.3], which includes the:  - integrate "Java Example" and 
"Barrier and Queue Tutorial"(Since some bugs in the examples and they are 
obsolete，we may delete something) into it.  - suggest users to use the recipes 
implements of Curator and link to the Curator's recipes doc.

3. write Zookeeper CLI[3.6], which includes the:  - about how to use the zk 
command line interface [./zkCli.sh]e.g  ls /; get ; rmr;create -e -p 
etc...  - look at an example from redis: https://redis.io/topics/rediscli

4. write shell[3.7], which includes the:   - list all usages of the shells 
under the zookeeper/bin. (e.g zkTxnLogToolkit.sh,zkCleanup.sh)

5. write Configuration flags[3.8], which includes the:   - list all usages of 
configurations properties(e.g zookeeper.snapCount):- move the original 
Advanced Configuration part of zookeeperAdmin.md into it. look at an 
example from:https://coreos.com/etcd/docs/latest/op-guide/configuration.html   
6. write Troubleshooting & Tuning[3.9], which includes the:   - move the 
original "Gotchas: Common Problems and Troubleshooting" part of Administrator's 
Guide.md into it.   - move the original "FAQ" into into it.   - add some new 
contents （e.g 
https://www.yumpu.com/en/document/read/29574266/building-an-impenetrable-zookeeper-pdf-github）.
   look at an example from:https://redis.io/topics/problems 

回复：A big refactor for the documetations

[Justin Ling Mao]

ping @anmolnar and others, any insights or suggestions ?
- 原始邮件 -发件人："Justin Ling Mao" 
收件人："dev" 
主题：A big refactor for the documetations
日期：2019年02月12日 11点50分

Hi gugs:I'am working on doing a big refactor for the documetations.it aims to  
- 1.make a better reading experiences and help users know more about zookeeper 
quickly,as good as other projects' doc(e.g redis,hbase). - 2.have less changes 
to diff with the original docs as far as possible. - 3.solve the problem when 
we have some new features or improvements,but cannot find a good place to doc 
it.
The new catalog may looks kile this:* is new one added. ** is the one 
to keep unchanged as far as possible. *** is the one 
modified.--|---Overview
|---Welcome ** [1.1]|---Overview ** [1.2]|---Getting Started ** 
[1.3]|---Release Notes ** [1.4]|---Developer|---API *** [2.1]
|---Programmer's Guide ** [2.2]|---Recipes *** [2.3]|---Clients * [2.4] 
   |---Use Cases * [2.5]|---Admin & Ops|---Administrator's Guide ** [3.1]   
 |---Quota Guide ** [3.2]|---JMX ** [3.3]|---Observers Guide ** [3.4]   
 |---Dynamic Reconfiguration ** [3.5]|---Zookeeper CLI * [3.6]|---Shell 
* [3.7]|---Configuration flags * [3.8]|---Troubleshooting & Tuning  * 
[3.9]|---Contributor Guidelines|---General Guidelines * [4.1]
|---ZooKeeper Internals ** [4.2]|---Miscellaneous|---Wiki ** [5.1]
|---Mailing Lists ** 
[5.2]--
The Roadmap is:1.(I pick up it : D)  1.1 write API[2.1], which includes the：
 1.1.1  original API Docs which is a Auto-generated java doc,just give a link.  
  1.1.2. Restful-api (the apis under the 
/zookeeper-contrib-rest/src/main/java/org/apache/zookeeper/server/jersey/resources)
  1.2 write Clients[2.4], which includes the:   1.2.1 C client   1.2.2 
zk-python, kazoo  1.2.3 Curator etc...  look at an example from: 
https://redis.io/clients

2. write Recipes[2.3], which includes the:  - integrate "Java Example" and 
"Barrier and Queue Tutorial"(Since some bugs in the examples and they are 
obsolete，we may delete something) into it.  - suggest users to use the recipes 
implements of Curator and link to the Curator's recipes doc.
3. write Zookeeper CLI[3.6], which includes the:  - about how to use the zk 
command line interface [./zkCli.sh]e.g  ls /; get ; rmr;create -e -p 
etc...  - look at an example from redis: https://redis.io/topics/rediscli
4. write shell[3.7], which includes the:   - list all usages of the shells 
under the zookeeper/bin. (e.g zkTxnLogToolkit.sh,zkCleanup.sh)
5. write Configuration flags[3.8], which includes the:   - list all usages of 
configurations properties(e.g zookeeper.snapCount):- move the original 
Advanced Configuration part of zookeeperAdmin.md into it. look at an 
example from:https://coreos.com/etcd/docs/latest/op-guide/configuration.html   
6. write Troubleshooting & Tuning[3.9], which includes the:   - move the 
original "Gotchas: Common Problems and Troubleshooting" part of Administrator's 
Guide.md into it.   - move the original "FAQ" into into it.   - add some new 
contents （e.g 
https://www.yumpu.com/en/document/read/29574266/building-an-impenetrable-zookeeper-pdf-github）.
   look at an example from:https://redis.io/topics/problems 
  https://coreos.com/etcd/docs/latest/tuning.html
7. write General Guidelines[4.1], which includes the:  - move the original 
"Logging" part of ZooKeeper Internals into it as the logger specification.  - 
write specifications about code, git commit messages,github PR  etc ...look 
at an example from:http://hbase.apache.org/book.html#hbase.commit.msg.format

8. write Use Cases[2.5], which includes the:  - just move the context from: 
https://cwiki.apache.org/confluence/display/ZOOKEEPER/PoweredBy into it.  - add 
some new contents.(e.g Apache Projects:Spark;Companies:twitter,fb)
--BTW:- Any 
insights or suggestions are very welcomed.After the dicussions,I will create a 
series of tickets(An umbrella)- Since these works can be done parallelly, if 
you are interested in them, please don't hesitate,just assign to yourself, pick 
it up. (Notice: give me a ping to avoid the duplicated work).

A big refactor for the documetations

[Justin Ling Mao]

Hi gugs:I'am working on doing a big refactor for the documetations.it aims to  
- 1.make a better reading experiences and help users know more about zookeeper 
quickly,as good as other projects' doc(e.g redis,hbase). - 2.have less changes 
to diff with the original docs as far as possible. - 3.solve the problem when 
we have some new features or improvements,but cannot find a good place to doc 
it.
The new catalog may looks kile this:* is new one added. ** is the one 
to keep unchanged as far as possible. *** is the one 
modified.--|---Overview
|---Welcome ** [1.1]|---Overview ** [1.2]|---Getting Started ** 
[1.3]|---Release Notes ** [1.4]|---Developer|---API *** [2.1]
|---Programmer's Guide ** [2.2]|---Recipes *** [2.3]|---Clients * [2.4] 
   |---Use Cases * [2.5]|---Admin & Ops|---Administrator's Guide ** [3.1]   
 |---Quota Guide ** [3.2]|---JMX ** [3.3]|---Observers Guide ** [3.4]   
 |---Dynamic Reconfiguration ** [3.5]|---Zookeeper CLI * [3.6]|---Shell 
* [3.7]|---Configuration flags * [3.8]|---Troubleshooting & Tuning  * 
[3.9]|---Contributor Guidelines|---General Guidelines * [4.1]
|---ZooKeeper Internals ** [4.2]|---Miscellaneous|---Wiki ** [5.1]
|---Mailing Lists ** 
[5.2]--
The Roadmap is:1.(I pick up it : D)  1.1 write API[2.1], which includes the：
 1.1.1  original API Docs which is a Auto-generated java doc,just give a link.  
  1.1.2. Restful-api (the apis under the 
/zookeeper-contrib-rest/src/main/java/org/apache/zookeeper/server/jersey/resources)
  1.2 write Clients[2.4], which includes the:   1.2.1 C client   1.2.2 
zk-python, kazoo  1.2.3 Curator etc...  look at an example from: 
https://redis.io/clients

2. write Recipes[2.3], which includes the:  - integrate "Java Example" and 
"Barrier and Queue Tutorial"(Since some bugs in the examples and they are 
obsolete，we may delete something) into it.  - suggest users to use the recipes 
implements of Curator and link to the Curator's recipes doc.
3. write Zookeeper CLI[3.6], which includes the:  - about how to use the zk 
command line interface [./zkCli.sh]e.g  ls /; get ; rmr;create -e -p 
etc...  - look at an example from redis: https://redis.io/topics/rediscli
4. write shell[3.7], which includes the:   - list all usages of the shells 
under the zookeeper/bin. (e.g zkTxnLogToolkit.sh,zkCleanup.sh)
5. write Configuration flags[3.8], which includes the:   - list all usages of 
configurations properties(e.g zookeeper.snapCount):- move the original 
Advanced Configuration part of zookeeperAdmin.md into it. look at an 
example from:https://coreos.com/etcd/docs/latest/op-guide/configuration.html   
6. write Troubleshooting & Tuning[3.9], which includes the:   - move the 
original "Gotchas: Common Problems and Troubleshooting" part of Administrator's 
Guide.md into it.   - move the original "FAQ" into into it.   - add some new 
contents （e.g 
https://www.yumpu.com/en/document/read/29574266/building-an-impenetrable-zookeeper-pdf-github）.
   look at an example from:https://redis.io/topics/problems 
  https://coreos.com/etcd/docs/latest/tuning.html
7. write General Guidelines[4.1], which includes the:  - move the original 
"Logging" part of ZooKeeper Internals into it as the logger specification.  - 
write specifications about code, git commit messages,github PR  etc ...look 
at an example from:http://hbase.apache.org/book.html#hbase.commit.msg.format

8. write Use Cases[2.5], which includes the:  - just move the context from: 
https://cwiki.apache.org/confluence/display/ZOOKEEPER/PoweredBy into it.  - add 
some new contents.(e.g Apache Projects:Spark;Companies:twitter,fb)
--BTW:- Any 
insights or suggestions are very welcomed.After the dicussions,I will create a 
series of tickets(An umbrella)- Since these works can be done parallelly, if 
you are interested in them, please don't hesitate,just assign to yourself, pick 
it up. (Notice: give me a ping to avoid the duplicated work).