I checked the https://issues.apache.org/jira/browse/ZOOKEEPER-3107 - checkstyle ticket. I think we should definitely leave this after maven migration is complete. It needs a lot of code cleanup, and as Enrico have mentioned, probably breaks most current PRs. Also, I think this needs to be voted by the community, if we want this feature or not. If we do, I think it should be done ASAP, because git blame already gives false information on github due to not supporting --follow as a consequence of maven migration. This will surely have similar effects, so the two should be close as possible to minimize inconvenience caused by this. We already had some discussion here about the git blame BTW, it's unfortunate, but the history is not lost!
Regards, Norbert On Thu, Dec 20, 2018 at 10:45 AM Enrico Olivelli <[email protected]> wrote: > Il giorno gio 20 dic 2018 alle ore 10:44 Norbert Kalmar > <[email protected]> ha scritto: > > > > Hi, > > > > I created this jira for static code analysis - > > https://issues.apache.org/jira/browse/ZOOKEEPER-3223 > > I have picked this up and renamed to "Configure spotbugs" > > Enrico > > > Enrico mentioned he doesn't recommend it. I didn't go into this topic in > > detail yet, so maybe you should check in on that jira. I'll have a look > > later today. > > Thanks for bringing attention to this jira! > > > > Regards, > > Norbert > > > > On Thu, Dec 20, 2018 at 10:32 AM 毛蛤丝 <[email protected]> wrote: > > > > > @Norbert > > > this issue about Checkstyle is on your plan? > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3107 > > > ----- 原始邮件 ----- > > > 发件人：Norbert Kalmar <[email protected]> > > > 收件人：[email protected] > > > 主题：Re: ZooKeeper 3.5 blocker issues > > > 日期：2018年12月20日 17点07分 > > > > > > Subtasks: > > > Findbugs, checkstyle - > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3223 > > > CI integration - https://issues.apache.org/jira/browse/ZOOKEEPER-3224 > > > Code coverage - https://issues.apache.org/jira/browse/ZOOKEEPER-3225 > - I > > > laready started this one and some of it is committed with the patch, > so I > > > will continue to work on it. > > > Recipes and contrib - > https://issues.apache.org/jira/browse/ZOOKEEPER-3171 > > > - Already on it, recipes is done, PR soon available. > > > Assembly - https://issues.apache.org/jira/browse/ZOOKEEPER-3029 > > > This are the tasks left I can think of. If anything is missing, feel > free > > > to create a jira, or let me know. > > > The ones I'm already working on - 3225, 3171 - I made a comment, Those > > > should be ready this week. > > > Thanks, > > > Norbert > > > On Thu, Dec 20, 2018 at 9:07 AM Enrico Olivelli <[email protected]> > > > wrote: > > > > Great. > > > > Can you create JIRA tickets for remaining subtask? So that I can pick > > > them > > > > up? > > > > I volounter for spotbugs and for CI integration, but let's see the > list > > > > Enrico > > > > > > > > Il gio 20 dic 2018, 07:21 Andor Molnar <[email protected]> ha > scritto: > > > > > > > > > Ok. Looks like ant still works properly, so let’s commit this > patch and > > > > > you guys can collaborate to polish the Maven build. > > > > > > > > > > For now, it’s master-only. > > > > > > > > > > Thanks, > > > > > Andor > > > > > > > > > > > > > > > > > > > > > On 2018. Dec 19., at 16:44, Norbert Kalmar > > > > <[email protected]> > > > > > wrote: > > > > > > > > > > > > Thank you Enrico, I agree, that we could commit this patch at > it's > > > > > current > > > > > > state, it fulfills the original jira anyways. > > > > > > > > > > > > I'll see what's wrong with the java tests, but honestly, it looks > > > like > > > > > > they're just flaky... runs well on local builds with 8 thread. > > > > > > > > > > > > Regards, > > > > > > Norbert > > > > > > > > > > > > On Wed, Dec 19, 2018 at 2:50 PM Tamas Penzes > > > > <[email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > >> Hi All, > > > > > >> > > > > > >> For assembly task I would promote the way how HBase works. > > > > > >> They create a pure source and a bin tarball separately. Please > see > > > how > > > > > they > > > > > >> create a release here: > > > > > >> > https://github.com/apache/hbase/blob/master/dev-support/make_rc.sh > > > > > >> We could probably use the well known "copy+paste technology" to > have > > > > it > > > > > >> within ZooKeeper the same way. ;-) > > > > > >> > > > > > >> Regards, Tamaas > > > > > >> > > > > > >> On Wed, Dec 19, 2018 at 2:28 PM Enrico Olivelli < > > > [email protected]> > > > > > >> wrote: > > > > > >> > > > > > >>> Great work Norbert > > > > > >>> I you want I can help,especially for rat, findbugs (need to > switch > > > to > > > > > >>> spotbugs anyway) and OWASP stuff (recently I started using > Maven > > > > > >>> Plugin in other projects) > > > > > >>> But I am not sure how can I help you concretely if we do not > commit > > > > > your > > > > > >>> work. > > > > > >>> We could commit the work as it is now, leaving "ant" as > official > > > > build > > > > > >>> method, but having the poms committed will ease collaboration. > > > > > >>> > > > > > >>> We will also have to work on CI jobs, I can help on that part > as > > > well > > > > > >>> > > > > > >>> Enrico > > > > > >>> > > > > > >>> Il giorno mer 19 dic 2018 alle ore 12:26 Norbert Kalmar > > > > > >>> <[email protected]> ha scritto: > > > > > >>>> > > > > > >>>> Hi everyone, > > > > > >>>> > > > > > >>>> Some update on the maven migration: I had a few bumps here and > > > there > > > > > >>> (just > > > > > >>>> looking at the latest patch Andor linked - > > > > > >>>> https://github.com/apache/zookeeper/pull/708 - you can see > on the > > > > > >>> commits). > > > > > >>>> Current state is that the build works, tests run, but reports > like > > > > > >>>> findbugs, clover etc. are not yet implemented. Maven has > plugins > > > for > > > > > >> them > > > > > >>>> usually, but it's not always trivial, especially with the C > > > client. > > > > > The > > > > > >>>> assembly is also left to be done, but it should be fairly > easy to > > > > do a > > > > > >>>> similar tarball then ant does (although this will be also an > > > > > >> interesting > > > > > >>>> task, as ant does some strange things, like duplicated > sources of > > > > most > > > > > >>>> contrib projects). > > > > > >>>> > > > > > >>>> I had a seperate jira to do the recipes and contrib maven > build. I > > > > do > > > > > >> not > > > > > >>>> have open PR for it, but recipes is done and I am now working > on > > > the > > > > > >>>> contrib projects. Most of them is manually build and never > gets > > > > called > > > > > >>> from > > > > > >>>> the main build.xml. I will not integrate these either to the > maven > > > > > >> build. > > > > > >>>> The reason is that there are plans to remove some of them > from ZK > > > > repo > > > > > >>>> anyway. The other reason is that for starters, we want to > > > replicate > > > > > the > > > > > >>> ant > > > > > >>>> build as closely as possible, without doing any nasty > workarounds > > > in > > > > > >>> maven > > > > > >>>> to achieve that. And from there, we can improve, use maven's > > > > > advantages > > > > > >>> to > > > > > >>>> shape the build of ZooKeeper. Once it is stable and proven to > have > > > > all > > > > > >>> the > > > > > >>>> functionality required for build and release. > > > > > >>>> > > > > > >>>> Right now, I am trying to stabilize the build as much as > possible. > > > > > >> Andor > > > > > >>>> also fixed some flaky C tests that for some strange reasons, > > > become > > > > > >>>> extremely flaky with the maven build: > > > > > >>>> https://github.com/apache/zookeeper/pull/740 > > > > > >>>> > > > > > >>>> Regards, > > > > > >>>> Norbert > > > > > >>>> > > > > > >>>> On Tue, Dec 18, 2018 at 9:52 AM Andor Molnar > > > > > >> <[email protected] > > > > > >>>> > > > > > >>>> wrote: > > > > > >>>> > > > > > >>>>> Sure, good point. Let's put it on the list. > > > > > >>>>> > > > > > >>>>> Andor > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> On Tue, Dec 18, 2018 at 12:17 AM Patrick Hunt < > [email protected]> > > > > > >>> wrote: > > > > > >>>>> > > > > > >>>>>> Are folks OK to wait on that OWASP issue I documented over > the > > > > > >>> weekend? > > > > > >>>>>> afaict we are not affected but it would be good to get > another > > > > pair > > > > > >>> of > > > > > >>>>> eyes > > > > > >>>>>> on it. > > > > > >>>>>> > > > > > >>>>>> Patrick > > > > > >>>>>> > > > > > >>>>>> On Mon, Dec 17, 2018 at 2:55 PM Andor Molnár < > [email protected]> > > > > > >>> wrote: > > > > > >>>>>> > > > > > >>>>>>> Hi team, > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> I'm proudly announce that thanks to the joint effort from > the > > > > > >>>>> community, > > > > > >>>>>>> the 3.5 blockers list has become empty: > > > > > >>>>>>> > > > > > >>>>>>> "project = ZooKeeper AND resolution = Unresolved AND > > > fixVersion = > > > > > >>> 3.5.5 > > > > > >>>>>>> AND priority in (blocker, critical) ORDER BY priority > DESC, key > > > > > >>> ASC" > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> Well... almost. All the blocker issues have gone, but we > still > > > > > >>> have the > > > > > >>>>>>> Maven migration to complete before the stable release. If > you > > > > > >> have > > > > > >>> some > > > > > >>>>>>> free cycles, please join us testing the Maven build on > this PR: > > > > > >>>>>>> > > > > > >>>>>>> https://github.com/apache/zookeeper/pull/708 > > > > > >>>>>>> > > > > > >>>>>>> I hope we can merge it pretty soon. > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> In terms of the builds, the weather at 3.5 branch is quite > > > sunny > > > > > >>>>>> nowadays: > > > > > >>>>>>> > > > > > >>>>>>> https://builds.apache.org/view/S-Z/view/ZooKeeper/ > > > > > >>>>>>> > > > > > >>>>>>> The Java 11 build is still having some difficulties, which > > > > > >>> hopefully I > > > > > >>>>>>> can address before the holidays: > > > > > >>>>>>> > > > > > >>>>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3204 > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> If you happen to know about something which is important > from > > > > > >> 3.5's > > > > > >>>>>>> perspective and missing from the above, please don't > hesitate > > > to > > > > > >>> share. > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> Happy ZooKeeping! > > > > > >>>>>>> > > > > > >>>>>>> Andor > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> On 11/2/18 21:12, Fangmin Lv wrote: > > > > > >>>>>>>> Andor, > > > > > >>>>>>>> > > > > > >>>>>>>> Here is the PR to port ZK-3104 from master to 3.4: > > > > > >>>>>>>> https://github.com/apache/zookeeper/pull/685. > > > > > >>>>>>>> > > > > > >>>>>>>> Fangmin > > > > > >>>>>>>> > > > > > >>>>>>>> On Fri, Nov 2, 2018 at 11:46 AM Fangmin Lv < > > > > > >> [email protected]> > > > > > >>>>>> wrote: > > > > > >>>>>>>> > > > > > >>>>>>>>> Hi Andor, > > > > > >>>>>>>>> > > > > > >>>>>>>>> Is anyone working on ZK-2778? I can pick it up if there > is no > > > > > >>> one > > > > > >>>>>>> working > > > > > >>>>>>>>> on it yet. > > > > > >>>>>>>>> > > > > > >>>>>>>>> I'll open a 3.5 PR for ZK-3104 today. > > > > > >>>>>>>>> > > > > > >>>>>>>>> Fangmin > > > > > >>>>>>>>> > > > > > >>>>>>>>> On Fri, Oct 26, 2018 at 3:33 AM Andor Molnar < > > > > > >> [email protected]> > > > > > >>>>>> wrote: > > > > > >>>>>>>>> > > > > > >>>>>>>>>> Hi folks, > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> You’ve probably realised lots of update emails coming > from > > > > > >>> Jira. > > > > > >>>>>> Please > > > > > >>>>>>>>>> be aware that we’ve updated a bunch of open > blocker/critical > > > > > >>> 3.5 > > > > > >>>>>>> tickets to > > > > > >>>>>>>>>> reflect to what we discussed in this email. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> If you open up the following jira filter: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> project = ZooKeeper and resolution = Unresolved and > > > > > >> fixVersion > > > > > >>> = > > > > > >>>>>> 3.5.5 > > > > > >>>>>>>>>> AND priority in (blocker, critical) ORDER BY priority > DESC, > > > > > >>> key ASC > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> You’ll see the most up-to-date list of tickets which > need to > > > > > >> be > > > > > >>>>>>> addressed > > > > > >>>>>>>>>> before the stable 3.5 release. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Thank you for your efforts to get this done. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Fangmin, ZK-3104 is waiting for backport, but ticket has > > > > > >>> already > > > > > >>>>> been > > > > > >>>>>>>>>> resolved. Have you created a separate ticket for the > > > backport > > > > > >>> or > > > > > >>>>>> shall > > > > > >>>>>>> I > > > > > >>>>>>>>>> just reopen it with the right fix versions? > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Thanks, > > > > > >>>>>>>>>> Andor > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>>> On 2018. Oct 8., at 12:34, Andor Molnar < > [email protected]> > > > > > >>> wrote: > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> Hi, > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> Let me summarize and give a quick update on the > outstanding > > > > > >>> issues > > > > > >>>>>> for > > > > > >>>>>>>>>> 3.5 GA: > > > > > >>>>>>>>>>> - ZOOKEEPER-1818 (Fix don't care for trunk) > > > > > >>>>>>>>>>> - ZOOKEEPER-2778 (Potential server deadlock between > > > follower > > > > > >>> sync > > > > > >>>>>> with > > > > > >>>>>>>>>> leader and follower receiving external connection > requests.) > > > > > >>>>>>>>>>> - ZOOKEEPER-3021 Migrate project structure to Maven > > > > > >> (ongoing) > > > > > >>>>>>>>>>> - ZOOKEEPER-925 Docs generation to Maven > > > > > >>>>>>>>>>> - ZOOKEEPER-3104 (waiting for backport) > > > > > >>>>>>>>>>> - ZOOKEEPER-3125 (waiting for backport PR #647) > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> The 2 Maven related tickets are no-brainers as well as > the > > > > > >>>>>> backports. > > > > > >>>>>>>>>> ZK-2778 has been picked up by Maoling (thanks!) as far > as I > > > > > >> can > > > > > >>>>> see, > > > > > >>>>>>>>>> ZK-1818 is the only one waiting for a volunteer. > > > > > >>>>>>>>>>> Please correct me if I’ve missed something. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> Regards, > > > > > >>>>>>>>>>> Andor > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>>> On 2018. Sep 28., at 18:32, Tamas Penzes > > > > > >>>>>> <[email protected] > > > > > >>>>>>>> > > > > > >>>>>>>>>> wrote: > > > > > >>>>>>>>>>>> Hi All, > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> I would add ZOOKEEPER-3021 > > > > > >>>>>>>>>>>> <https://issues.apache.org/jira/browse/ZOOKEEPER-3021 > > > > > > > >>> Migrate > > > > > >>>>>>> project > > > > > >>>>>>>>>>>> structure to Maven build as a blocker too. Since the > > > > > >>> migration > > > > > >>>>> has > > > > > >>>>>>>>>> started > > > > > >>>>>>>>>>>> it would be good to finish before releasing ZK 3.5.x > GA. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> ZOOKEEPER-925 < > > > > > >>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-925 > > > > > >>>>>>> > > > > > >>>>>>>>>> replace > > > > > >>>>>>>>>>>> our forrest site and documentation generation might > also > > > > > >> be a > > > > > >>>>> good > > > > > >>>>>>>>>> idea, > > > > > >>>>>>>>>>>> since then we could deliver the new MarkDown based > > > > > >>> documentation. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Regards, Tamaas > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> On Fri, Sep 14, 2018 at 10:09 AM Fangmin Lv < > > > > > >>> [email protected] > > > > > >>>>>> > > > > > >>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>> Oh, sorry for the confusion, I should provide more > > > > > >> context. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Leader will use on disk txn sync with followers to > if the > > > > > >>> peer > > > > > >>>>>> zxid > > > > > >>>>>>>>>> is not > > > > > >>>>>>>>>>>>> in it's in memory commit logs, the code is here: > Leader > > > on > > > > > >>> disk > > > > > >>>>>> txn > > > > > >>>>>>>>>> sync > > > > > >>>>>>>>>>>>> < > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > https://github.com/apache/zookeeper/blob/master/src/java/main/org/apache/zookeeper/server/quorum/LearnerHandler.java#L774 > > > > > >>>>>>>>>>>>>> . > > > > > >>>>>>>>>>>>> There is bug that potentially there will be gap in > the > > > txn > > > > > >>>>> files, > > > > > >>>>>>> like > > > > > >>>>>>>>>>>>> after snap sync, etc, so it's possible the peer will > miss > > > > > >>> txns > > > > > >>>>> due > > > > > >>>>>>> to > > > > > >>>>>>>>>> this. > > > > > >>>>>>>>>>>>> The option to disable it is snapshotSizeFactor > > > > > >>>>>>>>>>>>> < > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > https://github.com/apache/zookeeper/blob/master/src/java/main/org/apache/zookeeper/server/ZKDatabase.java#L81 > > > > > >>>>>>>>>>>>>> , > > > > > >>>>>>>>>>>>> set it to -1 will disable this feature. On 3.5, it's > > > > > >> better > > > > > >>> to > > > > > >>>>>> have > > > > > >>>>>>> a > > > > > >>>>>>>>>> PR to > > > > > >>>>>>>>>>>>> set this to -1 by default. It might have more SNAP > sync, > > > > > >> but > > > > > >>>>> from > > > > > >>>>>>> our > > > > > >>>>>>>>>> prod > > > > > >>>>>>>>>>>>> it doesn't seem to be a big problem to me. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> I can send out the diff to disable it by default on > 3.5 > > > if > > > > > >>> you > > > > > >>>>>> guys > > > > > >>>>>>>>>> think > > > > > >>>>>>>>>>>>> this is the right way to do. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Thanks, > > > > > >>>>>>>>>>>>> Fangmin > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> On Thu, Sep 13, 2018 at 1:58 AM Andor Molnar < > > > > > >>> [email protected]> > > > > > >>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>> What’s needed to turn it off? > > > > > >>>>>>>>>>>>>> Do we need a PR or it’s just a config option? > > > > > >>>>>>>>>>>>>> Shall we implement a feature switch for that and > turn it > > > > > >>> off by > > > > > >>>>>>>>>> default? > > > > > >>>>>>>>>>>>>> Sorry I don’t have too much insight on disk txn > sync. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Andor > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> On 2018. Sep 13., at 9:16, Fangmin Lv < > > > > > >>> [email protected]> > > > > > >>>>>>> wrote: > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> And to be clear, ZOOKEEPER-2418 is actually just > one > > > > > >> case > > > > > >>> of > > > > > >>>>>>>>>>>>>> inconsistency > > > > > >>>>>>>>>>>>>>> which could caused by on disk txn sync, as I > mentioned > > > > > >> in > > > > > >>> a > > > > > >>>>>> newer > > > > > >>>>>>>>>> JIRA > > > > > >>>>>>>>>>>>>>> ZOOKEEPER-2846 < > > > > > >>>>>>>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-2846>, > > > > > >>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>> snap sync or txn sync could also leave txns gap in > the > > > > > >> txn > > > > > >>>>> file, > > > > > >>>>>>>>>> which > > > > > >>>>>>>>>>>>>> is a > > > > > >>>>>>>>>>>>>>> more common case could trigger this issue. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> I would suggest to turn off the on disk txn sync by > > > > > >>> default > > > > > >>>>> for > > > > > >>>>>>> now > > > > > >>>>>>>>>> to > > > > > >>>>>>>>>>>>>>> avoid this issue, after we finished > ZOOKEEPER-3114, we > > > > > >>> can use > > > > > >>>>>>> that > > > > > >>>>>>>>>> to > > > > > >>>>>>>>>>>>>>> validate the on disk txns during syncing. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Thanks, > > > > > >>>>>>>>>>>>>>> Fangmin > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> On Wed, Sep 12, 2018 at 9:55 AM Fangmin Lv < > > > > > >>>>> [email protected] > > > > > >>>>>>> > > > > > >>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>> Andor, > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> ZOOKEEPER-3114 is about adding real time digest > > > > > >> checking > > > > > >>> to > > > > > >>>>>> help > > > > > >>>>>>>>>>>>>> detecting > > > > > >>>>>>>>>>>>>>>> inconsistency, it's a new feature with amounts of > code > > > > > >>>>> change. > > > > > >>>>>>> I'll > > > > > >>>>>>>>>>>>>> start > > > > > >>>>>>>>>>>>>>>> upstream it part by part, but I don't expect it's > > > being > > > > > >>>>> merged > > > > > >>>>>> in > > > > > >>>>>>>>>> the > > > > > >>>>>>>>>>>>>> next > > > > > >>>>>>>>>>>>>>>> few weeks. So yes, it's a nice to have, but > definitely > > > > > >>> not a > > > > > >>>>>>> block > > > > > >>>>>>>>>> for > > > > > >>>>>>>>>>>>>> 3.5. > > > > > >>>>>>>>>>>>>>>> Thanks, > > > > > >>>>>>>>>>>>>>>> Fangmin > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> On Wed, Sep 12, 2018 at 2:55 AM Andor Molnar < > > > > > >>>>> [email protected] > > > > > >>>>>>> > > > > > >>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>> Fangmin, > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Sorry, I just noticed that you want to include > the > > > > > >>>>> consistency > > > > > >>>>>>>>>> fixes > > > > > >>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>> the stable version which is fine. Let’s finish > the > > > > > >>> backports > > > > > >>>>>> and > > > > > >>>>>>>>>>>>> we’ll > > > > > >>>>>>>>>>>>>> be > > > > > >>>>>>>>>>>>>>>>> done with them. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> ZOOKEEPER-3114 is essentially a new feature, I > > > > > >> wouldn’t > > > > > >>>>> block > > > > > >>>>>>> 3.5 > > > > > >>>>>>>>>>>>> with > > > > > >>>>>>>>>>>>>>>>> that. What do you think? > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Andor > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> On 2018. Sep 12., at 11:52, Andor Molnar < > > > > > >>> [email protected] > > > > > >>>>>> > > > > > >>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>> Cool, thanks for the clarification. > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> The updated list is as follows: > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> - ZOOKEEPER-236 (SSL/TLS support for Atomic > > > Broadcast > > > > > >>>>>> protocol) > > > > > >>>>>>>>>>>>>>>>>> - ZOOKEEPER-1818 (Fix don't care for trunk) > > > > > >>>>>>>>>>>>>>>>>> - ZOOKEEPER-2778 (Potential server deadlock > between > > > > > >>>>> follower > > > > > >>>>>>> sync > > > > > >>>>>>>>>>>>> with > > > > > >>>>>>>>>>>>>>>>> leader and follower receiving external connection > > > > > >>> requests.) > > > > > >>>>>>>>>>>>>>>>>> The following are not critical and no blockers > for > > > > > >> the > > > > > >>>>> stable > > > > > >>>>>>>>>>>>> release: > > > > > >>>>>>>>>>>>>>>>>> Waiting for to be ported to 3.5: > > > > > >>>>>>>>>>>>>>>>>> - ZOOKEEPER-3104 > > > > > >>>>>>>>>>>>>>>>>> - ZOOKEEPER-3125 > > > > > >>>>>>>>>>>>>>>>>> - ZOOKEEPER-3127 > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> New feature: > > > > > >>>>>>>>>>>>>>>>>> - ZOOKEEPER-3114 (fixes ZOOKEEPER-2184 too) > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> Regards, > > > > > >>>>>>>>>>>>>>>>>> Andor > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> On 2018. Sep 12., at 0:42, Fangmin Lv < > > > > > >>>>> [email protected]> > > > > > >>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>> Hi Andor, > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> That's the on disk txn feature, which was > disabled > > > > > >>>>>> internally > > > > > >>>>>>>>>> after > > > > > >>>>>>>>>>>>>> we > > > > > >>>>>>>>>>>>>>>>>>> found the potentially inconsistent issue. The > only > > > > > >>>>> solution > > > > > >>>>>> we > > > > > >>>>>>>>>> have > > > > > >>>>>>>>>>>>>>>>> for now > > > > > >>>>>>>>>>>>>>>>>>> is waiting for the new digest checking feature > I > > > > > >>> mentioned > > > > > >>>>>> in > > > > > >>>>>>>>>>>>>>>>>>> ZOOKEEPER-3114. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> I think there are some other critical > consistent > > > > > >>> issues we > > > > > >>>>>>> just > > > > > >>>>>>>>>>>>> fixed > > > > > >>>>>>>>>>>>>>>>> on > > > > > >>>>>>>>>>>>>>>>>>> master recently: ZOOKEEPER-3104, > ZOOKEEPER-3125, > > > > > >>>>>>>>>> ZOOKEEPER-3127, I > > > > > >>>>>>>>>>>>>>>>> think we > > > > > >>>>>>>>>>>>>>>>>>> should include that in the official 3.5 > release as > > > > > >>> well. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> Thanks, > > > > > >>>>>>>>>>>>>>>>>>> Fangmin > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> On Tue, Sep 11, 2018 at 11:58 AM Andor Molnár < > > > > > >>>>>>> [email protected] > > > > > >>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>> Hi Jeelani, > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> Thanks for letting me know. I'm happy to > remove it > > > > > >>> from > > > > > >>>>> the > > > > > >>>>>>>>>> list > > > > > >>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>> get > > > > > >>>>>>>>>>>>>>>>>>>> closer to a stable release. :) > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> What's the feature which can be disabled to > avoid > > > > > >>> data > > > > > >>>>>>>>>>>>>> inconsistency? > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> Andor > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> On 09/10/2018 11:33 PM, Mohamed Jeelani wrote: > > > > > >>>>>>>>>>>>>>>>>>>>> Thanks Andor for compiling this. Should we be > > > > > >>> ignoring > > > > > >>>>>>>>>>>>>>>>> ZOOKEEPER-2418 as > > > > > >>>>>>>>>>>>>>>>>>>> well? This exists in 3.4 as well and the > feature > > > > > >> can > > > > > >>> be > > > > > >>>>>>>>>> disabled. > > > > > >>>>>>>>>>>>> We > > > > > >>>>>>>>>>>>>>>>> are > > > > > >>>>>>>>>>>>>>>>>>>> working on a longer term fix for it in 3.6. > > > > > >>>>>>>>>>>>>>>>>>>>> Regards, > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> Jeelani > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> On 9/10/18, 5:19 AM, "Andor Molnar" > > > > > >>>>>>>>>> <[email protected] > > > > > >>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>>> Fine. > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> I'm happy to ignore 1549, 2846 and 2930. > Still we > > > > > >>> have > > > > > >>>>> the > > > > > >>>>>>>>>> list > > > > > >>>>>>>>>>>>>> of: > > > > > >>>>>>>>>>>>>>>>>>>>> - ZOOKEEPER-236 (SSL/TLS support for Atomic > > > > > >>> Broadcast > > > > > >>>>>>>>>> protocol) > > > > > >>>>>>>>>>>>>>>>>>>>> - ZOOKEEPER-1818 (Fix don't care for trunk) > > > > > >>>>>>>>>>>>>>>>>>>>> - ZOOKEEPER-2418 (txnlog diff sync can skip > > > > > >> sending > > > > > >>> some > > > > > >>>>>>>>>>>>>>>>>>>> transactions to > > > > > >>>>>>>>>>>>>>>>>>>>> followers) > > > > > >>>>>>>>>>>>>>>>>>>>> - ZOOKEEPER-2778 (Potential server deadlock > > > > > >> between > > > > > >>>>>> follower > > > > > >>>>>>>>>>>>> sync > > > > > >>>>>>>>>>>>>>>>>>>> with > > > > > >>>>>>>>>>>>>>>>>>>>> leader and follower receiving external > connection > > > > > >>>>>> requests.) > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> SSL (ZK-236) is a feature which essential > for the > > > > > >>> 3.5 > > > > > >>>>>>> release, > > > > > >>>>>>>>>>>>>>>>> hence > > > > > >>>>>>>>>>>>>>>>>>>> I > > > > > >>>>>>>>>>>>>>>>>>>>> wouldn't leave it out or postpone it for the > next > > > > > >>> stable > > > > > >>>>>>>>>>>>> release. > > > > > >>>>>>>>>>>>>>>>> PR > > > > > >>>>>>>>>>>>>>>>>>>> has > > > > > >>>>>>>>>>>>>>>>>>>>> been out for a long time, get on reviewing > > > please. > > > > > >>>>>>>>>>>>>>>>>>>>> The rest are also long outstanding issues > which > > > > > >> have > > > > > >>>>> been > > > > > >>>>>>>>>> found > > > > > >>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>>>>> the 3.5 > > > > > >>>>>>>>>>>>>>>>>>>>> branch. > > > > > >>>>>>>>>>>>>>>>>>>>> ZK-1818 is something which was found in 3.4 > and > > > > > >>> fixed in > > > > > >>>>>>> 3.4, > > > > > >>>>>>>>>>>>> but > > > > > >>>>>>>>>>>>>>>>>>>> never has > > > > > >>>>>>>>>>>>>>>>>>>>> been fixed in 3.5. Quite a serious issue if > still > > > > > >>>>> present. > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> I think we should at least run some manual > > > testing > > > > > >>> and > > > > > >>>>> see > > > > > >>>>>>> if > > > > > >>>>>>>>>> we > > > > > >>>>>>>>>>>>>>>>>>>> could > > > > > >>>>>>>>>>>>>>>>>>>>> repro any of these issues before going ahead > with > > > > > >> a > > > > > >>>>> stable > > > > > >>>>>>>>>>>>>> release. > > > > > >>>>>>>>>>>>>>>>>>>>> Regards, > > > > > >>>>>>>>>>>>>>>>>>>>> Andor > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> On Fri, Sep 7, 2018 at 3:24 AM, Michael Han < > > > > > >>>>>>> [email protected]> > > > > > >>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>>>> I haven't went through the entire list, but > > > looks > > > > > >>> like > > > > > >>>>>> lots > > > > > >>>>>>>>>> of > > > > > >>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>>>>> JIRA > > > > > >>>>>>>>>>>>>>>>>>>>>> issues listed in this thread, such as > > > > > >>> ZOOKEEPER-1549, > > > > > >>>>>> 2846, > > > > > >>>>>>>>>> also > > > > > >>>>>>>>>>>>>>>>>>>> affects > > > > > >>>>>>>>>>>>>>>>>>>>>> 3.4 releases. Should we scope these issues > out? > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> I think historically the single outstanding > > > > > >>> blocking > > > > > >>>>>> issue > > > > > >>>>>>>>>> for a > > > > > >>>>>>>>>>>>>>>>>>>> stable 3.5 > > > > > >>>>>>>>>>>>>>>>>>>>>> release is the reconfig feature and security > > > > > >>> concerns > > > > > >>>>>>> around > > > > > >>>>>>>>>> it > > > > > >>>>>>>>>>>>>>>>>>>> (somehow > > > > > >>>>>>>>>>>>>>>>>>>>>> addressed in ZOOKEEPER-2014), and the alpha > and > > > > > >>> beta > > > > > >>>>>>> releases > > > > > >>>>>>>>>>>>> were > > > > > >>>>>>>>>>>>>>>>>>>> created > > > > > >>>>>>>>>>>>>>>>>>>>>> to stabilize that feature. > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__zookeeper-2Duser.578899.n2.nabble.com_Zookeeper-2Dwith-2D&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Vl4oKanLQehvaulUvoKg8A&m=wqlhnot9c-pQLdkGkccSGNpELUNUnB-wy_h0iA3PRqI&s=_tGtL3nMWtuPrXKXDx27AIWOzyyT7W-CjIVLDFZwT0E&e= > > > > > >>>>>>>>>>>>>>>>>>>>>> SSL-release-date-tt7581744.html > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> So it looks like we are in good shape to > > > release. > > > > > >>>>>> Something > > > > > >>>>>>>>>>>>> might > > > > > >>>>>>>>>>>>>>>>>>>> worth > > > > > >>>>>>>>>>>>>>>>>>>>>> doing to claim the quality of 3.5 is on par > with > > > > > >>> 3.4 > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> * Run Jepsen on 3.5 - 3.4 passed the test > for > > > the > > > > > >>>>> record > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__aphyr.com_posts_291-2Djepsen-2Dzookeeper&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Vl4oKanLQehvaulUvoKg8A&m=wqlhnot9c-pQLdkGkccSGNpELUNUnB-wy_h0iA3PRqI&s=VjORkX5s7hrJyl8mW9Q4cfeSWF4qfTdyRjcuAiBt0y4&e= > > > > > >>>>>>>>>>>>>>>>>>>>>> * Fix all flaky tests on 3.5 - 3.4 has > little or > > > > > >> no > > > > > >>>>> flaky > > > > > >>>>>>>>>> tests > > > > > >>>>>>>>>>>>> at > > > > > >>>>>>>>>>>>>>>>>>>> all. > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> On Tue, Sep 4, 2018 at 1:48 AM, Andor Molnar > > > > > >>>>>>>>>>>>>>>>>>>> <[email protected]> > > > > > >>>>>>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> Thanks Maoling! That would be huge help, I > > > > > >>> appreciate > > > > > >>>>>> it. > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> Andor > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>> > > > > > >> > > > > > > > > > > -- > > > > > > > > > > > > -- Enrico Olivelli > > > > > > > >
