Hello, Valentin. > Nikolay, Maxim, are you OK with this route?
-1 to have another repo for Ignite3 development. > 13 нояб. 2020 г., в 03:04, Valentin Kulichenko > <valentin.kuliche...@gmail.com> написал(а): > > Folks, > > We already have multiple IEPs for Ignite 3.0, and as far as I know, there are > contributors that would like to work on them (or probably already started). > That said, we should make a decision as soon as possible. > > At this point, it doesn't seem that there are any strong objections to the > technical side of things. So I would suggest the following: > > 1. Proceed with Alexey's approach to the development process, as it seems to > be the best (in my opinion - the only) way to address all the technical > concerns and issues expressed in the thread. We'll start by creating a new > repo and a new TC project. > 2. Start a separate discussion around transparency. If there are any changes > we need to make to our contributor guidelines, I am happy to talk them > through, but I don't think it's reasonable to delay feature development > because of this. In the short term, I will make sure that everything that > happens within the new repo is as open to the community as possible. > > Nikolay, Maxim, are you OK with this route? > > -Val > > On Tue, Nov 10, 2020 at 4:55 PM Valentin Kulichenko > <valentin.kuliche...@gmail.com> wrote: > Maxim, > > 2.x and 3.x will have to coexist for some time - I don't see how we can avoid > this considering the set of proposed changes. That said, we effectively will > need to have two "masters" - one for each major version. Master for 3.x can > technically be a branch in the existing repo, but having a separate repo > seems cleaner, simply because it will not be a "branch" in the traditional > sense. > > Note that the new repo will still be under the Apache org, with the same set > of committers, managed by the community, etc. All the development happening > for 3.0 must follow the rules that we currently have (if anything, it's an > opportunity to improve those rules). > > As I said during the call on Friday, I strongly believe that if there is a > transparency issue, it will exist regardless of the approach we choose for > 3.0. If community members develop without IEPs or public discussions, this > will happen for both 2.x and 3.x unless we address this separately. I don't > see how this is related to Alexey's suggestion, which targets *technical* > issues with the product more than anything else. This a way to achieve better > modularity, introduce better coverage with unit tests, reduce conflicts > during development, etc. > > Coming back to transparency, let's identify the issues and fix them. It > probably makes sense to have a separate discussion on this topic. > > -Val > > On Tue, Nov 10, 2020 at 1:05 PM Maxim Muzafarov <mmu...@apache.org> wrote: > Sergey, > > > Your summary makes sense to me. > > However, how we come up from *Development transparency* to *create a > separate public repository dedicated for 3.0*? For me *development > transparency* is about making changes in the master branch. These > changes will definitely be seen by all the Ignite developers. > > A dedicated public repository is technically public and visible for > everyone, but it allows development without IEPs, without public > discussion (since all the code changes are not related to the master > branch) it also allows a large number of assumptions and deviations > (like code-style violations). It also not about *development > transparency* since developers which are working on 3.0 is only a > subset of all Ignite developers which may continue working on 2.x. For > me, this would be a huge step backwards. > > Ignite veterans should remember how long the branch stabilization took > for the 2.x version with the PDS. > > > I think each breaking change should be passed through the master branch. > > On Tue, 10 Nov 2020 at 22:18, Alexei Scherbakov > <alexey.scherbak...@gmail.com> wrote: > > > > Makes sense to me. > > > > вт, 10 нояб. 2020 г. в 18:47, Sergey Chugunov <sergey.chugu...@gmail.com>: > > > > > Igniters, > > > > > > I thought over Friday meeting ideas and concerns and summarized them in > > > these three points: > > > > > > > > > 1. *Components design unification approach.* New proposed components > > > will be developed by different contributors, but they need to be > > > unified > > > and should integrate with each other easily. To ensure that I suggest > > > calling an architecture group that will create design guidelines for > > > all > > > components and high-level overview of overall architecture. How code is > > > split into components, what are component boundaries, how component > > > lifecycle works and what are its interfaces - all these and other > > > questions > > > should be covered. > > > > > > 2. *Scope management.* Apache 3.0 should be implemented within a > > > reasonable time, so we need some procedure to decide whether a > > > particular > > > feature should be dropped from the scope of 3.0 and postponed to 3.1 > > > release. To do so I suggest to range all features by two parameters: > > > criticality for 3.0 and amount of breaking changes. 3.0 scope should > > > include features of high criticality AND features with a big amount of > > > breaking changes. All other features can be made optional. > > > > > > 3. *Development transparency.* Development of all components should be > > > made as transparent for everyone as possible. Any contributor should be > > > able to look over any component at any stage of development. To achieve > > > this I suggest to create a separate public repository dedicated for 3.0 > > > development. It will make the code available for everyone but when > > > development of 3.0 is done we won't loose any stars of our current > > > repository as we merge dev repo into main one and drop dev. > > > > > > Do these ideas make sense to you? Are there any concerns not covered by > > > these suggestions? > > > > > > On Fri, Nov 6, 2020 at 7:36 PM Kseniya Romanova <romanova.ks....@gmail.com > > > > > > > wrote: > > > > > > > Here are the slides from Alexey Goncharuk. Let's think this over and > > > > continue on Monday: > > > > > > > > > > > https://go.gridgain.com/rs/491-TWR-806/images/Ignite_3_Plans_and_development_process.pdf > > > > > > > > чт, 5 нояб. 2020 г. в 11:13, Anton Vinogradov <a...@apache.org>: > > > > > > > > > Folks, > > > > > > > > > > Should we perform cleanup work before (r)evolutional changes? > > > > > My huge proposal is to get rid of things which we don't need anyway > > > > > - local caches, > > > > > - strange tx modes, > > > > > - code overcomplexity because of RollingUpgrade feature never attended > > > at > > > > > AI, > > > > > - etc, > > > > > before choosing the way. > > > > > > > > > > On Tue, Nov 3, 2020 at 3:31 PM Valentin Kulichenko < > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > Ksenia, thanks for scheduling this on such short notice! > > > > > > > > > > > > As for the original topic, I do support Alexey's idea. We're not > > > going > > > > to > > > > > > rewrite anything from scratch, as most of the components are going > > > > > > to > > > > be > > > > > > moved as-is or with minimal modifications. However, the changes that > > > > are > > > > > > proposed imply serious rework of the core parts of the code, which > > > are > > > > > not > > > > > > properly decoupled from each other and from other parts. This makes > > > the > > > > > > incremental approach borderline impossible. Developing in a new > > > > > > repo, > > > > > > however, addresses this concern. As a bonus, we can also refactor > > > > > > the > > > > > code, > > > > > > introduce better decoupling, get rid of kernel context, and develop > > > > unit > > > > > > tests (finally!). > > > > > > > > > > > > Basically, this proposal only affects the *process*, not the set of > > > > > changes > > > > > > we had discussed before. Ignite 3.0 is our unique chance to make > > > things > > > > > > right. > > > > > > > > > > > > -Val > > > > > > > > > > > > On Tue, Nov 3, 2020 at 3:06 AM Kseniya Romanova < > > > > > romanova.ks....@gmail.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Pavel, all the interesting points will be anyway published here in > > > > > > English > > > > > > > (as the principal "if it's not on devlist it doesn't happened" is > > > > still > > > > > > > relevant). This is just a quick call for a group of developers. > > > Later > > > > > we > > > > > > > can do a separate presentation of idea and discussion in English > > > > > > > as > > > > we > > > > > > did > > > > > > > for the Ignite 3.0 draft of changes. > > > > > > > > > > > > > > вт, 3 нояб. 2020 г. в 13:52, Pavel Tupitsyn <ptupit...@apache.org > > > >: > > > > > > > > > > > > > > > Kseniya, > > > > > > > > > > > > > > > > Thanks for scheduling this call. > > > > > > > > Do you think we can switch to English if non-Russian speaking > > > > > community > > > > > > > > members decide to join? > > > > > > > > > > > > > > > > On Tue, Nov 3, 2020 at 1:32 PM Kseniya Romanova < > > > > > > > romanova.ks....@gmail.com > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Let's do this community discussion open. Here's the link on > > > zoom > > > > > call > > > > > > > in > > > > > > > > > Russian for Friday 6 PM: > > > > > > > > > > > > > > https://www.meetup.com/Moscow-Apache-Ignite-Meetup/events/274360378/ > > > > > > > > > > > > > > > > > > вт, 3 нояб. 2020 г. в 12:49, Nikolay Izhikov < > > > > nizhi...@apache.org > > > > > >: > > > > > > > > > > > > > > > > > > > Time works for me. > > > > > > > > > > > > > > > > > > > > > 3 нояб. 2020 г., в 12:40, Alexey Goncharuk < > > > > > > > > alexey.goncha...@gmail.com > > > > > > > > > > > > > > > > > > > > написал(а): > > > > > > > > > > > > > > > > > > > > > > Nikolay, > > > > > > > > > > > > > > > > > > > > > > I am up for the call. I will try to explain my reasoning > > > > > > > > > > > in > > > > > > greater > > > > > > > > > > detail > > > > > > > > > > > and will be glad to hear the concerns. Will this Friday, > > > Nov > > > > > 6th, > > > > > > > > work? > > > > > > > > > > > > > > > > > > > > > > вт, 3 нояб. 2020 г. в 10:09, Nikolay Izhikov < > > > > > > nizhi...@apache.org > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > >> Igniters, should we have a call for this topic? > > > > > > > > > > >> > > > > > > > > > > >>> 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn < > > > > > ptupit...@apache.org > > > > > > > > > > > > > > > > > >> написал(а): > > > > > > > > > > >>> > > > > > > > > > > >>>> not intend to rewrite everything from scratch > > > > > > > > > > >>> > > > > > > > > > > >>>> Every single test from Ignite 2.x should be moved to > > > > Ignite > > > > > 3 > > > > > > > > > > >>>> regardless of how we choose to proceed. > > > > > > > > > > >>> > > > > > > > > > > >>> Alexey, thank you for the explanation, this addresses > > > > > > > > > > >>> all > > > > of > > > > > my > > > > > > > > > > concerns. > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > > >>> On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov < > > > > > > > > > > >> andrey.mashen...@gmail.com> > > > > > > > > > > >>> wrote: > > > > > > > > > > >>> > > > > > > > > > > >>>> Hi, Igniters. > > > > > > > > > > >>>> > > > > > > > > > > >>>> * AFAIU, we need a new repo if we want to apply > > > different > > > > > > > > > restrictions > > > > > > > > > > >> to > > > > > > > > > > >>>> pull requests, > > > > > > > > > > >>>> otherwise I see no difference for myself. > > > > > > > > > > >>>> E.g. make static analysis (do we have?), compile, > > > styles, > > > > > and > > > > > > > > > javadoc > > > > > > > > > > >>>> checks mandatory. > > > > > > > > > > >>>> > > > > > > > > > > >>>> I think that relaxed requirements here will lead to bad > > > > > > product > > > > > > > > > > quality. > > > > > > > > > > >>>> > > > > > > > > > > >>>> * Agree with Pavel, we should 'keep' integrations tests > > > > > > somehow. > > > > > > > > > > >>>> During active development tests will be broken most of > > > > time, > > > > > > so, > > > > > > > > > > >>>> I'd port them e.g. suite-by-suite once we will have a > > > > stable > > > > > > and > > > > > > > > > > >> featured > > > > > > > > > > >>>> environment to run them and of course make test's code > > > > clear > > > > > > and > > > > > > > > > avoid > > > > > > > > > > >>>> bad/non-relevant ones. > > > > > > > > > > >>>> > > > > > > > > > > >>>> * I like bottom-up approach. > > > > > > > > > > >>>> With it we could make a better framework. I mean clear > > > > > > component > > > > > > > > > > >> lifecycle, > > > > > > > > > > >>>> component wiring mechanics, general methods to approach > > > > core > > > > > > > > > > components > > > > > > > > > > >>>> such as exchange/communication > > > > > > > > > > >>>> to avoid code mess like we have in ExchangeFuture with > > > all > > > > > > these > > > > > > > > > > custom > > > > > > > > > > >>>> callbacks for each component, interfaces like > > > > > > > > > > >>>> PartitionsExchangeAware, IgniteChangeGlobalStateSupport > > > > and > > > > > > > > > > >>>> a pack of > > > > > > > > > > >> > > > > > start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected > > > > > > > > > > >>>> and so on in various unexpected places. > > > > > > > > > > >>>> Hope, we will be able to port most of the good code to > > > the > > > > > new > > > > > > > > > > framework > > > > > > > > > > >>>> version. > > > > > > > > > > >>>> > > > > > > > > > > >>>> > > > > > > > > > > >>>> > > > > > > > > > > >>>> On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk < > > > > > > > > > > >>>> alexey.goncha...@gmail.com> > > > > > > > > > > >>>> wrote: > > > > > > > > > > >>>> > > > > > > > > > > >>>>> Nikolay, Pavel, > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> Thanks for the feedback! First of all, I wanted to > > > stress > > > > > > that > > > > > > > I > > > > > > > > do > > > > > > > > > > not > > > > > > > > > > >>>>> intend to rewrite everything from scratch (I never > > > > > > > > > > >>>>> used > > > > > this > > > > > > > > > phrase). > > > > > > > > > > >>>> There > > > > > > > > > > >>>>> are significant parts of code that would be moved with > > > > > > minimal > > > > > > > > > > >>>>> modifications. Second, I never said that we will get > > > rid > > > > of > > > > > > the > > > > > > > > old > > > > > > > > > > >> tests > > > > > > > > > > >>>>> codebase. Every single test from Ignite 2.x should be > > > > moved > > > > > > to > > > > > > > > > > Ignite 3 > > > > > > > > > > >>>>> regardless of how we choose to proceed. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> My point is that for some parts of the code a clean > > > > > bottom-up > > > > > > > > > > >>>>> implementation will be cheaper in many ways. Let me > > > give > > > > > you > > > > > > a > > > > > > > > few > > > > > > > > > > >>>> concrete > > > > > > > > > > >>>>> examples: > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> - I think no one can object that we need a cleanly > > > > > separated > > > > > > > > > > >>>> persistence > > > > > > > > > > >>>>> layer for Ignite. There is a very raw draft IEP for > > > this > > > > > > > > already. > > > > > > > > > On > > > > > > > > > > >>>> the > > > > > > > > > > >>>>> other hand, I think we also can agree that we need a > > > > > > > split-brain > > > > > > > > > > >>>>> resistant > > > > > > > > > > >>>>> replication protocol for caches. There is also an IEP > > > > for > > > > > > > this. > > > > > > > > > > >>>> Neither > > > > > > > > > > >>>>> of > > > > > > > > > > >>>>> the changes is a good fit for 2.x because they are > > > > likely > > > > > to > > > > > > > > > > >> introduce > > > > > > > > > > >>>>> breaking changes in the persistence layer, > > > configuration > > > > > and > > > > > > > > > > >> behavior. > > > > > > > > > > >>>>> Additionally, these components are now tightly > > > coupled, > > > > so > > > > > > > there > > > > > > > > > is > > > > > > > > > > >> no > > > > > > > > > > >>>>> way > > > > > > > > > > >>>>> these two changes can be implemented in parallel and > > > > then > > > > > > > merged > > > > > > > > > > >>>>> together > > > > > > > > > > >>>>> easily. So what we will end up with is having to > > > > implement > > > > > > > these > > > > > > > > > > >>>> changes > > > > > > > > > > >>>>> sequentially, fixing all existing tests twice, and > > > > > > essentially > > > > > > > > > > >>>> throwing > > > > > > > > > > >>>>> away half of the work done because the other part of > > > the > > > > > > > change > > > > > > > > is > > > > > > > > > > >>>>> re-implemented > > > > > > > > > > >>>>> - Similar example goes with getting rid of > > > > > > > IgniteInternalFuture > > > > > > > > > and > > > > > > > > > > >>>>> replacing it with CompletableFuture, and any other > > > > change > > > > > > that > > > > > > > > > > >> touches > > > > > > > > > > >>>>> the > > > > > > > > > > >>>>> asynchronous part of the code. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> Third, I do not see how this choice affects the UX of > > > > > Ignite. > > > > > > > The > > > > > > > > > end > > > > > > > > > > >>>> user > > > > > > > > > > >>>>> experience must be fixed in the IEP regardless of the > > > > > > > development > > > > > > > > > > >> process > > > > > > > > > > >>>>> and the fact that we have gaps in this area in Ignite > > > 2.x > > > > > > just > > > > > > > > > > confirms > > > > > > > > > > >>>>> that. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> Pavel, agree that a repo/branch is a technicality, I > > > > guess > > > > > if > > > > > > > > > > >>>> reformulate, > > > > > > > > > > >>>>> my point is that we might agree to have a single > > > > > development > > > > > > > > master > > > > > > > > > > >>>> branch > > > > > > > > > > >>>>> with 'disassembled' end-to-end functionality for some > > > > > period > > > > > > of > > > > > > > > > time > > > > > > > > > > to > > > > > > > > > > >>>>> speed up development, and re-assemble the core > > > > > > > > > > >>>>> features > > > > > after > > > > > > > > > having > > > > > > > > > > >>>>> submodules tested independently. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> Nikolay, > > > > > > > > > > >>>>>> We have many features that have to evolve. > > > > > > > > > > >>>>>> Snapshots, rebalance, tooling, tracing, zookeeper > > > > support, > > > > > > > etc. > > > > > > > > > > >>>>> This is not very specific. In the end, resources are > > > > > limited > > > > > > > and > > > > > > > > we > > > > > > > > > > >> will > > > > > > > > > > >>>>> not be able to drive both tracks simultaneously, > > > > especially > > > > > > > > after a > > > > > > > > > > >>>> couple > > > > > > > > > > >>>>> of features having been implemented for Ignite 3.0. If > > > > > there > > > > > > > are > > > > > > > > > > indeed > > > > > > > > > > >>>>> some major changes that we want to do in Ignite 2.x > > > > instead > > > > > > of > > > > > > > > > > putting > > > > > > > > > > >>>>> effort into 3.0 - let's discuss them. I am just not > > > aware > > > > > of > > > > > > > any, > > > > > > > > > > >> that's > > > > > > > > > > >>>>> why I am eager to move forward with Ignite 3.0. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>>> We have bugs and issues that can be fixed in 2.x > > > without > > > > > > > > breaking > > > > > > > > > > >>>> backward > > > > > > > > > > >>>>> compatibility. > > > > > > > > > > >>>>>> We have many users who are happy with the 2.x with > > > > > > > > > > >>>>>> all > > > > > it’s > > > > > > > > > issues. > > > > > > > > > > >>>>> These changes will be covered by end-to-end tests and > > > > > > migrated > > > > > > > to > > > > > > > > > > >> Ignite > > > > > > > > > > >>>>> 3.0, so I see no issues here. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> Finally, Anton & Nikolay > > > > > > > > > > >>>>> I do not have an estimate for this simply because the > > > > > > activity > > > > > > > is > > > > > > > > > > >>>>> community-driven and it depends on the number of > > > > > > > > > > >>>>> people > > > > > > willing > > > > > > > > to > > > > > > > > > > >>>>> contribute. With the current pace, I would hope to > > > > > > > > > > >>>>> have > > > > an > > > > > RC > > > > > > > of > > > > > > > > > > Ignite > > > > > > > > > > >>>> 3.0 > > > > > > > > > > >>>>> to be ready by the end of 2021. My gut feeling is that > > > by > > > > > > > moving > > > > > > > > > with > > > > > > > > > > >>>>> incremental changes, we will not be able to implement > > > > even > > > > > > half > > > > > > > > of > > > > > > > > > > the > > > > > > > > > > >>>>> wishlist by that time. > > > > > > > > > > >>>>> I doubt that releasing several major releases with > > > > breaking > > > > > > > > changes > > > > > > > > > > >> will > > > > > > > > > > >>>>> make Ignite users happy either because each upgrade > > > will > > > > > cost > > > > > > > > > Ignite > > > > > > > > > > >>>> users > > > > > > > > > > >>>>> money, so the fewer major versions we release, the > > > > better. > > > > > > Thus > > > > > > > > my > > > > > > > > > > wish > > > > > > > > > > >>>> to > > > > > > > > > > >>>>> include all breaking changes in one release. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> I'll be now quiet for a while, let's see what other > > > > > community > > > > > > > > > members > > > > > > > > > > >>>>> think. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> пн, 2 нояб. 2020 г. в 15:52, Pavel Tupitsyn < > > > > > > > > ptupit...@apache.org > > > > > > > > > >: > > > > > > > > > > >>>>> > > > > > > > > > > >>>>>> 1. Rewriting from scratch is never a good idea. > > > > > > > > > > >>>>>> We don't want to follow the path of Netscape and lose > > > > all > > > > > > our > > > > > > > > > users > > > > > > > > > > >>>>>> by the time we have a working 3.0 [1] > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> 2. Not sure about new repo - seems like some pain and > > > no > > > > > > gain, > > > > > > > > > > what's > > > > > > > > > > >>>> the > > > > > > > > > > >>>>>> problem with a branch? > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> 3. We should keep existing integration tests when > > > > > possible. > > > > > > > > > > >>>>>> We have accumulated a lot of edge case knowledge over > > > > the > > > > > > > years, > > > > > > > > > > >>>>>> it is not a good idea to send all of that down the > > > > drain. > > > > > > > > > > >>>>>> Yes, integration tests are slow, but they are the > > > > > > > > > > >>>>>> most > > > > > > > valuable. > > > > > > > > > > >>>>>> I think we can move more stuff into nightly runs and > > > > have > > > > > a > > > > > > > fast > > > > > > > > > and > > > > > > > > > > >>>>> modern > > > > > > > > > > >>>>>> basic suite. > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> Alexey, you are much more familiar with the Ignite > > > core > > > > > > > codebase > > > > > > > > > > than > > > > > > > > > > >>>>> most > > > > > > > > > > >>>>>> of us, > > > > > > > > > > >>>>>> can you please explain in more detail which > > > > > > > > > > >>>>>> particular > > > > > > > feature, > > > > > > > > in > > > > > > > > > > >> your > > > > > > > > > > >>>>>> opinion, > > > > > > > > > > >>>>>> mandates this "start from scratch" approach? > > > > > > > > > > >>>>>> Is it really not possible at all to follow a less > > > > radical > > > > > > way? > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> [1] > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>> > > > > > > > > > > >>>> > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> On Mon, Nov 2, 2020 at 2:25 PM Nikolay Izhikov < > > > > > > > > > nizhi...@apache.org > > > > > > > > > > > > > > > > > > > > > >>>>>> wrote: > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>>> Hello, Alexey. > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>> I think that «rewriting from scratch» approach has a > > > > high > > > > > > > risk > > > > > > > > to > > > > > > > > > > >>>> make > > > > > > > > > > >>>>>> new > > > > > > > > > > >>>>>>> features unusable. > > > > > > > > > > >>>>>>> At the time Ignite2 was started no-one wants to do > > > bad > > > > UX > > > > > > or > > > > > > > > bad > > > > > > > > > > >>>>>> features. > > > > > > > > > > >>>>>>> Nevertheless, it happen. > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>> I think we can avoid it with the Ignite3 and > > > successors > > > > > if > > > > > > we > > > > > > > > > will > > > > > > > > > > >>>> move > > > > > > > > > > >>>>>>> step by step without keeping backward compatibility > > > > > > > > > > >>>>>>> With the step by step approach, we can focus on each > > > > > > > component > > > > > > > > > > >>>>>> separately. > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>>> What new features are we planning to implement for > > > > > Ignite > > > > > > > 2.x? > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>> We have many features that have to evolve. > > > > > > > > > > >>>>>>> Snapshots, rebalance, tooling, tracing, zookeeper > > > > > support, > > > > > > > etc. > > > > > > > > > > >>>>>>> We have bugs and issues that can be fixed in 2.x > > > > without > > > > > > > > breaking > > > > > > > > > > >>>>>> backward > > > > > > > > > > >>>>>>> compatibility. > > > > > > > > > > >>>>>>> We have many users who are happy with the 2.x with > > > all > > > > > it’s > > > > > > > > > issues. > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>>> 2 нояб. 2020 г., в 14:09, Anton Vinogradov < > > > > > a...@apache.org > > > > > > > > > > > > > > > > > >>>>> написал(а): > > > > > > > > > > >>>>>>>> > > > > > > > > > > >>>>>>>> Alexey, > > > > > > > > > > >>>>>>>> > > > > > > > > > > >>>>>>>> Do we have any estimates of how fast we'll be able > > > to > > > > > gain > > > > > > > > > > >>>>>>> production-ready > > > > > > > > > > >>>>>>>> AI 3.0 in case of a "new repo" choice? > > > > > > > > > > >>>>>>>> > > > > > > > > > > >>>>>>>> On Mon, Nov 2, 2020 at 2:01 PM Alexey Goncharuk < > > > > > > > > > > >>>>>>> alexey.goncha...@gmail.com> > > > > > > > > > > >>>>>>>> wrote: > > > > > > > > > > >>>>>>>> > > > > > > > > > > >>>>>>>>> Nikolay, > > > > > > > > > > >>>>>>>>> > > > > > > > > > > >>>>>>>>> What new features are we planning to implement for > > > > > Ignite > > > > > > > > 2.x? > > > > > > > > > I > > > > > > > > > > >>>>> think > > > > > > > > > > >>>>>>> once > > > > > > > > > > >>>>>>>>> we commence working on Ignite 3.0, we should > > > > gradually > > > > > > > cease > > > > > > > > > the > > > > > > > > > > >>>>>>> activity > > > > > > > > > > >>>>>>>>> on Ignite 2.x to mere bugfixes because such > > > parallel > > > > > > > > > development > > > > > > > > > > >>>>> will > > > > > > > > > > >>>>>> be > > > > > > > > > > >>>>>>>>> overwhelming regardless of how we choose to > > > proceed. > > > > > > > > > > >>>>>>>>> > > > > > > > > > > >>>>>>>>> пн, 2 нояб. 2020 г. в 13:38, Nikolay Izhikov < > > > > > > > > > > nizhi...@apache.org > > > > > > > > > > >>>>> : > > > > > > > > > > >>>>>>>>> > > > > > > > > > > >>>>>>>>>> To be clear: > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>>>> I would suggest creating a new repository for > > > > Ignite > > > > > > 3.0 > > > > > > > > > > >>>>> (perhaps, a > > > > > > > > > > >>>>>>>>> new > > > > > > > > > > >>>>>>>>>> clean branch, but a new repo looks nicer to me) > > > and > > > > a > > > > > > new > > > > > > > > > Ignite > > > > > > > > > > >>>>> 3.0 > > > > > > > > > > >>>>>>>>>> TeamCity project. > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>>> +1 for new Team City project. > > > > > > > > > > >>>>>>>>>> +1 for new branch for Ignite3. > > > > > > > > > > >>>>>>>>>> -1 for new repo. > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>>>> 2 нояб. 2020 г., в 13:35, Nikolay Izhikov < > > > > > > > > > > >>>> nizhikov....@gmail.com > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>>>>>> написал(а): > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>> Hello, Alexey. > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>> I think it will hurt our project more than help. > > > > > > > > > > >>>>>>>>>>> Developing new features for 2 separate branches > > > > with > > > > > > the > > > > > > > > > > >>>> different > > > > > > > > > > >>>>>>> APIs > > > > > > > > > > >>>>>>>>>> and internal structure is overwhelming > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>> Maybe we should relax a bit requirements for > > > > Ignite3? > > > > > > > > > > >>>>>>>>>>> Maybe we should move step by step and make > > > Ignite3 > > > > > with > > > > > > > new > > > > > > > > > > >>>>>>>>>> configuration than Ignite4 with new transactions, > > > > etc? > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> 2 нояб. 2020 г., в 13:14, Alexey Goncharuk < > > > > > > > > > > >>>>>>>>> alexey.goncha...@gmail.com> > > > > > > > > > > >>>>>>>>>> написал(а): > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> Igniters, > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> I wanted to pitch a rather radical idea > > > regarding > > > > > the > > > > > > > > Ignite > > > > > > > > > > >>>> 3.0 > > > > > > > > > > >>>>>>>>>>>> development which has occurred to me some time > > > > ago. > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> We already have several IEPs targeted to Ignite > > > > 3.0 > > > > > > > which > > > > > > > > > > imply > > > > > > > > > > >>>>>> major > > > > > > > > > > >>>>>>>>>>>> changes to the codebase (the change in > > > replication > > > > > > > > protocol > > > > > > > > > > and > > > > > > > > > > >>>>>> thus > > > > > > > > > > >>>>>>>>>>>> transactions, change in binary format, updated > > > > > > > > metastorage, > > > > > > > > > > >>>> etc). > > > > > > > > > > >>>>>> We > > > > > > > > > > >>>>>>>>>> also > > > > > > > > > > >>>>>>>>>>>> planned significant changes in public APIs: > > > > > > > configuration > > > > > > > > > > >>>> format > > > > > > > > > > >>>>>>>>> change, > > > > > > > > > > >>>>>>>>>>>> improvements in cache APIs, SQL APIs, > > > transaction > > > > > mode > > > > > > > > > rework. > > > > > > > > > > >>>>> The > > > > > > > > > > >>>>>>>>>> wishlist > > > > > > > > > > >>>>>>>>>>>> of changes for Ignite 3.0 is huge. > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> So, I was wondering whether it makes sense to > > > try > > > > to > > > > > > > > change > > > > > > > > > > the > > > > > > > > > > >>>>> old > > > > > > > > > > >>>>>>>>>>>> codebase, or start with a new baseline and move > > > > old > > > > > > > pieces > > > > > > > > > of > > > > > > > > > > >>>>> code > > > > > > > > > > >>>>>>>>> that > > > > > > > > > > >>>>>>>>>> do > > > > > > > > > > >>>>>>>>>>>> not require significant rework. Personally, I > > > > would > > > > > go > > > > > > > > with > > > > > > > > > > the > > > > > > > > > > >>>>>>> second > > > > > > > > > > >>>>>>>>>>>> option for the following reasons: > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> - We have a chance to shift the development > > > > paradigm > > > > > > in > > > > > > > > the > > > > > > > > > > >>>>> project > > > > > > > > > > >>>>>>>>> and > > > > > > > > > > >>>>>>>>>>>> introduce the practice of true unit-tests. In > > > the > > > > > new > > > > > > > > > baseline > > > > > > > > > > >>>> at > > > > > > > > > > >>>>>> the > > > > > > > > > > >>>>>>>>>>>> beginning there will be no ability to run an > > > > > > end-to-end > > > > > > > > > > >>>> scenario, > > > > > > > > > > >>>>>>>>> thus > > > > > > > > > > >>>>>>>>>> we > > > > > > > > > > >>>>>>>>>>>> will be forced to write unit-tests. So far, > > > > > > > > > > >>>>>>>>>>>> such > > > > > > > practice > > > > > > > > > was > > > > > > > > > > >>>>> hard > > > > > > > > > > >>>>>> to > > > > > > > > > > >>>>>>>>>>>> implement because of tight coupling between > > > Ignite > > > > > > > > > components > > > > > > > > > > >>>> and > > > > > > > > > > >>>>>>>>>> inability > > > > > > > > > > >>>>>>>>>>>> to instantiate components without an instance > > > > > > > > > > >>>>>>>>>>>> of > > > > > > > > > > KernalContext. > > > > > > > > > > >>>>> For > > > > > > > > > > >>>>>>>>>>>> example, we should be able to thoroughly test > > > > > internal > > > > > > > > > > >>>>> primitives, > > > > > > > > > > >>>>>>>>>> such as > > > > > > > > > > >>>>>>>>>>>> replication protocol (without actual > > > > communication), > > > > > > > > > > >>>> distributed > > > > > > > > > > >>>>>>>>>>>> metastorage contracts, persistence layer, etc. > > > > > > > > > > >>>>>>>>>>>> - We will significantly reduce the development > > > > cycle > > > > > > in > > > > > > > > the > > > > > > > > > > >>>>>> beginning > > > > > > > > > > >>>>>>>>>>>> (right now the RunAll takes two hours of > > > > > astronomical > > > > > > > time > > > > > > > > > > with > > > > > > > > > > >>>>>> empty > > > > > > > > > > >>>>>>>>>> TC; > > > > > > > > > > >>>>>>>>>>>> in the new approach developer will be able to > > > run > > > > > ALL > > > > > > > > tests > > > > > > > > > > >>>>> locally > > > > > > > > > > >>>>>>>>> in > > > > > > > > > > >>>>>>>>>> a > > > > > > > > > > >>>>>>>>>>>> matter of minutes) > > > > > > > > > > >>>>>>>>>>>> - We can get rid of TC bot and enforce green TC > > > by > > > > > > > > > integrating > > > > > > > > > > >>>> TC > > > > > > > > > > >>>>>>>>> build > > > > > > > > > > >>>>>>>>>>>> results with GitHub PRs (the same way Travis is > > > > > > > currently > > > > > > > > > > >>>>>> integrated > > > > > > > > > > >>>>>>>>>> to PR > > > > > > > > > > >>>>>>>>>>>> check). We should restrict PR merge without a > > > > > > > > > > >>>>>>>>>>>> TC > > > > > check > > > > > > > > > > >>>>>>>>>>>> - We will still have to re-write all tests, but > > > > only > > > > > > > once. > > > > > > > > > If > > > > > > > > > > >>>> we > > > > > > > > > > >>>>>> try > > > > > > > > > > >>>>>>>>> to > > > > > > > > > > >>>>>>>>>>>> modify the old codebase, we would need to > > > > > > > > > > >>>>>>>>>>>> modify > > > > all > > > > > > the > > > > > > > > > tests > > > > > > > > > > >>>>> for > > > > > > > > > > >>>>>>>>>> every > > > > > > > > > > >>>>>>>>>>>> major change (public API change, configuration > > > > > change) > > > > > > > > > > >>>>>>>>>>>> - We will have fewer conflicts when working > > > > > together. > > > > > > > For > > > > > > > > > > >>>>> example, > > > > > > > > > > >>>>>> I > > > > > > > > > > >>>>>>>>>>>> cannot imagine how one would merge two changes > > > of > > > > > > > getting > > > > > > > > > rid > > > > > > > > > > >>>> of > > > > > > > > > > >>>>>>>>>>>> IgniteFuture and changes in replication > > > protocol, > > > > > for > > > > > > > > > example > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> Technically, I would suggest creating a new > > > > > repository > > > > > > > for > > > > > > > > > > >>>> Ignite > > > > > > > > > > >>>>>> 3.0 > > > > > > > > > > >>>>>>>>>>>> (perhaps, a new clean branch, but a new repo > > > looks > > > > > > nicer > > > > > > > > to > > > > > > > > > > me) > > > > > > > > > > >>>>>> and a > > > > > > > > > > >>>>>>>>>> new > > > > > > > > > > >>>>>>>>>>>> Ignite 3.0 TeamCity project. > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> While it may seem quite radical, I do believe > > > that > > > > > > this > > > > > > > > > > >>>> approach > > > > > > > > > > >>>>>> will > > > > > > > > > > >>>>>>>>>> give > > > > > > > > > > >>>>>>>>>>>> us more benefits than trying to make such major > > > > > > changes > > > > > > > in > > > > > > > > > the > > > > > > > > > > >>>>>>>>> existing > > > > > > > > > > >>>>>>>>>>>> codebase. If needed, let's schedule a discord > > > chat > > > > > > like > > > > > > > > > before > > > > > > > > > > >>>> to > > > > > > > > > > >>>>>>>>>> discuss > > > > > > > > > > >>>>>>>>>>>> this. > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> WDYT? > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>> > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>> > > > > > > > > > > >>>> > > > > > > > > > > >>>> > > > > > > > > > > >>>> -- > > > > > > > > > > >>>> Best regards, > > > > > > > > > > >>>> Andrey V. Mashenkov > > > > > > > > > > >>>> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > Alexei Scherbakov
