I agree main needs to target 2.x, that is where most effort should be focused, and development there can be cherry picked / backported to a 1.x branch. The inverse approach creates too much of a moving target for 2.x.
Joe, I do not see anything else needed for cutting 1.20. ________________________________ From: Joe Witt <joe.w...@gmail.com> Sent: Wednesday, January 11, 2023 7:05:59 PM To: dev@nifi.apache.org <dev@nifi.apache.org> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0 Hello We dont have a release schedule and never really have. What we do have is 7+ years of track record which shows who and what we do. We have done a major release change before. We release every couple of months or so. This is very manageable and the stated goals are scoped as such. We have declared what our goals are for 2.0 and now it is time to simply make it happen. We will keep 1.x moving along in the meantime but one of these is going to get the ‘most’ energy as that is ultimately how all this works. Whatever is our ‘main’ line is where all PRs go by default and where people work by default. The burden of maintaining these lines is very real and will provide plenty of motivation to move 2.0 along rapidly. The true notion of business as usual is where PRs land and who does the work to review and merge them. The commentary around production usage worries suggests ideas or views that differ from the discussed, documented and now voted upon release goals. Someone starting on 2.0 should enjoy stability similar to a NiFi 1.20 release. Existing users will be provided tooling/automation/docs to move from 1.x to 2.x. Milestones may serve as a valuable tool for us. We can use them as we vet out migration tooling we put on the 1.x line. Do we need anything else for 1.20? Thanks On Wed, Jan 11, 2023 at 3:42 PM Adam Taft <a...@adamtaft.com> wrote: > This is really insightful and spot on ... > > Kevin wrote: > > Good migration tooling will take a while to develop and test, and the > core > > contributors to that effort may not have sufficient variety of flows to > > evaluate when the migration tools are "done" for the majority of the > > community to have success upgrading to 2.x. A milestone release would > allow > > us get more feedback on migration over a longer period than the vote > window > > of an RC candidate. > > It's exactly this case, that an early 2.0 release might not have had time > to fully work its way through existing production deployments, that's > concerning. The pace and voting of an "RC" is much too short to get any > quality feedback from users in the field. > > I think it's really smart to consider the "Milestone" release approach > here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time > for feedback. We can put these milestones on a calendar, as needed, so that > feedback is required some 'x' number of weeks/months after each milestone. > > And to this end, I'd personally rather see us keep the 'main' branch > current with the 1.x line _until_ we're ready and are satisfied with the > end goals of the 2.0 release objectives. When the milestone releases have > been completed and there's a comfort level with the 2.x line, it's at the > point we'd isolate the 1.x line into its own branch and switch main over to > the 2.x line. > > This is an attractive way of: > a) continuing business-as-usual with the 1.x line > b) making headway on the 2.x release milestones > c) giving adequate time for feedback against the 2.0 milestones coming from > the field > > I don't mind the known-unknowns. But it's really the unknown-unknowns that > are going to drive a delay in the 2.0 release. I think it's smart to be > able to get some of the unknowns ironed out before we finalize the 2.0 > release ceremony. The milestone approach really helps with that. > > /Adam > > On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kdo...@apache.org> wrote: > > > Sorry, Joe, I was not clear, and to be honest the two thoughts are > somewhat > > unrelated in my mind too :) > > > > I agree that good migration tooling is key. Otherwise, we risk users > > staying on 1.x or creating a schism of 1.x and 2.x users. > > > > Good migration tooling will take a while to develop and test, and the > core > > contributors to that effort may not have sufficient variety of flows to > > evaluate when the migration tools are "done" for the majority of the > > community to have success upgrading to 2.x. A milestone release would > allow > > us get more feedback on migration over a longer period than the vote > window > > of an RC candidate. > > > > Perhaps we could continue to release from the 1.x line (including minor > > releases with some features) until we are ready to drop the "milestone" > > qualifier from 2.0.0, and only then put 1.x into maintenance-only status. > > It would be the same proposal to move main to target 2.0.0-M1, but relax > > restrictions for what can land on the 1.x branch and be open to a 1.21, > > 1.22, etc. if 2.0.0 work takes longer than anticipated. For example, > maybe > > we would be open to landing new/backported processors on the 1.x branch, > > but not core framework features or API changes. > > > > This might not be necessary, but I think it is fair that saying "no new > > features on 1.x" and also "no new features in 2.0.0" puts the project in > a > > rough place if 2.0.0 takes longer than a few months, so if we go that > > route, we need to commit to a quick release of 2.0.0 that most users can > > move to easily. > > > > Thanks, > > Kevin > > > > On Jan 11, 2023 at 12:32:46, Joe Witt <joe.w...@gmail.com> wrote: > > > > > Kevin, > > > > > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are > > prior > > > to us officially considering it 2.0/stable. > > > > > > That said - the migration tooling will be key to provide as we need to > > make > > > the bridge as solid and stable as possible to help someone move from > 1.x > > to > > > 2.x. I dont know how related these two concepts (milestone releases > and > > > 1.x to 2.x ease really are). > > > > > > Thanks > > > > > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kdo...@apache.org> > wrote: > > > > > > [hit the wrong keyboard shortcut, here is the rest of my thoughts] > > > > > > > > > On this point from David: > > > > > > > > > We may need to have a longer release candidate period, or more > > incremental > > > > > > > fix releases > > > > > > > for the initial 2.0.0 release train, but I do not expect delaying a > > 2.0.0 > > > > > > > release for new features, as that is not part of the release goals. > > > > > > > > > > > > > > > > Would the community benefit from one or more milestone releases of 2.0, > > to > > > > > > allow for a wider group to run / live on the proposed 2.0 prior to > > > > > > releasing it as "stable"? I know we've never done a milestone release > in > > > > > > the past, and I'm not sure what ASF guidance is on the topic, but if it > > > > > > could be beneficial we could look into that. > > > > > > > > > Cheers, > > > > > > Kevin > > > > > > > > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kdo...@apache.org> wrote: > > > > > > > > > > I think this is a good, practical discussion. > > > > > > > > > > > > > > On the one hand, we can't put off 2.x any longer as we really need to > > > > > > > updated the minimum required Java to 11. Updating main development to > > > > > > > target 2.x feels like a good way drive progress on that along with > the > > > > > > > other 2.0 goals. > > > > > > > > > > > > > > On the other hand, the concerns are valid: moving all development to > > > > > > > target 2.x puts the project at risk if we cannot release 2.0.0 on a > > > > > > > reasonable timeline. The restricted scope of 2.0 helps, but this > stated > > > > > > > release goal feels risky to me: > > > > > > > > > > > > > > Implement Migration Tools for Upgrading Flows > > > > > > > > > > > > > > > > > > > > > - Implement automated migration where possible to remap properties > > and > > > > > > > features > > > > > > > - Implement migration tools for manual conversion of XML > > Templates > > > > > > > to JSON Flow Definitions > > > > > > > - Create documentation for manual steps necessary where > > > > > > > programmatic migration cannot be implemented > > > > > > > - NiFi 2.0 should be capable of starting with ghosted > components > > > > > > > for removed Processors or Controller Services > > > > > > > > > > > > > > > > > > > > > Removing deprecated components should be fairly straightforward and > > > > > > quick, > > > > > > > but automating and documenting migration is a large effort. > > > > > > > > > > > > > > On this po > > > > > > > > > > > > > > > > > > > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bbe...@gmail.com> wrote: > > > > > > > > > > > > > >> The plan as I understand it is not to diverge and create separate > > > > > > feature > > > > > > >> development on the 1.x line, so I would expect all PRs to continue > to > > be > > > > > > >> submitted only to main. We would release 1.x as needed with major > bug > > > > > > >> fixes > > > > > > >> or critical security updates, and these would be cherry-picked > and/or > > > > > > >> backported as necessary, mostly without the need for PRs, the same > as > > we > > > > > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) back > > to a > > > > > > >> maintenance line like (1.19.x). For precedent, we followed this same > > > > > > >> approach going from the 0.x line to 1.0.0 and there wasn't any major > > > > > > >> issue. > > > > > > >> > > > > > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler < > ottobackwa...@gmail.com> > > > > > > >> wrote: > > > > > > >> > > > > > > >> It was also mentioned in another thread that we need to have > > agreement > > > > > > on > > > > > > >> > > > > > > >> our explicit strategy and support for 1.x going forward, did that > > > > > > happen? > > > > > > >> > > > > > > >> > > > > > > >> From: Otto Fowler <ottobackwa...@gmail.com> < > ottobackwa...@gmail.com> > > > > > > >> > > > > > > >> Reply: Otto Fowler <ottobackwa...@gmail.com> < > ottobackwa...@gmail.com > > > > > > > > > >> > > > > > > >> Date: January 10, 2023 at 07:02:34 > > > > > > >> > > > > > > >> To: dev@nifi.apache.org <dev@nifi.apache.org> <dev@nifi.apache.org> > > > > > > >> > > > > > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0 > > > > > > >> > > > > > > >> > > > > > > >> There needs to be an update to the contributing guide as to how to > > > > > > submit > > > > > > >> > > > > > > >> PRs to 1.x or 2.x etc. > > > > > > >> > > > > > > >> > > > > > > >> From: Joe Witt <joew...@apache.org> <joew...@apache.org> > > > > > > >> > > > > > > >> Reply: dev@nifi.apache.org <dev@nifi.apache.org> < > dev@nifi.apache.org > > > > > > > > > >> > > > > > > >> Date: January 9, 2023 at 15:53:16 > > > > > > >> > > > > > > >> To: dev@nifi.apache.org <dev@nifi.apache.org> <dev@nifi.apache.org> > > > > > > >> > > > > > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0 > > > > > > >> > > > > > > >> > > > > > > >> Team, > > > > > > >> > > > > > > >> > > > > > > >> As David mentioned in [1] following a successful NiFi 2.0 release > goal > > > > > > >> > > > > > > >> planning - we are now going to start moving the 'main' line to be > the > > > > > > NiFi > > > > > > >> > > > > > > >> 2.0 line which will allow for the key work to take place. We will > also > > > > > > >> > > > > > > >> move niFi 1.x to its appropriate support line. > > > > > > >> > > > > > > >> > > > > > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and > we > > > > > > have > > > > > > >> > > > > > > >> work in there including security items so it is time to make a > > release. > > > > > > >> > > > > > > >> The intent then is to initiate 1.20 and immediate after that change > > > > > > 'main' > > > > > > >> > > > > > > >> to 2.0. > > > > > > >> > > > > > > >> > > > > > > >> Going forward then all work on the 1.x line should be focused on > > > > > > >> > > > > > > >> maintaining existing features, dependencies, and helping 1.x users > > > > > > migrate > > > > > > >> > > > > > > >> to the 2.x line. Otherwise, new feature work will happen on 'main' > as > > it > > > > > > >> > > > > > > >> normally does and will come out in the NiFi 2.x release line. > > > > > > >> > > > > > > >> > > > > > > >> Please flag key outstanding items as we narrow down the release > > > > > > candidate > > > > > > >> > > > > > > >> for NiFi 1.20. > > > > > > >> > > > > > > >> > > > > > > >> Thanks > > > > > > >> > > > > > > >> Joe > > > > > > >> > > > > > > >> > > > > > > >> [1] > https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > > > > > >
