Team, As you can see I've not kicked off the RC yet. Many bug fixes/dependency updates are happening. Ideally we'll wait until nar Maven plugin goes and we're trying to fix some nifi registry/nifi interaction issues as well. Still will get the RC out as soon as we can.
Thanks On Mon, Jan 23, 2023 at 11:12 AM Joe Witt <joe.w...@gmail.com> wrote: > Hello > > Here is a good picture of what the 1.20 RC looks like. I've > tagged several JIRAs today to ensure we get them in. A theme is really > around stabilizing nifi/nifi-registry integration as we're seeing > substantial uptick in usage and thus various community reported findings. > We'll get that quite smooth with these included. > > https://issues.apache.org/jira/projects/NIFI/versions/12352581 > > Thanks > > On Mon, Jan 23, 2023 at 8:50 AM Joe Witt <joe.w...@gmail.com> wrote: > >> Team, >> >> I'm going through the outstanding JIRAs/PRs and flagging which look like >> they should be 'must have' for 1.20 and then will work the RC as soon as >> those land. >> >> Hopefully have the RC up within a day or two but we'll see how these land >> as some have review comments pending action. >> >> Thanks >> >> On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo < >> isha.lam...@virtualsciences.nl> wrote: >> >>> Hi all, >>> >>> I would like to contribute to the migration tooling (mostly testing I >>> suppose) when that comes up. >>> >>> My team's largest client has a completely template-based pipeline with >>> external scripts replacing variable values before deploying to target >>> clusters, so we've already started looking at this when the goals for 2.0 >>> were discussed and approved. The migration to flowdefinitions and >>> parameters is quite complex and we've hit several blockers when we tried to >>> implement a direct translation. >>> >>> I expect that any time I spend helping to improve the tooling will pay >>> off handsomely for our clients. >>> >>> Regards, >>> >>> Isha >>> >>> -----Oorspronkelijk bericht----- >>> Van: Adam Taft <a...@adamtaft.com> >>> Verzonden: woensdag 11 januari 2023 23:42 >>> Aan: dev@nifi.apache.org >>> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0 >>> >>> This is really insightful and spot on ... >>> >>> Kevin wrote: >>> > Good migration tooling will take a while to develop and test, and the >>> > core contributors to that effort may not have sufficient variety of >>> > flows to evaluate when the migration tools are "done" for the majority >>> > of the community to have success upgrading to 2.x. A milestone release >>> > would >>> allow >>> > us get more feedback on migration over a longer period than the vote >>> window >>> > of an RC candidate. >>> >>> It's exactly this case, that an early 2.0 release might not have had >>> time to fully work its way through existing production deployments, that's >>> concerning. The pace and voting of an "RC" is much too short to get any >>> quality feedback from users in the field. >>> >>> I think it's really smart to consider the "Milestone" release approach >>> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time >>> for feedback. We can put these milestones on a calendar, as needed, so that >>> feedback is required some 'x' number of weeks/months after each milestone. >>> >>> And to this end, I'd personally rather see us keep the 'main' branch >>> current with the 1.x line _until_ we're ready and are satisfied with the >>> end goals of the 2.0 release objectives. When the milestone releases have >>> been completed and there's a comfort level with the 2.x line, it's at the >>> point we'd isolate the 1.x line into its own branch and switch main over to >>> the 2.x line. >>> >>> This is an attractive way of: >>> a) continuing business-as-usual with the 1.x line >>> b) making headway on the 2.x release milestones >>> c) giving adequate time for feedback against the 2.0 milestones coming >>> from the field >>> >>> I don't mind the known-unknowns. But it's really the unknown-unknowns >>> that are going to drive a delay in the 2.0 release. I think it's smart to >>> be able to get some of the unknowns ironed out before we finalize the 2.0 >>> release ceremony. The milestone approach really helps with that. >>> >>> /Adam >>> >>> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kdo...@apache.org> wrote: >>> >>> > Sorry, Joe, I was not clear, and to be honest the two thoughts are >>> > somewhat unrelated in my mind too :) >>> > >>> > I agree that good migration tooling is key. Otherwise, we risk users >>> > staying on 1.x or creating a schism of 1.x and 2.x users. >>> > >>> > Good migration tooling will take a while to develop and test, and the >>> > core contributors to that effort may not have sufficient variety of >>> > flows to evaluate when the migration tools are "done" for the majority >>> > of the community to have success upgrading to 2.x. A milestone release >>> > would allow us get more feedback on migration over a longer period >>> > than the vote window of an RC candidate. >>> > >>> > Perhaps we could continue to release from the 1.x line (including >>> > minor releases with some features) until we are ready to drop the >>> "milestone" >>> > qualifier from 2.0.0, and only then put 1.x into maintenance-only >>> status. >>> > It would be the same proposal to move main to target 2.0.0-M1, but >>> > relax restrictions for what can land on the 1.x branch and be open to >>> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For >>> > example, maybe we would be open to landing new/backported processors >>> > on the 1.x branch, but not core framework features or API changes. >>> > >>> > This might not be necessary, but I think it is fair that saying "no >>> > new features on 1.x" and also "no new features in 2.0.0" puts the >>> > project in a rough place if 2.0.0 takes longer than a few months, so >>> > if we go that route, we need to commit to a quick release of 2.0.0 >>> > that most users can move to easily. >>> > >>> > Thanks, >>> > Kevin >>> > >>> > On Jan 11, 2023 at 12:32:46, Joe Witt <joe.w...@gmail.com> wrote: >>> > >>> > > Kevin, >>> > > >>> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are >>> > prior >>> > > to us officially considering it 2.0/stable. >>> > > >>> > > That said - the migration tooling will be key to provide as we need >>> > > to >>> > make >>> > > the bridge as solid and stable as possible to help someone move from >>> > > 1.x >>> > to >>> > > 2.x. I dont know how related these two concepts (milestone releases >>> > > and 1.x to 2.x ease really are). >>> > > >>> > > Thanks >>> > > >>> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kdo...@apache.org> >>> wrote: >>> > > >>> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts] >>> > > >>> > > >>> > > On this point from David: >>> > > >>> > > >>> > > We may need to have a longer release candidate period, or more >>> > incremental >>> > > >>> > > > fix releases >>> > > >>> > > > for the initial 2.0.0 release train, but I do not expect delaying >>> > > > a >>> > 2.0.0 >>> > > >>> > > > release for new features, as that is not part of the release goals. >>> > > >>> > > > >>> > > >>> > > >>> > > Would the community benefit from one or more milestone releases of >>> > > 2.0, >>> > to >>> > > >>> > > allow for a wider group to run / live on the proposed 2.0 prior to >>> > > >>> > > releasing it as "stable"? I know we've never done a milestone >>> > > release in >>> > > >>> > > the past, and I'm not sure what ASF guidance is on the topic, but if >>> > > it >>> > > >>> > > could be beneficial we could look into that. >>> > > >>> > > >>> > > Cheers, >>> > > >>> > > Kevin >>> > > >>> > > >>> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kdo...@apache.org> wrote: >>> > > >>> > > >>> > > > I think this is a good, practical discussion. >>> > > >>> > > > >>> > > >>> > > > On the one hand, we can't put off 2.x any longer as we really need >>> > > > to >>> > > >>> > > > updated the minimum required Java to 11. Updating main development >>> > > > to >>> > > >>> > > > target 2.x feels like a good way drive progress on that along with >>> > > > the >>> > > >>> > > > other 2.0 goals. >>> > > >>> > > > >>> > > >>> > > > On the other hand, the concerns are valid: moving all development >>> > > > to >>> > > >>> > > > target 2.x puts the project at risk if we cannot release 2.0.0 on >>> > > > a >>> > > >>> > > > reasonable timeline. The restricted scope of 2.0 helps, but this >>> > > > stated >>> > > >>> > > > release goal feels risky to me: >>> > > >>> > > > >>> > > >>> > > > Implement Migration Tools for Upgrading Flows >>> > > >>> > > > >>> > > >>> > > > >>> > > >>> > > > - Implement automated migration where possible to remap >>> > > > properties >>> > and >>> > > >>> > > > features >>> > > >>> > > > - Implement migration tools for manual conversion of XML >>> > Templates >>> > > >>> > > > to JSON Flow Definitions >>> > > >>> > > > - Create documentation for manual steps necessary where >>> > > >>> > > > programmatic migration cannot be implemented >>> > > >>> > > > - NiFi 2.0 should be capable of starting with ghosted >>> > > > components >>> > > >>> > > > for removed Processors or Controller Services >>> > > >>> > > > >>> > > >>> > > > >>> > > >>> > > > Removing deprecated components should be fairly straightforward >>> > > > and >>> > > >>> > > quick, >>> > > >>> > > > but automating and documenting migration is a large effort. >>> > > >>> > > > >>> > > >>> > > > On this po >>> > > >>> > > > >>> > > >>> > > > >>> > > >>> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bbe...@gmail.com> wrote: >>> > > >>> > > > >>> > > >>> > > >> The plan as I understand it is not to diverge and create separate >>> > > >>> > > feature >>> > > >>> > > >> development on the 1.x line, so I would expect all PRs to >>> > > >> continue to >>> > be >>> > > >>> > > >> submitted only to main. We would release 1.x as needed with major >>> > > >> bug >>> > > >>> > > >> fixes >>> > > >>> > > >> or critical security updates, and these would be cherry-picked >>> > > >> and/or >>> > > >>> > > >> backported as necessary, mostly without the need for PRs, the >>> > > >> same as >>> > we >>> > > >>> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) >>> > > >> back >>> > to a >>> > > >>> > > >> maintenance line like (1.19.x). For precedent, we followed this >>> > > >> same >>> > > >>> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any >>> > > >> major >>> > > >>> > > >> issue. >>> > > >>> > > >> >>> > > >>> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler >>> > > >> <ottobackwa...@gmail.com> >>> > > >>> > > >> wrote: >>> > > >>> > > >> >>> > > >>> > > >> It was also mentioned in another thread that we need to have >>> > agreement >>> > > >>> > > on >>> > > >>> > > >> >>> > > >>> > > >> our explicit strategy and support for 1.x going forward, did that >>> > > >>> > > happen? >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> From: Otto Fowler <ottobackwa...@gmail.com> >>> > > >> <ottobackwa...@gmail.com> >>> > > >>> > > >> >>> > > >>> > > >> Reply: Otto Fowler <ottobackwa...@gmail.com> >>> > > >> <ottobackwa...@gmail.com >>> > > >>> > > >>> > > >> >>> > > >>> > > >> Date: January 10, 2023 at 07:02:34 >>> > > >>> > > >> >>> > > >>> > > >> To: dev@nifi.apache.org <dev@nifi.apache.org> >>> > > >> <dev@nifi.apache.org> >>> > > >>> > > >> >>> > > >>> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0 >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> There needs to be an update to the contributing guide as to how >>> > > >> to >>> > > >>> > > submit >>> > > >>> > > >> >>> > > >>> > > >> PRs to 1.x or 2.x etc. >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> From: Joe Witt <joew...@apache.org> <joew...@apache.org> >>> > > >>> > > >> >>> > > >>> > > >> Reply: dev@nifi.apache.org <dev@nifi.apache.org> >>> > > >> <dev@nifi.apache.org >>> > > >>> > > >>> > > >> >>> > > >>> > > >> Date: January 9, 2023 at 15:53:16 >>> > > >>> > > >> >>> > > >>> > > >> To: dev@nifi.apache.org <dev@nifi.apache.org> >>> > > >> <dev@nifi.apache.org> >>> > > >>> > > >> >>> > > >>> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0 >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> Team, >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> As David mentioned in [1] following a successful NiFi 2.0 release >>> > > >> goal >>> > > >>> > > >> >>> > > >>> > > >> planning - we are now going to start moving the 'main' line to be >>> > > >> the >>> > > >>> > > NiFi >>> > > >>> > > >> >>> > > >>> > > >> 2.0 line which will allow for the key work to take place. We will >>> > > >> also >>> > > >>> > > >> >>> > > >>> > > >> move niFi 1.x to its appropriate support line. >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 >>> > > >> and we >>> > > >>> > > have >>> > > >>> > > >> >>> > > >>> > > >> work in there including security items so it is time to make a >>> > release. >>> > > >>> > > >> >>> > > >>> > > >> The intent then is to initiate 1.20 and immediate after that >>> > > >> change >>> > > >>> > > 'main' >>> > > >>> > > >> >>> > > >>> > > >> to 2.0. >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> Going forward then all work on the 1.x line should be focused on >>> > > >>> > > >> >>> > > >>> > > >> maintaining existing features, dependencies, and helping 1.x >>> > > >> users >>> > > >>> > > migrate >>> > > >>> > > >> >>> > > >>> > > >> to the 2.x line. Otherwise, new feature work will happen on >>> > > >> 'main' as >>> > it >>> > > >>> > > >> >>> > > >>> > > >> normally does and will come out in the NiFi 2.x release line. >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> Please flag key outstanding items as we narrow down the release >>> > > >>> > > candidate >>> > > >>> > > >> >>> > > >>> > > >> for NiFi 1.20. >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> Thanks >>> > > >>> > > >> >>> > > >>> > > >> Joe >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> [1] >>> > > >> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2 >>> > > >> Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat >>> > > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48 >>> > > >> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809 >>> > > >> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo >>> > > >> >>> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0 >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >> >>> > > >>> > > >>> > > >>> > >>> >>
