Hello Here is a good picture of what the 1.20 RC looks like. I've tagged several JIRAs today to ensure we get them in. A theme is really around stabilizing nifi/nifi-registry integration as we're seeing substantial uptick in usage and thus various community reported findings. We'll get that quite smooth with these included.
https://issues.apache.org/jira/projects/NIFI/versions/12352581 Thanks On Mon, Jan 23, 2023 at 8:50 AM Joe Witt <joe.w...@gmail.com> wrote: > Team, > > I'm going through the outstanding JIRAs/PRs and flagging which look like > they should be 'must have' for 1.20 and then will work the RC as soon as > those land. > > Hopefully have the RC up within a day or two but we'll see how these land > as some have review comments pending action. > > Thanks > > On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo < > isha.lam...@virtualsciences.nl> wrote: > >> Hi all, >> >> I would like to contribute to the migration tooling (mostly testing I >> suppose) when that comes up. >> >> My team's largest client has a completely template-based pipeline with >> external scripts replacing variable values before deploying to target >> clusters, so we've already started looking at this when the goals for 2.0 >> were discussed and approved. The migration to flowdefinitions and >> parameters is quite complex and we've hit several blockers when we tried to >> implement a direct translation. >> >> I expect that any time I spend helping to improve the tooling will pay >> off handsomely for our clients. >> >> Regards, >> >> Isha >> >> -----Oorspronkelijk bericht----- >> Van: Adam Taft <a...@adamtaft.com> >> Verzonden: woensdag 11 januari 2023 23:42 >> Aan: dev@nifi.apache.org >> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0 >> >> This is really insightful and spot on ... >> >> Kevin wrote: >> > Good migration tooling will take a while to develop and test, and the >> > core contributors to that effort may not have sufficient variety of >> > flows to evaluate when the migration tools are "done" for the majority >> > of the community to have success upgrading to 2.x. A milestone release >> > would >> allow >> > us get more feedback on migration over a longer period than the vote >> window >> > of an RC candidate. >> >> It's exactly this case, that an early 2.0 release might not have had time >> to fully work its way through existing production deployments, that's >> concerning. The pace and voting of an "RC" is much too short to get any >> quality feedback from users in the field. >> >> I think it's really smart to consider the "Milestone" release approach >> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time >> for feedback. We can put these milestones on a calendar, as needed, so that >> feedback is required some 'x' number of weeks/months after each milestone. >> >> And to this end, I'd personally rather see us keep the 'main' branch >> current with the 1.x line _until_ we're ready and are satisfied with the >> end goals of the 2.0 release objectives. When the milestone releases have >> been completed and there's a comfort level with the 2.x line, it's at the >> point we'd isolate the 1.x line into its own branch and switch main over to >> the 2.x line. >> >> This is an attractive way of: >> a) continuing business-as-usual with the 1.x line >> b) making headway on the 2.x release milestones >> c) giving adequate time for feedback against the 2.0 milestones coming >> from the field >> >> I don't mind the known-unknowns. But it's really the unknown-unknowns >> that are going to drive a delay in the 2.0 release. I think it's smart to >> be able to get some of the unknowns ironed out before we finalize the 2.0 >> release ceremony. The milestone approach really helps with that. >> >> /Adam >> >> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kdo...@apache.org> wrote: >> >> > Sorry, Joe, I was not clear, and to be honest the two thoughts are >> > somewhat unrelated in my mind too :) >> > >> > I agree that good migration tooling is key. Otherwise, we risk users >> > staying on 1.x or creating a schism of 1.x and 2.x users. >> > >> > Good migration tooling will take a while to develop and test, and the >> > core contributors to that effort may not have sufficient variety of >> > flows to evaluate when the migration tools are "done" for the majority >> > of the community to have success upgrading to 2.x. A milestone release >> > would allow us get more feedback on migration over a longer period >> > than the vote window of an RC candidate. >> > >> > Perhaps we could continue to release from the 1.x line (including >> > minor releases with some features) until we are ready to drop the >> "milestone" >> > qualifier from 2.0.0, and only then put 1.x into maintenance-only >> status. >> > It would be the same proposal to move main to target 2.0.0-M1, but >> > relax restrictions for what can land on the 1.x branch and be open to >> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For >> > example, maybe we would be open to landing new/backported processors >> > on the 1.x branch, but not core framework features or API changes. >> > >> > This might not be necessary, but I think it is fair that saying "no >> > new features on 1.x" and also "no new features in 2.0.0" puts the >> > project in a rough place if 2.0.0 takes longer than a few months, so >> > if we go that route, we need to commit to a quick release of 2.0.0 >> > that most users can move to easily. >> > >> > Thanks, >> > Kevin >> > >> > On Jan 11, 2023 at 12:32:46, Joe Witt <joe.w...@gmail.com> wrote: >> > >> > > Kevin, >> > > >> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are >> > prior >> > > to us officially considering it 2.0/stable. >> > > >> > > That said - the migration tooling will be key to provide as we need >> > > to >> > make >> > > the bridge as solid and stable as possible to help someone move from >> > > 1.x >> > to >> > > 2.x. I dont know how related these two concepts (milestone releases >> > > and 1.x to 2.x ease really are). >> > > >> > > Thanks >> > > >> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kdo...@apache.org> >> wrote: >> > > >> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts] >> > > >> > > >> > > On this point from David: >> > > >> > > >> > > We may need to have a longer release candidate period, or more >> > incremental >> > > >> > > > fix releases >> > > >> > > > for the initial 2.0.0 release train, but I do not expect delaying >> > > > a >> > 2.0.0 >> > > >> > > > release for new features, as that is not part of the release goals. >> > > >> > > > >> > > >> > > >> > > Would the community benefit from one or more milestone releases of >> > > 2.0, >> > to >> > > >> > > allow for a wider group to run / live on the proposed 2.0 prior to >> > > >> > > releasing it as "stable"? I know we've never done a milestone >> > > release in >> > > >> > > the past, and I'm not sure what ASF guidance is on the topic, but if >> > > it >> > > >> > > could be beneficial we could look into that. >> > > >> > > >> > > Cheers, >> > > >> > > Kevin >> > > >> > > >> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kdo...@apache.org> wrote: >> > > >> > > >> > > > I think this is a good, practical discussion. >> > > >> > > > >> > > >> > > > On the one hand, we can't put off 2.x any longer as we really need >> > > > to >> > > >> > > > updated the minimum required Java to 11. Updating main development >> > > > to >> > > >> > > > target 2.x feels like a good way drive progress on that along with >> > > > the >> > > >> > > > other 2.0 goals. >> > > >> > > > >> > > >> > > > On the other hand, the concerns are valid: moving all development >> > > > to >> > > >> > > > target 2.x puts the project at risk if we cannot release 2.0.0 on >> > > > a >> > > >> > > > reasonable timeline. The restricted scope of 2.0 helps, but this >> > > > stated >> > > >> > > > release goal feels risky to me: >> > > >> > > > >> > > >> > > > Implement Migration Tools for Upgrading Flows >> > > >> > > > >> > > >> > > > >> > > >> > > > - Implement automated migration where possible to remap >> > > > properties >> > and >> > > >> > > > features >> > > >> > > > - Implement migration tools for manual conversion of XML >> > Templates >> > > >> > > > to JSON Flow Definitions >> > > >> > > > - Create documentation for manual steps necessary where >> > > >> > > > programmatic migration cannot be implemented >> > > >> > > > - NiFi 2.0 should be capable of starting with ghosted >> > > > components >> > > >> > > > for removed Processors or Controller Services >> > > >> > > > >> > > >> > > > >> > > >> > > > Removing deprecated components should be fairly straightforward >> > > > and >> > > >> > > quick, >> > > >> > > > but automating and documenting migration is a large effort. >> > > >> > > > >> > > >> > > > On this po >> > > >> > > > >> > > >> > > > >> > > >> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bbe...@gmail.com> wrote: >> > > >> > > > >> > > >> > > >> The plan as I understand it is not to diverge and create separate >> > > >> > > feature >> > > >> > > >> development on the 1.x line, so I would expect all PRs to >> > > >> continue to >> > be >> > > >> > > >> submitted only to main. We would release 1.x as needed with major >> > > >> bug >> > > >> > > >> fixes >> > > >> > > >> or critical security updates, and these would be cherry-picked >> > > >> and/or >> > > >> > > >> backported as necessary, mostly without the need for PRs, the >> > > >> same as >> > we >> > > >> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) >> > > >> back >> > to a >> > > >> > > >> maintenance line like (1.19.x). For precedent, we followed this >> > > >> same >> > > >> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any >> > > >> major >> > > >> > > >> issue. >> > > >> > > >> >> > > >> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler >> > > >> <ottobackwa...@gmail.com> >> > > >> > > >> wrote: >> > > >> > > >> >> > > >> > > >> It was also mentioned in another thread that we need to have >> > agreement >> > > >> > > on >> > > >> > > >> >> > > >> > > >> our explicit strategy and support for 1.x going forward, did that >> > > >> > > happen? >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> From: Otto Fowler <ottobackwa...@gmail.com> >> > > >> <ottobackwa...@gmail.com> >> > > >> > > >> >> > > >> > > >> Reply: Otto Fowler <ottobackwa...@gmail.com> >> > > >> <ottobackwa...@gmail.com >> > > >> > > >> > > >> >> > > >> > > >> Date: January 10, 2023 at 07:02:34 >> > > >> > > >> >> > > >> > > >> To: dev@nifi.apache.org <dev@nifi.apache.org> >> > > >> <dev@nifi.apache.org> >> > > >> > > >> >> > > >> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0 >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> There needs to be an update to the contributing guide as to how >> > > >> to >> > > >> > > submit >> > > >> > > >> >> > > >> > > >> PRs to 1.x or 2.x etc. >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> From: Joe Witt <joew...@apache.org> <joew...@apache.org> >> > > >> > > >> >> > > >> > > >> Reply: dev@nifi.apache.org <dev@nifi.apache.org> >> > > >> <dev@nifi.apache.org >> > > >> > > >> > > >> >> > > >> > > >> Date: January 9, 2023 at 15:53:16 >> > > >> > > >> >> > > >> > > >> To: dev@nifi.apache.org <dev@nifi.apache.org> >> > > >> <dev@nifi.apache.org> >> > > >> > > >> >> > > >> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0 >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> Team, >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> As David mentioned in [1] following a successful NiFi 2.0 release >> > > >> goal >> > > >> > > >> >> > > >> > > >> planning - we are now going to start moving the 'main' line to be >> > > >> the >> > > >> > > NiFi >> > > >> > > >> >> > > >> > > >> 2.0 line which will allow for the key work to take place. We will >> > > >> also >> > > >> > > >> >> > > >> > > >> move niFi 1.x to its appropriate support line. >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 >> > > >> and we >> > > >> > > have >> > > >> > > >> >> > > >> > > >> work in there including security items so it is time to make a >> > release. >> > > >> > > >> >> > > >> > > >> The intent then is to initiate 1.20 and immediate after that >> > > >> change >> > > >> > > 'main' >> > > >> > > >> >> > > >> > > >> to 2.0. >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> Going forward then all work on the 1.x line should be focused on >> > > >> > > >> >> > > >> > > >> maintaining existing features, dependencies, and helping 1.x >> > > >> users >> > > >> > > migrate >> > > >> > > >> >> > > >> > > >> to the 2.x line. Otherwise, new feature work will happen on >> > > >> 'main' as >> > it >> > > >> > > >> >> > > >> > > >> normally does and will come out in the NiFi 2.x release line. >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> Please flag key outstanding items as we narrow down the release >> > > >> > > candidate >> > > >> > > >> >> > > >> > > >> for NiFi 1.20. >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> Thanks >> > > >> > > >> >> > > >> > > >> Joe >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> [1] >> > > >> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2 >> > > >> Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat >> > > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48 >> > > >> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809 >> > > >> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo >> > > >> >> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0 >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> > > >> > >> >
