Joe/David, Both of you make fair points on the technical side, but I think the wording and optics probably needs some careful work before we make things official to avoid giving the impression that this is a hoop teams need to jump through very soon or bad things could happen.
On Thu, Nov 7, 2024 at 11:01 AM David Handermann < exceptionfact...@apache.org> wrote: > Team, > > One general thing to note regarding migration: declaring the version 1 > series EOL should not be coupled to migration timelines. > > Users are free to set their own paths for upgrading as needed. The > point of declaring version 1 EOL is to emphasize that migration needs > to be scheduled. We see enough users of NiFi 1 versions from years ago > to know that upgrades take time. > > The important point to emphasize is that the NiFi project cannot > provide support for components that are already unsupported. > > Regards, > David Handermann > > On Thu, Nov 7, 2024 at 9:51 AM David Handermann > <exceptionfact...@apache.org> wrote: > > > > Mike, > > > > Although I understand the desire to provide basic support for > > historical versions of NiFi, attempting to upgrade these foundational > > dependencies in the NiFi 1 series is not a tenable path forward. > > > > The level of effort to modernize the frameworks for both frontend and > > backend in NiFi 2 was very high, covering all aspects of the project. > > The frontend involved a complete rebuild, and the backend required > > changes to most framework components. > > > > With the need to upgrade to Java 17 at minimum, such an approach does > > not seem like it would be helpful, even if it were worth considering > > the massive effort involved. > > > > As Joe has highlighted in previous comments, we outlined these issues > > several years ago as the path for NiFi 2. Given that background, > > providing ongoing support for NiFi 1 is not an option based on the > > foundational library and Java constraints. > > > > Regards, > > David Handermann > > > > > > > > > > On Thu, Nov 7, 2024 at 9:31 AM Joe Witt <joe.w...@gmail.com> wrote: > > > > > > Mike, > > > > > > I do not believe there is a viable path that addresses the JVM base, > Spring > > > base, Jetty base, Front-end Code changes, and the many other realities > of > > > how crusty the 1.x line has become that would not end up effectively > just > > > being what NiFi 2.0 is already. And for sure what you're suggesting > we can > > > estimate rather well given we did all these things already. > > > > > > As far as declaring EOL now, in 3 months, one year.... All of these > > > timelines are fine provided we (the PMC) have specific people > volunteering > > > to do what is proposed. > > > > > > Can you please suggest or specify what you would change relative to the > > > latest proposal? > > > > > > Thanks > > > > > > On Thu, Nov 7, 2024 at 8:16 AM Mike Thomsen <mikerthom...@gmail.com> > wrote: > > > > > > > Joe, > > > > > > > > One way forward to help out users on 1.X would be to explore > creating an > > > > EOL support release of 1.28 that incorporates the 2.0 Spring 6 > changes, > > > > bumps the JDK requirement to 17 and provides a "just good enough" > Java 17 > > > > baseline support. I don't know what the LOE would be on the front > end side > > > > for Angular support, but I think a 1.29 release that only gets point > > > > releases that incorporate CVE fixes and that is supported for a year > would > > > > be a compromise most users could accept. > > > > > > > > I share Russ's concerns and think that we run the risk of a creating > a > > > > major perception problem if we don't provide a longer migration > period for > > > > teams on 1.X. > > > > > > > > On Wed, Nov 6, 2024 at 1:56 PM Joe Witt <joe.w...@gmail.com> wrote: > > > > > > > > > "I'm not sure what "we will not be doing further analysis/triage of > > > > > security > > > > > reports" means, especially the term "we". Seems like if we do that > during > > > > > the release process already then we shouldn't stop. If it refers to > > > > > community members having to do such analysis themselves, I'm not > sure why > > > > > we'd make a point of saying we won't." > > > > > > > > > > It is a proposal. So 'we' is all of us - as proposed. More > specifically > > > > > though the 'we' represented there are those who are on and > participate in > > > > > the security mailing list which is where reports of > vulnerabilities or > > > > > suspected vulnerabilities or just questions that people aren't > sure are > > > > > security related or not come into. When things are reported > against 1.x > > > > > they require analysis to determine if it is actually vulnerable > and to > > > > > craft/consider fixes. > > > > > > > > > > It is essential we are clear about our posture there. We do not > want > > > > users > > > > > expecting/assuming reports against 1.x will be receiving a level of > > > > > scrutiny and follow-through that they will not. My proposal is > that we > > > > > make it clear and don't address such reports on the 1.x line any > longer > > > > and > > > > > are able to cite/refer to the items in the proposal. > > > > > > > > > > We would still of course still do dependency bumps as normal > course but > > > > > that alone with bug fixes are subject to people in the community > doing > > > > that > > > > > work. > > > > > > > > > > > > > > > On Wed, Nov 6, 2024 at 11:35 AM Matt Burgess <mattyb...@apache.org > > > > > > wrote: > > > > > > > > > > > My two cents: > > > > > > > > > > > > 1. The EOL/EOS label is fine with me, as Apache software I'm not > even > > > > > sure > > > > > > what that means in general (sounds like more of a marketing > thing), > > > > but I > > > > > > have seen other projects like Hive vote for EOL of older release > lines. > > > > > For > > > > > > your proposed definition, reasonable periodic bug fixes and > dependency > > > > > > upgrades based on discovered vulnerabilities also sound good. > However > > > > I'm > > > > > > not sure what "we will not be doing further analysis/triage of > security > > > > > > reports" means, especially the term "we". Seems like if we do > that > > > > during > > > > > > the release process already then we shouldn't stop. If it refers > to > > > > > > community members having to do such analysis themselves, I'm not > sure > > > > why > > > > > > we'd make a point of saying we won't. > > > > > > > > > > > > I certainly agree that there should be no new features added in > 1.28.x, > > > > > but > > > > > > in terms of bug fix / vulnerability things, I would like to just > have > > > > > > dot-releases as necessary, so no 1.29 but perhaps a 1.28.2 or .3 > as the > > > > > > need arises. The community is free to vote or abstain from > voting as > > > > they > > > > > > choose. I also don't think there needs to be any cadence, just > someone > > > > > > bringing it up for discussion (and having a good reason to > produce a > > > > > > release). > > > > > > > > > > > > I agree with points 2-6. > > > > > > > > > > > > Regards, > > > > > > Matt > > > > > > > > > > > > On Tue, Nov 5, 2024 at 11:31 AM Joe Witt <joe.w...@gmail.com> > wrote: > > > > > > > > > > > > > To be actionable and concrete here is a proposal: > > > > > > > > > > > > > > 1. Declare the NiFi 1.28.x line as the 'End of Life/End of > Support' > > > > > line. > > > > > > > This means we may still do periodic bug fixes or when > > > > > possible/reasonable > > > > > > > bump vulnerable libs. But we will not be doing further > > > > analysis/triage > > > > > > of > > > > > > > security reports nor adding features. > > > > > > > > > > > > > > 2. Add a DISCLAIMER to the source and key binaries of the > 1.x/1.28 > > > > line > > > > > > > [1]. > > > > > > > > > > > > > > 3. Update the downloads page making the links for nifi source > and the > > > > > > main > > > > > > > nifi assembly of whatever is the latest NiFi 1.28.x release > there but > > > > > > > clearly articulated as the end of support line for which bug > fixes > > > > and > > > > > > some > > > > > > > narrow dependency updates may occur. Advise users of the 1.x > line of > > > > > the > > > > > > > importance of planning to migrate to the 2.x line. > > > > > > > > > > > > > > 4. Conduct a VOTE to codify this. > > > > > > > > > > > > > > 5. Conduct an Apache NiFi 1.28.1 release to pickle up (2) and > the bug > > > > > > fixes > > > > > > > already available. > > > > > > > > > > > > > > 6. As we gather more user input on things which are helpful to > them > > > > we > > > > > > > factor these into migration guidance/tooling as appropriate on > the > > > > 2.x > > > > > > > line. > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > [1] https://github.com/apache/nifi/pull/9491 > > > > > > > > > > > > > > On Tue, Nov 5, 2024 at 9:04 AM Joe Witt <joe.w...@gmail.com> > wrote: > > > > > > > > > > > > > > > Team, > > > > > > > > > > > > > > > > I will start a discussion thread on the users list so we can > hear > > > > > more > > > > > > > > inputs from them and from that perspective. > > > > > > > > > > > > > > > > This thread needs to focus on what the > contributors/committers/PMC > > > > in > > > > > > the > > > > > > > > community can/will/should do and the PMC in particular as > we're > > > > > > obligated > > > > > > > > to ensure we're putting out software for which we can stand > behind > > > > > its > > > > > > > > security posture. > > > > > > > > > > > > > > > > We do not need to get worried about customers. They have > vendors > > > > > that > > > > > > > > support them. What we need to worry about and continue to > do an > > > > > > > excellent > > > > > > > > job caring for is the apache nifi user base and we need to > ensure > > > > > they > > > > > > > > don't have the belief that the NiFi 1.x line will be fixed > in the > > > > > > > presence > > > > > > > > of vulnerability reports. I'll ask on the users list how > folks > > > > would > > > > > > > like > > > > > > > > us to communicate about the state of things. > > > > > > > > > > > > > > > > What I think we need to ask here is more in the spirit of > what this > > > > > > > thread > > > > > > > > was started about. When do we as a > contributor/committer/PMC base > > > > > want > > > > > > > to > > > > > > > > make it official in our own sense that we will not be > producing > > > > > > releases > > > > > > > > for the 1.x line? How we best communicate/help the user > base then > > > > > > > follows > > > > > > > > from that. Stated another way those who feel they will be > in a > > > > good > > > > > > > > position to do security reviews, vulnerability scans and > > > > remediation, > > > > > > > > conduct releases for some period of time please share what > you > > > > think > > > > > > > you'll > > > > > > > > be able to do and roughly for how long. > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > On Tue, Nov 5, 2024 at 3:36 AM Pierre Villard < > > > > > > > pierre.villard...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > >> Hi all, > > > > > > > >> > > > > > > > >> While I think we could set an EOL date a bit further in the > > > > future, > > > > > it > > > > > > > is > > > > > > > >> important to keep in mind what EOL means. It only means we > won't > > > > be > > > > > > > >> providing security fixes / bug fixes through new releases. > It does > > > > > not > > > > > > > >> mean that NiFi 1.x is gone. If that is a big concern for > some > > > > users > > > > > > when > > > > > > > >> running EOL software then we should remind those users that > > > > they've > > > > > > been > > > > > > > >> doing it for 2+ years already when using NiFi 1.x (taking > Jetty as > > > > > an > > > > > > > >> example here). And Joe is definitely right when saying that > we > > > > have > > > > > a > > > > > > > >> smaller and smaller group of people willing to spend an > extensive > > > > > > amount > > > > > > > >> of > > > > > > > >> time taking care of PR/reviews, of release candidates, > > > > > > > testing/validating > > > > > > > >> RCs, etc, for the 1.x line. > > > > > > > >> > > > > > > > >> I also agree that, even if many users are already using > NiFi 2 in > > > > > > > >> production, many places have strict policies to not adopt a > new > > > > > major > > > > > > > >> release. I don't want to start a debate whether this is > making > > > > sense > > > > > > or > > > > > > > >> not > > > > > > > >> but we know those rules exist in many places :) And the > fact that > > > > we > > > > > > had > > > > > > > >> milestone releases for one year is not going to be enough > of an > > > > > > > argument. > > > > > > > >> > > > > > > > >> Given what we've seen in the past, we usually make a new > release > > > > > > every 3 > > > > > > > >> months or so. It's probably fair to assume a 2.1.0 release > will > > > > > happen > > > > > > > >> early next year. With that in mind, I tend to agree with > Michael > > > > > > > >> suggesting > > > > > > > >> an EOL date at the end of January (3 months from now). We > could > > > > also > > > > > > say > > > > > > > >> that 1.28.1 will happen at this time and will be the last > one in > > > > the > > > > > > > >> community. > > > > > > > >> > > > > > > > >> Vendors have already announced support for NiFi 1.x for > multiple > > > > > > > >> additional > > > > > > > >> years so this approach follows what we see in other > projects where > > > > > > > >> extended > > > > > > > >> support is only provided through paid options with specific > > > > > companies. > > > > > > > >> > > > > > > > >> It is awesome to finally see 2.0 out and this decision will > help > > > > > drive > > > > > > > >> users to that new release, which is much better in so many > ways... > > > > > > > >> > > > > > > > >> Thanks, > > > > > > > >> Pierre > > > > > > > >> > > > > > > > >> > > > > > > > >> Le mar. 5 nov. 2024 à 10:36, Isha Lamboo < > > > > > > > isha.lam...@virtualsciences.nl> > > > > > > > >> a > > > > > > > >> écrit : > > > > > > > >> > > > > > > > >> > Hi all, > > > > > > > >> > > > > > > > > >> > I understand the reasons to declare an EOL quickly, given > the > > > > > > external > > > > > > > >> > dependencies, but like Russell said before the short > notice is > > > > > going > > > > > > > to > > > > > > > >> > cause trouble with our bigger corporate customers. It > would have > > > > > > been > > > > > > > >> nice > > > > > > > >> > to have the EOL date announced about a year ago, even if > it had > > > > > > been a > > > > > > > >> > provisional one. The more you can delay it now, the less > > > > > > credibility I > > > > > > > >> (and > > > > > > > >> > NiFi itself) lose :-\ > > > > > > > >> > > > > > > > > >> > I've been pushing since the first announcement of NiFi > 2.0 for > > > > our > > > > > > > >> > customers to prepare. The smaller NiFi instances are all > > > > prepared. > > > > > > But > > > > > > > >> > there are also big customers with hundreds of flows that > depend > > > > on > > > > > > > >> > variables and XML templates, and as you can imagine this > was > > > > > never a > > > > > > > >> > priority for them without either a NiFi 2.0 GA to move to > or an > > > > > > actual > > > > > > > >> EOL > > > > > > > >> > date to get security officers upping the priority. > > > > > > > >> > > > > > > > > >> > Now we have a GA release finally, but corporate Q4 plans > are set > > > > > in > > > > > > > >> stone > > > > > > > >> > and Q1 2025 plans are already filling up. Telling the > customers' > > > > > > > >> > development teams to upend their plans and tell their > business > > > > > > > >> customers to > > > > > > > >> > forget deliveries because NiFi needs to be fixed ASAP is > > > > probably > > > > > > not > > > > > > > >> going > > > > > > > >> > to fly and instead going to seriously dent NiFi's > reputation and > > > > > > > >> position. > > > > > > > >> > Unless we can automate the flow migration process it's > going to > > > > > be a > > > > > > > >> > year-long migration at least. > > > > > > > >> > > > > > > > > >> > That said, are there any tools or scripts to make the > migration > > > > > > > >> smoother? > > > > > > > >> > Configuring multiple levels of parameter contexts with > > > > inheritance > > > > > > is > > > > > > > a > > > > > > > >> > labor-intensive process if we are to mirror the current > setup > > > > with > > > > > > > >> > variables being inherited from main canvas, team PG, > subject PG > > > > > and > > > > > > > flow > > > > > > > >> > PG, etc. Anything that could go through the process > groups and > > > > > > > configure > > > > > > > >> > this automatically would be greatly appreciated. I will > look > > > > into > > > > > > that > > > > > > > >> > myself too, but anything helps really. > > > > > > > >> > > > > > > > > >> > Regards, > > > > > > > >> > > > > > > > > >> > Isha > > > > > > > >> > > > > > > > > >> > -----Oorspronkelijk bericht----- > > > > > > > >> > Van: Joe Witt <joe.w...@gmail.com> > > > > > > > >> > Verzonden: maandag 4 november 2024 23:44 > > > > > > > >> > Aan: dev@nifi.apache.org > > > > > > > >> > Onderwerp: Re: [DISCUSS] End-of-life timing for NiFi 1 > > > > > > > >> > > > > > > > > >> > The EOL discussion is not here because we have a new > problem. > > > > It > > > > > is > > > > > > > >> here > > > > > > > >> > because we finally have an answer. > > > > > > > >> > > > > > > > > >> > The inability to address reported vulnerabilities or > fundamental > > > > > end > > > > > > > of > > > > > > > >> > life status for key underlying components in the 1.x line > is a > > > > > > problem > > > > > > > >> that > > > > > > > >> > was fully recognized three years ago. > > > > > > > >> > > > > > > > > >> > In that time we created a plan for what NiFi 2.0 would be > and > > > > how > > > > > > we'd > > > > > > > >> > manage both maintaining the 1.x line while building to > the 2.x > > > > GA. > > > > > > In > > > > > > > >> the > > > > > > > >> > past year we've conducted four milestone releases of NiFi > 2.x > > > > and > > > > > > > we've > > > > > > > >> > continued putting out feature, bug fix, and security > improvement > > > > > > > >> releases > > > > > > > >> > of 1.x. > > > > > > > >> > > > > > > > > >> > Feature bearing releases of 1.x are no longer appropriate > as 2.x > > > > > is > > > > > > > here > > > > > > > >> > and GA and that was the plan all along. > > > > > > > >> > > > > > > > > >> > Bug fixes are still reasonable in spirit but you need > people to > > > > > > submit > > > > > > > >> the > > > > > > > >> > JIRAs, fix the JIRAs, peer review the changes, and to > conduct > > > > > > releases > > > > > > > >> and > > > > > > > >> > make votes. That is in increasingly short supply as it > has been > > > > > > quite > > > > > > > >> the > > > > > > > >> > task splitting attention across two major lines and > naturally > > > > > > > developers > > > > > > > >> > and users will gravitate toward the go forward path. > > > > > > > >> > > > > > > > > >> > Vulnerability/Security related considerations are where > things > > > > are > > > > > > > >> > fundamentally problematic. We had a security report > today about > > > > > the > > > > > > > >> super > > > > > > > >> > old/outdated front-end libraries we use in 1.x. That > won't > > > > > change. > > > > > > > We > > > > > > > >> had > > > > > > > >> > a report last week about Spring libraries needing updated > except > > > > > you > > > > > > > >> can't > > > > > > > >> > unless you have Pivotal support so not an option. Those > won't > > > > > > change. > > > > > > > >> We > > > > > > > >> > have had questions around Jetty changes but that is tied > to Java > > > > > 8. > > > > > > > >> We've > > > > > > > >> > had questions about Java 8 being end of life and even > Java 11 > > > > and > > > > > > even > > > > > > > >> now > > > > > > > >> > Java 17 in terms of its codebase permissive licensing > changing. > > > > > The > > > > > > > >> things > > > > > > > >> > we can reasonably address in the 1.x line are getting > smaller > > > > and > > > > > > > >> smaller > > > > > > > >> > and the time required to address any new thing is higher > and > > > > > higher. > > > > > > > >> > > > > > > > > >> > We as a community, regardless of good intentions, cannot > fix the > > > > > > > >> illities > > > > > > > >> > of the 1.x line and thus the 2.x line is here. The 1.x > line > > > > will > > > > > > > >> > absolutely continue to atrophy and it will accelerate. > If we do > > > > > not > > > > > > > >> signal > > > > > > > >> > EOL on 1.x that means we're saying we can keep fixing > problems. > > > > > > While > > > > > > > >> that > > > > > > > >> > is true for bugs, that is not true for vulnerabilities > broadly > > > > and > > > > > > for > > > > > > > >> our > > > > > > > >> > most critical components. > > > > > > > >> > > > > > > > > >> > If you still fix bugs people assume this means you still > > > > > reasonably > > > > > > > fix > > > > > > > >> > vulnerabilities/etc.. And unless we declare EOL on the > 1.x line > > > > > we > > > > > > > will > > > > > > > >> > continue to get non-serviceable reports and mislead the > user > > > > base. > > > > > > > >> > > > > > > > > >> > The answer is to clearly signal that users should > transition to > > > > > the > > > > > > > 2.x > > > > > > > >> > line and focus our help on answering questions people > might have > > > > > on > > > > > > > how > > > > > > > >> to > > > > > > > >> > do that. > > > > > > > >> > > > > > > > > >> > I am supportive of EOL for the 1.x line. I also like the > poetic > > > > > > > nature > > > > > > > >> of > > > > > > > >> > the decade timing. > > > > > > > >> > > > > > > > > >> > On Mon, Nov 4, 2024 at 2:47 PM David Handermann < > > > > > > > >> > exceptionfact...@apache.org> > > > > > > > >> > wrote: > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
