Re: [discuss] Time for a NiFi 2.0 M1 release?
Thanks for initiating the discussion Joe! I also would like to have the flow.xml removal work completed for a milestone release since that is an integral part of the changes. Once that is complete, there may be some other minor things, but I think getting that done would be a good point to move forward with an M1 release version. Regards, David Handermann On Tue, Sep 26, 2023, 12:58 PM Joe Witt wrote: > The Packager thing looks pretty close to being ready anyway. I dont see > that holding anything up on any line at this point. > > I don't have any heartburn waiting for the flowxml and templates to get > tossed out as indeed that needs to happen regardless. > > As far as M1 being a "the most disruptive" variant. While not strictly > your point - that I'd say is a non goal for any of the releases meaning > we're not trying to have a release where we pretend we're sure it is the > most disruptive. The purpose of the M1 will need to be squarely rooted in > getting to a production/stable release. The purpose of any subsequent Mn > or the actual official 2.0.0 release will be ensuring it is the best > possible migration path we intend to make available. > > On Tue, Sep 26, 2023 at 12:04 PM Adam Taft wrote: > > > I'm also hoping that both 1.x and 2.x lines can receive the > PackageFlowFile > > processor that Mike Moser recently proposed. That way, the M1 release and > > the most recent 1.x release will have a simple (or logical) replacement > for > > PostHTTP. > > > > In general, it would be nice to have 1.x lined up with 2.0-M1 so that the > > transitional experience is as disruptive as it's going to be when > 2.0-final > > is released. That is, I want all the things that can break to break, > once a > > 2.0 milestone is released. From that perspective, I agree with Pierre > that > > waiting for the flow.xml work to finalize makes the most sense, because > > then users can start getting a feel for how it will affect them. Lots of > > deployment scripts (think Ansible or equivalent) rely on the flow.xml.gz > > file specifically. > > > > The most disruptive parts of the 1.x to 2.x transition would ideally be > > realized as early as possible. Understand and agree with the urgency to > get > > 2.0-M1 released, but also concerned that it doesn't allow a proper > > evaluation of all breaking changes just yet. > > > > /Adam > > > > On Tue, Sep 26, 2023 at 9:18 AM Pierre Villard < > > pierre.villard...@gmail.com> > > wrote: > > > > > Hey Joe, > > > > > > Definitely a +1 to get a M1 release ASAP. I'd still recommend waiting > on > > > the flow.xml removal work to be merged. The reason being that users may > > > give useful feedback when they'll try NiFi 2.0 with existing flows > coming > > > from NiFi 1.x and getting rid of all of the XML based stuff. There is > > also > > > a PR coming soon for the frontend work of the templates removal. > > Hopefully > > > both can be completed this week or next week. > > > > > > Pierre > > > > > > Le mar. 26 sept. 2023 à 17:35, Joe Witt a écrit : > > > > > > > Team, > > > > > > > > The NiFi 2.0 release has more than 700 resolved JIRAs on it [1] and > > > growing > > > > every day. > > > > > > > > The NiFi 2.0 deprecation plan is well underway and largely complete > > [2]. > > > > > > > > We still need to remove a lot of now deprecated code, tests which are > > > never > > > > run and largely don't work, eliminate the flow.xml which has a > JIRA/PR > > > > underway. And more. But we're getting close and we need to start > > > getting > > > > this in the hands of users. > > > > > > > > The docker image can now be built in 'nifi-docker/dockermaven' after > a > > > full > > > > build from root with 'mvn install -Pdocker'. And it comes up with > > > Ubuntu, > > > > Java 21, Python 3.9, and NiFi 2.0 ready to roll with Python > processors > > > > enabled. > > > > > > > > I propose we start closing down soon to make a NiFi 2.0 M1 release > > happen > > > > even before we have all the things done. We need to start getting > > > feedback > > > > and giving people a chance to work with it. > > > > > > > > Lastly, a huge thank you to the folks in the community that have been > > > > helping push towards 2.x with code changes, removals, reviews, bug > > > reports, > > > > etc.. Super awesome to see. NiFi 2.x is shaping up nicely to be > > useful > > > > not only for our well established user base which spans the globe and > > > every > > > > industry but now we are also seeing a lot of opportunity and fit for > > NiFi > > > > in these exciting AI use cases particularly involving orchestrating > the > > > > data flows with embeddings, vector stores, and LLMs. And the Python > > > > capabilities in NiFi 2.x make NiFi far easier to use for the very > > > important > > > > data engineer user base. > > > > > > > > [1] https://issues.apache.org/jira/projects/NIFI/versions/12339599 > > > > [2] > > > > > > > > > > > > > >
Re: [discuss] Time for a NiFi 2.0 M1 release?
The Packager thing looks pretty close to being ready anyway. I dont see that holding anything up on any line at this point. I don't have any heartburn waiting for the flowxml and templates to get tossed out as indeed that needs to happen regardless. As far as M1 being a "the most disruptive" variant. While not strictly your point - that I'd say is a non goal for any of the releases meaning we're not trying to have a release where we pretend we're sure it is the most disruptive. The purpose of the M1 will need to be squarely rooted in getting to a production/stable release. The purpose of any subsequent Mn or the actual official 2.0.0 release will be ensuring it is the best possible migration path we intend to make available. On Tue, Sep 26, 2023 at 12:04 PM Adam Taft wrote: > I'm also hoping that both 1.x and 2.x lines can receive the PackageFlowFile > processor that Mike Moser recently proposed. That way, the M1 release and > the most recent 1.x release will have a simple (or logical) replacement for > PostHTTP. > > In general, it would be nice to have 1.x lined up with 2.0-M1 so that the > transitional experience is as disruptive as it's going to be when 2.0-final > is released. That is, I want all the things that can break to break, once a > 2.0 milestone is released. From that perspective, I agree with Pierre that > waiting for the flow.xml work to finalize makes the most sense, because > then users can start getting a feel for how it will affect them. Lots of > deployment scripts (think Ansible or equivalent) rely on the flow.xml.gz > file specifically. > > The most disruptive parts of the 1.x to 2.x transition would ideally be > realized as early as possible. Understand and agree with the urgency to get > 2.0-M1 released, but also concerned that it doesn't allow a proper > evaluation of all breaking changes just yet. > > /Adam > > On Tue, Sep 26, 2023 at 9:18 AM Pierre Villard < > pierre.villard...@gmail.com> > wrote: > > > Hey Joe, > > > > Definitely a +1 to get a M1 release ASAP. I'd still recommend waiting on > > the flow.xml removal work to be merged. The reason being that users may > > give useful feedback when they'll try NiFi 2.0 with existing flows coming > > from NiFi 1.x and getting rid of all of the XML based stuff. There is > also > > a PR coming soon for the frontend work of the templates removal. > Hopefully > > both can be completed this week or next week. > > > > Pierre > > > > Le mar. 26 sept. 2023 à 17:35, Joe Witt a écrit : > > > > > Team, > > > > > > The NiFi 2.0 release has more than 700 resolved JIRAs on it [1] and > > growing > > > every day. > > > > > > The NiFi 2.0 deprecation plan is well underway and largely complete > [2]. > > > > > > We still need to remove a lot of now deprecated code, tests which are > > never > > > run and largely don't work, eliminate the flow.xml which has a JIRA/PR > > > underway. And more. But we're getting close and we need to start > > getting > > > this in the hands of users. > > > > > > The docker image can now be built in 'nifi-docker/dockermaven' after a > > full > > > build from root with 'mvn install -Pdocker'. And it comes up with > > Ubuntu, > > > Java 21, Python 3.9, and NiFi 2.0 ready to roll with Python processors > > > enabled. > > > > > > I propose we start closing down soon to make a NiFi 2.0 M1 release > happen > > > even before we have all the things done. We need to start getting > > feedback > > > and giving people a chance to work with it. > > > > > > Lastly, a huge thank you to the folks in the community that have been > > > helping push towards 2.x with code changes, removals, reviews, bug > > reports, > > > etc.. Super awesome to see. NiFi 2.x is shaping up nicely to be > useful > > > not only for our well established user base which spans the globe and > > every > > > industry but now we are also seeing a lot of opportunity and fit for > NiFi > > > in these exciting AI use cases particularly involving orchestrating the > > > data flows with embeddings, vector stores, and LLMs. And the Python > > > capabilities in NiFi 2.x make NiFi far easier to use for the very > > important > > > data engineer user base. > > > > > > [1] https://issues.apache.org/jira/projects/NIFI/versions/12339599 > > > [2] > > > > > > > > > https://cwiki.apache.org/confluence/display/NIFI/Deprecated+Components+and+Features > > > > > > Thanks > > > Joe > > > > > >
Re: [discuss] Time for a NiFi 2.0 M1 release?
I'm also hoping that both 1.x and 2.x lines can receive the PackageFlowFile processor that Mike Moser recently proposed. That way, the M1 release and the most recent 1.x release will have a simple (or logical) replacement for PostHTTP. In general, it would be nice to have 1.x lined up with 2.0-M1 so that the transitional experience is as disruptive as it's going to be when 2.0-final is released. That is, I want all the things that can break to break, once a 2.0 milestone is released. From that perspective, I agree with Pierre that waiting for the flow.xml work to finalize makes the most sense, because then users can start getting a feel for how it will affect them. Lots of deployment scripts (think Ansible or equivalent) rely on the flow.xml.gz file specifically. The most disruptive parts of the 1.x to 2.x transition would ideally be realized as early as possible. Understand and agree with the urgency to get 2.0-M1 released, but also concerned that it doesn't allow a proper evaluation of all breaking changes just yet. /Adam On Tue, Sep 26, 2023 at 9:18 AM Pierre Villard wrote: > Hey Joe, > > Definitely a +1 to get a M1 release ASAP. I'd still recommend waiting on > the flow.xml removal work to be merged. The reason being that users may > give useful feedback when they'll try NiFi 2.0 with existing flows coming > from NiFi 1.x and getting rid of all of the XML based stuff. There is also > a PR coming soon for the frontend work of the templates removal. Hopefully > both can be completed this week or next week. > > Pierre > > Le mar. 26 sept. 2023 à 17:35, Joe Witt a écrit : > > > Team, > > > > The NiFi 2.0 release has more than 700 resolved JIRAs on it [1] and > growing > > every day. > > > > The NiFi 2.0 deprecation plan is well underway and largely complete [2]. > > > > We still need to remove a lot of now deprecated code, tests which are > never > > run and largely don't work, eliminate the flow.xml which has a JIRA/PR > > underway. And more. But we're getting close and we need to start > getting > > this in the hands of users. > > > > The docker image can now be built in 'nifi-docker/dockermaven' after a > full > > build from root with 'mvn install -Pdocker'. And it comes up with > Ubuntu, > > Java 21, Python 3.9, and NiFi 2.0 ready to roll with Python processors > > enabled. > > > > I propose we start closing down soon to make a NiFi 2.0 M1 release happen > > even before we have all the things done. We need to start getting > feedback > > and giving people a chance to work with it. > > > > Lastly, a huge thank you to the folks in the community that have been > > helping push towards 2.x with code changes, removals, reviews, bug > reports, > > etc.. Super awesome to see. NiFi 2.x is shaping up nicely to be useful > > not only for our well established user base which spans the globe and > every > > industry but now we are also seeing a lot of opportunity and fit for NiFi > > in these exciting AI use cases particularly involving orchestrating the > > data flows with embeddings, vector stores, and LLMs. And the Python > > capabilities in NiFi 2.x make NiFi far easier to use for the very > important > > data engineer user base. > > > > [1] https://issues.apache.org/jira/projects/NIFI/versions/12339599 > > [2] > > > > > https://cwiki.apache.org/confluence/display/NIFI/Deprecated+Components+and+Features > > > > Thanks > > Joe > > >
Re: [discuss] Time for a NiFi 2.0 M1 release?
Hey Joe, Definitely a +1 to get a M1 release ASAP. I'd still recommend waiting on the flow.xml removal work to be merged. The reason being that users may give useful feedback when they'll try NiFi 2.0 with existing flows coming from NiFi 1.x and getting rid of all of the XML based stuff. There is also a PR coming soon for the frontend work of the templates removal. Hopefully both can be completed this week or next week. Pierre Le mar. 26 sept. 2023 à 17:35, Joe Witt a écrit : > Team, > > The NiFi 2.0 release has more than 700 resolved JIRAs on it [1] and growing > every day. > > The NiFi 2.0 deprecation plan is well underway and largely complete [2]. > > We still need to remove a lot of now deprecated code, tests which are never > run and largely don't work, eliminate the flow.xml which has a JIRA/PR > underway. And more. But we're getting close and we need to start getting > this in the hands of users. > > The docker image can now be built in 'nifi-docker/dockermaven' after a full > build from root with 'mvn install -Pdocker'. And it comes up with Ubuntu, > Java 21, Python 3.9, and NiFi 2.0 ready to roll with Python processors > enabled. > > I propose we start closing down soon to make a NiFi 2.0 M1 release happen > even before we have all the things done. We need to start getting feedback > and giving people a chance to work with it. > > Lastly, a huge thank you to the folks in the community that have been > helping push towards 2.x with code changes, removals, reviews, bug reports, > etc.. Super awesome to see. NiFi 2.x is shaping up nicely to be useful > not only for our well established user base which spans the globe and every > industry but now we are also seeing a lot of opportunity and fit for NiFi > in these exciting AI use cases particularly involving orchestrating the > data flows with embeddings, vector stores, and LLMs. And the Python > capabilities in NiFi 2.x make NiFi far easier to use for the very important > data engineer user base. > > [1] https://issues.apache.org/jira/projects/NIFI/versions/12339599 > [2] > > https://cwiki.apache.org/confluence/display/NIFI/Deprecated+Components+and+Features > > Thanks > Joe >
[discuss] Time for a NiFi 2.0 M1 release?
Team, The NiFi 2.0 release has more than 700 resolved JIRAs on it [1] and growing every day. The NiFi 2.0 deprecation plan is well underway and largely complete [2]. We still need to remove a lot of now deprecated code, tests which are never run and largely don't work, eliminate the flow.xml which has a JIRA/PR underway. And more. But we're getting close and we need to start getting this in the hands of users. The docker image can now be built in 'nifi-docker/dockermaven' after a full build from root with 'mvn install -Pdocker'. And it comes up with Ubuntu, Java 21, Python 3.9, and NiFi 2.0 ready to roll with Python processors enabled. I propose we start closing down soon to make a NiFi 2.0 M1 release happen even before we have all the things done. We need to start getting feedback and giving people a chance to work with it. Lastly, a huge thank you to the folks in the community that have been helping push towards 2.x with code changes, removals, reviews, bug reports, etc.. Super awesome to see. NiFi 2.x is shaping up nicely to be useful not only for our well established user base which spans the globe and every industry but now we are also seeing a lot of opportunity and fit for NiFi in these exciting AI use cases particularly involving orchestrating the data flows with embeddings, vector stores, and LLMs. And the Python capabilities in NiFi 2.x make NiFi far easier to use for the very important data engineer user base. [1] https://issues.apache.org/jira/projects/NIFI/versions/12339599 [2] https://cwiki.apache.org/confluence/display/NIFI/Deprecated+Components+and+Features Thanks Joe
