Thanks David. Looks like a solid starting point.
On Fri, Aug 30, 2024 at 8:15 PM David Handermann < exceptionfact...@apache.org> wrote: > Team, > > I have drafted an initial version of the NiFi Improvement Proposal Process: > > > https://cwiki.apache.org/confluence/display/NIFI/NiFi+Improvement+Proposal+Process > > The document outlines the scope of project components requiring a > proposal, as well as the requirements for a proposal, and the review > process. > > The general approach for review and approval is designed to be similar > to the release process. > > The requirements for a proposal are similar to other project > improvement proposals. > > I recommend creating a new Jira project for handling Improvement > Proposals, as opposed to using Confluence pages. > > Please review the draft and I will look to call for a vote within the next > week. > > As mentioned earlier, with the nifi-api module being the core contract > for internal and external extensions, I would like to include moving > the nifi-api to a separate Git repository as part of the vote for > instituting the Improvement Proposal Process. If this proves to be too > much of a concern, we can divide the question and vote separately on > the Improvement Proposal Process and moving the nifi-api. > > Thanks for the feedback! > > Regards, > David Handermann > > On Wed, Aug 21, 2024 at 11:09 AM David Handermann > <exceptionfact...@apache.org> wrote: > > > > Thanks for the replies! > > > > I will move forward with drafting a NiFi Improvements Proposal page to > > keep the discussion going. > > > > Regards, > > David Handermann > > > > On Tue, Aug 20, 2024 at 3:22 PM Mark Payne <marka...@hotmail.com> wrote: > > > > > > David, > > > > > > +1 to all of this. Especially for guaranteeing compatibility and > maintainability, I think instrumenting a more > > > formal approach for updating the API is a step in the right direction. > The huge amount of purging, cleanup, > > > and refactoring that has gone into 2.0 helps to highlight the > importance of ensuring maintainability. > > > > > > Thanks > > > -Mark > > > > > > > > > > On Aug 20, 2024, at 2:00 AM, Joe Witt <joe.w...@gmail.com> wrote: > > > > > > > > David > > > > > > > > Yeah well written and good points. > > > > > > > > I dont think this says we would relax our existing commitment we > have shown > > > > for things such as the http based api but rather is calling out > specific > > > > things which we will make changes to even more formalized. > > > > > > > > To that end +1 on the concept of proposals and +1 to breaking out the > > > > nifi-api (fundamental designed extension points for flow > development) in > > > > particular. > > > > > > > > Thanks > > > > Joe > > > > > > > > On Fri, Aug 16, 2024 at 8:20 PM Adam Taft <a...@adamtaft.com> wrote: > > > > > > > >> These are all really appreciated concepts, David. Thank you for > putting the > > > >> thoughts and time in on this! > > > >> > > > >> Regarding this: > > > >>> The NiFi REST API does not have quite the same level of concern, > but may > > > >> warrant inclusion. > > > >> > > > >> I hear what you're saying. However, the REST API (from my > > > >> observation/experience) has gathered quite a number of useful tools > and > > > >> "hacks" for NiFi. Quite often, many different monitoring and > alerting tools > > > >> have been developed against the REST API by third parties and/or > > > >> integrators of NiFi against their internal workflows. Having stable > API > > > >> versioning in the REST API possibly makes just as much sense as > having the > > > >> same for the nifi-api itself. This is a prime entry point for > extensions > > > >> and other features developed alongside NiFi, maybe even the weird > stuff > > > >> that you can't do with the nifi-api directly. > > > >> > > > >> Food for thought of course, but I would hope that we can treat the > REST API > > > >> as a proper first-class citizen in terms of documented versioning. > It turns > > > >> out it's quite a useful means for interacting with a running NiFi > instance. > > > >> > > > >> /Adam > > > >> > > > >> On Fri, Aug 16, 2024 at 9:59 AM David Handermann < > > > >> exceptionfact...@apache.org> wrote: > > > >> > > > >>> Team, > > > >>> > > > >>> As we wrap up remaining items for NiFi 2, we should consider how to > > > >>> continue improving the quality and maintainability of the NiFi > > > >>> ecosystem. > > > >>> > > > >>> One primary focus of NiFi 2 has been the reduction of technical > debt, > > > >>> which involved the removal of numerous modules and thousands of > lines > > > >>> of code. In that process, it is worth highlighting that the core > NiFi > > > >>> API, and the NiFi Framework API, and the NiFi REST API have had > > > >>> comparatively few breaking changes. This a testament to the > quality of > > > >>> the API design itself. The NiFi Version Schema and API > Compatibility > > > >>> [1] has provided a strong direction thus far. > > > >>> > > > >>> With that background, we should consider adopting a more formal > > > >>> process around changes that impact the fundamental API contracts > that > > > >>> NiFi provides. NiFi Feature Proposals [2] have provided elements of > > > >>> this in the past, but did not include approval requirements. Kafka > [3] > > > >>> and Airflow [4] have more structured improvement proposal > processes, > > > >>> and that is what we should adopt going forward. > > > >>> > > > >>> Part of moving in this direction requires identifying the areas > that > > > >>> would require going through the Improvement Proposal process > itself. > > > >>> At minimum, this should include the nifi-api [5] module. The > > > >>> nifi-framework-api [6] is also worth consideration for inclusion in > > > >>> this category. The NiFi REST API does not have quite the same > level of > > > >>> concern, but may warrant inclusion. > > > >>> > > > >>> As part of this discussion, we should consider separating the > nifi-api > > > >>> module into its own repository, with its own versioning scheme. > This > > > >>> will provide a helpful distinction in terms of the scope of > changes, > > > >>> and allow the API to be released independently of the application, > > > >>> providing strong version compatibility guarantees. > > > >>> > > > >>> Based on feedback for the general idea, I would be glad to draft a > > > >>> NiFi Improvement Proposal page, outlining the recommended steps in > > > >>> more detail so we can come to consensus on how this should work. > > > >>> > > > >>> Regards, > > > >>> David Handermann > > > >>> > > > >>> [1] > > > >>> > > > >> > https://cwiki.apache.org/confluence/display/NIFI/Version+Scheme+and+API+Compatibility > > > >>> [2] > > > >>> > https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals > > > >>> [3] > > > >>> > > > >> > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > >>> [4] > > > >>> > > > >> > https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals > > > >>> [5] https://github.com/apache/nifi/tree/main/nifi-api > > > >>> [6] https://github.com/apache/nifi/tree/main/nifi-framework-api > > > >>> > > > >> > > > >
