Thanks David. The proposal looks good to me and I definitely like the idea of a separate NIP jira project, as well as separating nifi-api to its own repo.
On Sat, Aug 31, 2024 at 10:36 AM Joe Witt <joe.w...@gmail.com> wrote: > Thanks David. > > Looks like a solid starting point. > > > On Fri, Aug 30, 2024 at 8:15 PM David Handermann < > exceptionfact...@apache.org> wrote: > > > Team, > > > > I have drafted an initial version of the NiFi Improvement Proposal > Process: > > > > > > > https://cwiki.apache.org/confluence/display/NIFI/NiFi+Improvement+Proposal+Process > > > > The document outlines the scope of project components requiring a > > proposal, as well as the requirements for a proposal, and the review > > process. > > > > The general approach for review and approval is designed to be similar > > to the release process. > > > > The requirements for a proposal are similar to other project > > improvement proposals. > > > > I recommend creating a new Jira project for handling Improvement > > Proposals, as opposed to using Confluence pages. > > > > Please review the draft and I will look to call for a vote within the > next > > week. > > > > As mentioned earlier, with the nifi-api module being the core contract > > for internal and external extensions, I would like to include moving > > the nifi-api to a separate Git repository as part of the vote for > > instituting the Improvement Proposal Process. If this proves to be too > > much of a concern, we can divide the question and vote separately on > > the Improvement Proposal Process and moving the nifi-api. > > > > Thanks for the feedback! > > > > Regards, > > David Handermann > > > > On Wed, Aug 21, 2024 at 11:09 AM David Handermann > > <exceptionfact...@apache.org> wrote: > > > > > > Thanks for the replies! > > > > > > I will move forward with drafting a NiFi Improvements Proposal page to > > > keep the discussion going. > > > > > > Regards, > > > David Handermann > > > > > > On Tue, Aug 20, 2024 at 3:22 PM Mark Payne <marka...@hotmail.com> > wrote: > > > > > > > > David, > > > > > > > > +1 to all of this. Especially for guaranteeing compatibility and > > maintainability, I think instrumenting a more > > > > formal approach for updating the API is a step in the right > direction. > > The huge amount of purging, cleanup, > > > > and refactoring that has gone into 2.0 helps to highlight the > > importance of ensuring maintainability. > > > > > > > > Thanks > > > > -Mark > > > > > > > > > > > > > On Aug 20, 2024, at 2:00 AM, Joe Witt <joe.w...@gmail.com> wrote: > > > > > > > > > > David > > > > > > > > > > Yeah well written and good points. > > > > > > > > > > I dont think this says we would relax our existing commitment we > > have shown > > > > > for things such as the http based api but rather is calling out > > specific > > > > > things which we will make changes to even more formalized. > > > > > > > > > > To that end +1 on the concept of proposals and +1 to breaking out > the > > > > > nifi-api (fundamental designed extension points for flow > > development) in > > > > > particular. > > > > > > > > > > Thanks > > > > > Joe > > > > > > > > > > On Fri, Aug 16, 2024 at 8:20 PM Adam Taft <a...@adamtaft.com> > wrote: > > > > > > > > > >> These are all really appreciated concepts, David. Thank you for > > putting the > > > > >> thoughts and time in on this! > > > > >> > > > > >> Regarding this: > > > > >>> The NiFi REST API does not have quite the same level of concern, > > but may > > > > >> warrant inclusion. > > > > >> > > > > >> I hear what you're saying. However, the REST API (from my > > > > >> observation/experience) has gathered quite a number of useful > tools > > and > > > > >> "hacks" for NiFi. Quite often, many different monitoring and > > alerting tools > > > > >> have been developed against the REST API by third parties and/or > > > > >> integrators of NiFi against their internal workflows. Having > stable > > API > > > > >> versioning in the REST API possibly makes just as much sense as > > having the > > > > >> same for the nifi-api itself. This is a prime entry point for > > extensions > > > > >> and other features developed alongside NiFi, maybe even the weird > > stuff > > > > >> that you can't do with the nifi-api directly. > > > > >> > > > > >> Food for thought of course, but I would hope that we can treat the > > REST API > > > > >> as a proper first-class citizen in terms of documented versioning. > > It turns > > > > >> out it's quite a useful means for interacting with a running NiFi > > instance. > > > > >> > > > > >> /Adam > > > > >> > > > > >> On Fri, Aug 16, 2024 at 9:59 AM David Handermann < > > > > >> exceptionfact...@apache.org> wrote: > > > > >> > > > > >>> Team, > > > > >>> > > > > >>> As we wrap up remaining items for NiFi 2, we should consider how > to > > > > >>> continue improving the quality and maintainability of the NiFi > > > > >>> ecosystem. > > > > >>> > > > > >>> One primary focus of NiFi 2 has been the reduction of technical > > debt, > > > > >>> which involved the removal of numerous modules and thousands of > > lines > > > > >>> of code. In that process, it is worth highlighting that the core > > NiFi > > > > >>> API, and the NiFi Framework API, and the NiFi REST API have had > > > > >>> comparatively few breaking changes. This a testament to the > > quality of > > > > >>> the API design itself. The NiFi Version Schema and API > > Compatibility > > > > >>> [1] has provided a strong direction thus far. > > > > >>> > > > > >>> With that background, we should consider adopting a more formal > > > > >>> process around changes that impact the fundamental API contracts > > that > > > > >>> NiFi provides. NiFi Feature Proposals [2] have provided elements > of > > > > >>> this in the past, but did not include approval requirements. > Kafka > > [3] > > > > >>> and Airflow [4] have more structured improvement proposal > > processes, > > > > >>> and that is what we should adopt going forward. > > > > >>> > > > > >>> Part of moving in this direction requires identifying the areas > > that > > > > >>> would require going through the Improvement Proposal process > > itself. > > > > >>> At minimum, this should include the nifi-api [5] module. The > > > > >>> nifi-framework-api [6] is also worth consideration for inclusion > in > > > > >>> this category. The NiFi REST API does not have quite the same > > level of > > > > >>> concern, but may warrant inclusion. > > > > >>> > > > > >>> As part of this discussion, we should consider separating the > > nifi-api > > > > >>> module into its own repository, with its own versioning scheme. > > This > > > > >>> will provide a helpful distinction in terms of the scope of > > changes, > > > > >>> and allow the API to be released independently of the > application, > > > > >>> providing strong version compatibility guarantees. > > > > >>> > > > > >>> Based on feedback for the general idea, I would be glad to draft > a > > > > >>> NiFi Improvement Proposal page, outlining the recommended steps > in > > > > >>> more detail so we can come to consensus on how this should work. > > > > >>> > > > > >>> Regards, > > > > >>> David Handermann > > > > >>> > > > > >>> [1] > > > > >>> > > > > >> > > > https://cwiki.apache.org/confluence/display/NIFI/Version+Scheme+and+API+Compatibility > > > > >>> [2] > > > > >>> > > https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals > > > > >>> [3] > > > > >>> > > > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > > >>> [4] > > > > >>> > > > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals > > > > >>> [5] https://github.com/apache/nifi/tree/main/nifi-api > > > > >>> [6] https://github.com/apache/nifi/tree/main/nifi-framework-api > > > > >>> > > > > >> > > > > > > >
