Team, We are seeing an increase in the unique contributors on a monthly basis to NiFi and a clear and steady increase in the overall contribution rate over the past 1-2 years in particular. This is awesome but we do have important problems to cover and we need to make it easier to help newer contributors know where to jump in and add the most value.
For sure people will contribute various things just because they have a particular interest such as some new extension, etc.. But from a community direction perspective there are things we need to continue to improve upon and we want to help make the path to committership and PMC membership more clear as a function of adding value to the community. So not necessarily ordered, here are some key things I've observed that need more attention and are among the most valuable things for the health of the nifi system and community. Dependency Management This is a hugely important area as we have a massive amount of dependencies and this work is just plain not that fun. But if we want to present a beautiful landscape we need to pull weeds. We need to reduce dependencies and keep things recent. This is largely accomplished today by a very small group of people. We have to increase the contribution level here or perhaps the automation level. We have tooling that is helping now but still needs more attention. Vulnerability Management particular as it relates to dependencies Related to dependency management but broader. This one at times requires us to do more than simply maintain dependencies. It sometimes forces us to change libraries and sometimes even makes maintaining compatibility/configuration complicated. These are among the most critical and most difficult tasks to take on. This is largely done today by an extremely small group of people. Maintenance of extensions We have a ton of components in NiFi. Obviously not all of them really do require active maintenance but there is often a contribute and forget model involved. For us to keep these broad range of components under community care they need active maintenance. I already mentioned the dependency factor above but the broader point here is that APIs/versions of systems evolve. We need to be staying up to date. There are systems like Kafka, Elastic, Mongo, Relational Databases, etc.. that we should always be supporting the latest on. These are among the most wildly popular sources and destinations. We do tend to fall behind here and this is powerfully useful for people to contribute to. These are the things our users actually use NiFi for! Maintenance of things other than core nifi If you look at components ancillary to NiFi they receive far less attention and effort trending toward them going away. The NiFi registry is a great example of this as NiFi MiNiFi Java for instance. These are things which are useful and indeed there is a decent user base in both cases. But they require more active maintenance than they get. We need to evolve when it comes to security methods offered for authn/authz and that takes focus. Performance analysis and improvements targeted as a result This is done very rarely as far as I can tell outside a couple of people who I'm sure we could all name in a few seconds. The codebase is large and Java and other parts evolve. Continually emphasizing faster and more efficient operations is vital to NiFi's continued growth in the community at large. People who spend time running NiFi in a measured manner with realistic flows and doing things like profiling for CPU, memory, etc.. and finding bottlenecks and offering solutions - absolutely can be high impact and valuable. Even if not at a point where a confident contribution can be made, the very act of doing this level of evaluation and flagging concerns is valuable. Improvements related to tests both unit and integration - but also for users of NiFi As part of our push to NiFi 2.0 we have been running these tests far more often and more complete than ever before. Github Actions now pretty much does run all the things. We've deleted a wild amount of junk tests that would always fail or were spurious or required some complex local access/setup and therefore nobody ever ran them. We need to always improve coverage but we want to make tests faster/more efficient and more leveraging of things like test containers would be highly valuable. Importantly though here I also mean to highlight we need to do a lot more for users of NiFi that want to create automated tests for their flows. We should make it easy for them to provide a flow definition or perhaps a DSL for their flow and write tests which would leverage actual nifi to validate a given input results in an expected output. This is something that would add a ton of value for our user base but also lead into some really powerful testing we can do against our own framework. Development of blogs/videos/conference presentations/social media posts Building great software by itself doesn't grow the community nearly as much as if you build a strong developer advocacy game. People that create these various forms of blogs/videos and conference talks about NiFi, how it works, and how to make it better are extremely valuable. That is often just as valuable or even more so than a PR review, code contribution, etc.. Engagement in the Slack/Mailing lists Our community needs help. They ask a lot of questions about how to use NiFi, bugs they are hitting, etc.. Responding to these questions in a timely and meaningful manner is hugely valuable to them and is also pretty fun for the person who helps out. Meaningful engagement acting as a pull request reviewer This one is last but certainly not least! We have a review-then-commit model which means we need at least one committer or PMC member to review a contribution before it gets merged. Even if you're not a committer or PMC member, a review is still super helpful and can save time and also puts you on a great path to earning merit. Reviews are a critical part of the code and overall quality of NiFi. 'LGTM' is generally not that helpful and while warranted for some trivial things we even see it on larger more meaningful commits. Not every review needs to be a back and forth affair of code style inputs either. There is a middle ground and you'll find it as you engage in more reviews! Code reviews are in far more short supply than code contributions. I encourage everyone to please do at least one true pull request review for every contribution. In general the goal should be to contribute to more pull request reviews than new pull requests created. If you do so this by definition means the community is growing and you're directly helping make that happen! Would love to see additional inputs/missing categories/other thoughts. Perhaps this turns into a wiki page that helps give more context to people that want to find ways to add value and be on a path to committer and PMC membership as well. Thanks Joe
