Hi all,
Thanks for starting the discussion, David! Currently I'm leaning pro
status quo: I think while MiNiFi C++ is acting as a semi-separate
subproject in many respects, it benefits from being under the NiFi
umbrella. This lends the project legitimacy that a small independent
project of similar size would not otherwise have. I think MiNiFi Java is
in a similar position. Whether NiFi benefits from having these
subprojects is not as clear to me, I'll leave that up to the people who
work more closely with the main project codebase.
On the con side, I also see that release votes have been a struggle
lately, with cross-verification interest (or capability) between NiFi
people and MiNiFi C++ people decreasing, with 3 MiNiFi C++-affiliated
people in the PMC, this led to a bus factor of 1 for release votes. I
would try to encourage more collaboration between the two camps, so we
work like one big team, with some groups specializing in their areas of
interest, but occasionally collaborating. Hopefully this is enough to
keep the two camps united.
If this is not feasible or not enough, like David suggested, then I
would explore whether it's possible to grant special voting rights to
MiNiFi C++ regulars on its release, without the divorce, leaving wider
NiFi community voting rights on MiNiFi C++ releases unchanged.
MiNiFi Java doesn't suffer from these issues as much, since it's a
special distribution of NiFi, with a few unique components, rather than
a truly separate codebase, and since there are even fewer committers
specialized in maintaining this distribution, I'm not sure it would be a
viable top level project on its own.
Another question to consider if we create a new TLP is trademark: the
MiNiFi names contain and refer to NiFi, being a special distribution of
it and a partial reimplementation of it. Since the previously mentioned
NiFi-related but independent projects also have NiFi in their name, this
doesn't seem to be a huge issue. If we go for a new top level project, I
would also propose renaming at least MiNiFi C++ to avoid confusion with
MiNiFi Java, but that's for the new PMC to decide.
In summary, I'd try to make it work together, if it's not nuisance to
the wider NiFi community.
Regards,
Marton
On 11/5/25 11:29 PM, David Handermann wrote:
Tony,
Thanks for the reply, and I appreciate the phrasing. Based on general
experience, attempting to compel greater individual ownership does not
seem like a feasible way forward.
Rather than describing options to counteract the diffusion, I think it
is better to frame things as recognizing the diffusion and promoting
the differentiation.
There is a natural affinity and directional focus as projects grow, so
recognizing that and finding the best way to make the most of it has a
number of potential benefits.
Regards,
David Handermann
On Wed, Nov 5, 2025 at 3:48 PM Tony Kurc <[email protected]> wrote:
David,
I was thinking about the lack of votes too.. I didn't attribute it to lack
of knowledge, I attributed it a bit to the bystander effect more than
anything. Reviewing is not *all *technical; focusing on the build docs, the
licensing due diligence, and so on is pretty universal. I think there is
some merit to you saying a subproject has a higher likelihood of causing
diffusion of responsibilities, but that hurts my soul a bit (especially
biting as a non-voter on that release).
I appreciate your thoughts even if they cause me to be disappointed in
myself. Since we both recognize this pattern, are we really discussing
measures to counteract this diffusion or compel greater individual
ownership (PMC)?
Thanks,
Tony
On Wed, Nov 5, 2025 at 3:43 PM David Handermann <[email protected]>
wrote:
Thanks for all the interaction thus far!
One important thing worth highlighting is the reality of operation
versus the perceived benefits of one path or another.
The current reality is that MiNiFi is effectively operating as an
independent project. Although this has worked to some degree with a
smaller PMC membership, it becomes more challenging as participation
grows on both projects.
As one concrete example, the recent release for MiNiFi 0.99.2 [1] had
to be extended given the lack of binding votes. This reflects the
reality that most NiFi PMC members are not familiar enough to provide
confident binding votes.
One solution under the current model is to add more NiFi PMC members
who contribute solely to MiNiFi. Given the much larger scope of NiFi,
however, and the fact that there is no notion of subdivision from a
PMC governance perspective, I am not comfortable proceeding in that
direction.
It could also be argued that adding NiFi PMC members who have no
understanding of MiNiFi is problematic, although less so based on
relative project focus.
I am not discounting the benefits of collaboration, but most of those
benefits can and should continue apart from project governance
direction. Participation in Slack communities and email lists is open
to everyone, regardless of project focus.
Thanks again for the helpful consideration thus far, I look forward to
additional replies.
Regards,
David Handermann
[1] https://lists.apache.org/thread/qpff11cxfc4lxm7pxghcnqgn16fc1d3w
On Wed, Nov 5, 2025 at 1:24 PM Gábor Gyimesi <[email protected]> wrote:
I've been reflecting on this topic over the last two days, and in my
opinion MiNiFi benefits more as a subproject than as a top level
project. I also believe that the overall NiFi ecosystem gains more
value from having both NiFi and MiNiFi developers and users
participate in discussions, even when certain topics don't directly
affect their primary area of focus.
One of the key advantages of maintaining a single project is improved
communication with the user community. Engaging with users across
shared channels (such as common mailing lists or Slack workspaces)
makes it easier to discuss use cases, provide guidance, and recommend
the best options when both NiFi and MiNiFi developers are involved.
This collaboration helps users better understand which project fits
their needs. Shared communication channels also make it easier to
assess how decisions impact both projects, even when those effects
might not be immediately obvious from one side's perspective.
NiFi's larger and more established community also contributes to
greater visibility for MiNiFi. Since NiFi's user base is significantly
larger, this can help grow MiNiFi's audience, provided the
subproject's goals are clearly communicated. In this respect, I agree
there's room for improvement. I don't think having a project page
under the NiFi website is a bad idea, but enhancing its UX design,
improving discoverability, and highlighting relevant use cases could
draw more attention to MiNiFi than transitioning it into a TLP.
There's also the matter of shared features and integration between
NiFi and MiNiFi. One of MiNiFi's core use cases is serving as an edge
data collector that forwards data to NiFi via the Site-to-Site
protocol. Supporting use cases like this is more effective within a
unified project community, where collaboration on integration features
(like updates to the Site-to-Site protocol) is simpler. Similarly,
with the recent introduction of Python processors in NiFi, maintaining
a single project structure would make it easier to develop and support
the Python API for both NiFi and MiNiFi.
Finally, MiNiFi C++ currently lags behind NiFi in terms of available
features. Keeping both within a single project would streamline the
process of porting useful improvements from NiFi to MiNiFi. Even in
early design and feature discussions, having both NiFi and MiNiFi
developers involved adds value by bringing different perspectives on
use cases and implementation details.
I know some of these points may not be directly related to project
governance, but I believe they could be unfortunate consequences of
further isolating the projects from one another. All in all, I think
maintaining MiNiFi as a subproject rather than a top-level project
best supports collaboration, interoperability, and consistency across
the broader NiFi ecosystem.
Regards,
Gabor
On Wed, 5 Nov 2025 at 18:47, Tony Kurc <[email protected]> wrote:
My bias is that the subproject approach has benefits that outweigh the
benefits of a broken out TLP
1. I think there is merit in shared governance - it's unrealistic that
every PMC member knows all parts of a project, and I don't think
there are
any ASF projects where everyone knows everything from a technical
perspective, and there are components where only a few contribute, or
even
understand. I think the breadth of experience in both technical and
non-technical gives all the subprojects resilience. We have people who
know
and care about these subcomponents that might not otherwise.
2. I think the direct presence that David mentions is a bit of an
argument
of a lack of visibility (i.e. MiNiFi gets "outshined"), but I could be
wrong. That is currently a tacit choice we're making as a community and
project. We could be making choices to highlight on the website or in
other
communication, but we don't. If lack of presence is an issue, we could
address in other ways.
3. I think there is value in shared branding and user trust. At least
with
me, a tight sub project association mitigates risk of fragmenting the
somewhat joint "story" of NiFi and MiNiFis for users, folks like
integrators, and maybe even ASF?
4. I think splitting has significant overhead costs in coordination,
such
as cross-project apis or alignment, amplified by some of the reporting
that
comes along with being a TLP.
What I'm not sure I'm seeing is any community divergence or
independence
that I would expect to see for considering TLP vs PMC. I unsuccessfully
searched through some of the other projects mailing lists to try to
understand other peoples' considerations for subproject->TLP decisions.
Tony
On Wed, Nov 5, 2025 at 12:14 PM Joe Witt <[email protected]> wrote:
I think all that is being suggested here is should we consider having
MiNiFi be its own TLP... 5 members in that PMC is enough and I'd
bet at
least a few of us would also be part of that PMC as well.
On Wed, Nov 5, 2025 at 10:08 AM Michael Moser <[email protected]>
wrote:
I still think #2 is an option. It would be up to those 5 or more
committers to determine what to do with it.
There are more NiFi adjacent projects out there (NyPyApi and
NiFiKop to
name a couple) that are maintained as open source but not governed
by the
ASF or NiFi PMC.
-- Mike
On Wed, Nov 5, 2025 at 11:26 AM Arpad Boda <[email protected]>
wrote:
Mike,
2) is not really an option, it's being contributed to actively
by ~5
committers.
So the question is deciding between 1) and 3).
I'm not sure at the moment which I find better, will get back to
this
thread with a longer response.
Cheers,
Arpad
On Wed, Nov 5, 2025 at 5:17 PM Michael Moser <[email protected]
wrote:
I don't think there is a "correct" way forward with MiNiFi
C++, but I
can
think of a few options.
1) Status quo. Keep MiNiFi C++ in Apache NiFi. There will be
NiFi
committers and PMC members who do not touch MiNiFi C++ and
those who
do
not
touch NiFi. The risk of harm to the codebases because of this
is
acceptably low.
2) Jettison the MiNiFi C++ project completely to the point the
NiFi
PMC
itself is no longer responsible for its governance or is
involved
with
its
maintenance or continued open source status.
3) The NiFi PMC votes to split MiNiFi C++ into an Apache
Incubator
podling
and provides mentors. I would worry about getting enough
participation
in
the new podling.
The biggest issue for me is that MiNiFi C++ has separate issue
tracking
from NiFi. This makes it essentially a separately run
"division" of
the
NiFi organization with minimal interaction with other
divisions. Its
separate git repository, while a very sensical decision, also
contributes
to less interaction with the core nifi repository.
If I had to vote, as someone who has never used and probably
never
will
use
MiNiFi C++ because MiNiFi Java meets all my needs, I would
choose #2.
-- Mike
On Mon, Nov 3, 2025 at 10:28 PM David Handermann <
[email protected]> wrote:
Team,
I would like to discuss the future relationship of the NiFi
and
MiNiFi
projects, considering in particular whether MiNiFi should
pursue
promotion to an independent Apache Top Level Project.
The NiFi and MiNiFi projects have had a long history of
interoperability through common contracts and shared
implementation
strategies. The Java version of MiNiFi was merged into the
primary
NiFi repository several years ago, but maintains a module
structure
that is mostly decoupled from the NiFi framework. The C++
version
of
MiNiFi has had an independent repository, issue tracking
system,
and
release cadence since its inception.
In terms of project governance, the NiFi Project Management
Committee
is responsible for everything. In practice, however,
responsibility
for MiNiFi, and the C++ version in particular, resides with a
subset
of NiFi PMC members. The arrangement has worked more or
less, but
as
work continues on both projects, it is becoming increasingly
clear
that knowledge of both projects is not common. This is a
natural
result of independent languages, and distinct
implementations. For
this reason, bringing on new members to the NiFi PMC does not
necessarily benefit both projects. Although issue tracking is
independent, it also means that project discussions
generally skew
in
the direction of NiFi. These are not problems in themselves,
but
these
realities highlight the functional independence of NiFi and
MiNiFi.
There are technical questions to consider, in terms of API
surfaces
like C2 protocol manifests and common Python Processors
interfaces.
However, these technical questions can be addressed in
various
ways,
such as promoting certain shared elements to independent
repositories
similar to the Apache NiFi API.
The real questions concern governance. As mentioned, the C++
version
of MiNiFi already has functional independence in several key
areas.
Promotion to a TLP would clarify participation and
qualification
for
PMC membership in both NiFi and MiNiFi. As a TLP, MiNiFi
would also
enjoy a more direct presence, rather than a project page
under the
NiFi website.
This would not necessitate any changes to current NiFi PMC or
committer membership, as participation in multiple Apache
projects
is
common in places like the Iceberg ecosystem.
This would require both technical and procedural work, but
having
clarity of governance should provide a better path forward
for both
projects.
Regards,
David Handermann
