[
https://issues.apache.org/jira/browse/NIFI-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17836665#comment-17836665
]
Joe Witt commented on NIFI-12998:
---------------------------------
This started in an effort to remove a couple clearly extraneous entries in the
nifi-nar-bundle pom. What I found as I pulled the thread was that our repo
holistically has grown very messy and tangled in ways that make adding
capabilities and maintenance much harder. Some members of the community have
done tremendous work to improve the situation by moving more things to the top
level pom dependency management section and introducing BOMs where available
for things like Jetty and various others. These all help and are part of the
strategy but several challenges remain due to years of technical debt accrual.
1. We have made the 'nifi-nar-bundles' module into a frankenstein of awkward
inter dependencies between the framework, actual extensions, etc.. You really
can't build components outside of that with the way it works right now without
a lot of manual work.
2. We have no clear delineation of what is actually provided in the application
of NiFi (or minifi java or stateless for that matter) when build extension nars
on top.
3. Related to 2 it isn't clear or obvious to users actually how any of our
classloading magic truly works. Which artifacts are you actually having to
contend with being in a parent classloader and which ones are you free to take
a different version/approach with?
4. If you want to build a new nar or modify one which exists you really don't
know and frankly don't have much options what you can use. The parent modules
have forced dependencies (not managed dependencies but hard dependencies) which
complicate things. You also effectively have to build them within the
nifi-nar-bundles module hierarchy to have it work for now. Going forward there
should be in effect three patterns we enable. (a) you want to build a nar and
you only need base/root dependencies which you cannot ovveride. (b) you want
to build a nar and you're going to leverage one or more standard service apis.
(c) you want to build a nar and you're going to leverage standard shared
libraries and service apis. Nars should be free to choose whichever level of
integration/tie in they want and they should be able to live easily wherever
they need in the nifi codebase or in anyone elses repo. It was always supposed
to be dead simple to build extensions.
5. If you analyze dependency trees POMs are riddled with absurd inclusions that
have absolutely nothing to do with the component involved. Lots of copy and
paste is clearly a factor but also inclusion of dependencies because you think
some upstream user would need those. Dependencies which got left around after
various refactorings. Dependencies which are marked as provided but not
actually provided/etc.. I envision various devs playing dependency bingo until
the build works. We are fighting maven and its great powers instead of letting
it do the hard work for us.
The PR associated with the above will be very large as it unwinds these various
complexities. The result should be far more maintainable and easier to reason
over. It heavily leverages the BOM pattern both for external dependencies but
also within NiFi. It moves modules to their more logical/independent spot. It
goes module by module and does full dependency analysis to eliminate any
incorrect dependencies either from things being listed as dependencies which
aren't actually used as well as things which override parent managed versions
when it should not as well.
> nifi-nar-bundle has improper dependencies - the full source tree needs
> dependency cleanup and management
> --------------------------------------------------------------------------------------------------------
>
> Key: NIFI-12998
> URL: https://issues.apache.org/jira/browse/NIFI-12998
> Project: Apache NiFi
> Issue Type: Task
> Affects Versions: 2.0.0-M2
> Reporter: Joe Witt
> Assignee: Joe Witt
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> found in nifi-nar-bundles pom
> <dependency>
> <groupId>com.maxmind.geoip2</groupId>
> <artifactId>geoip2</artifactId>
> <version>4.2.0</version>
> </dependency>
> This should not be here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
