I think in light of the fact that the bcprov item was an accidental inclusion and not explicitly meant to be handled as such in the framework reorganizing seems like the best option as well. The original intent of the PR was to minimize that footprint where we were double dipping given the inclusion of nifi-processor-utils. However, if we manage to shuffle items around a bit, the net size reduction could ultimately be the same. Most importantly, from a maintenance perspective, bundles will only include the library in the NARs that need it which allows that dependency graph to be significantly more comprehensible than the scope tweaking originally presented.
Will this also work to adjust the dependencies of nifi-processor-utils (and potentially nifi-security-utils)? Currently, the archetype also provides nifi-processor-utils, which could lead to similar conflicts for folks that were looking to make use of different versions of JARs within a given NAR. I think the listing Joe provided in the ticket is great and we should be mindful of any deltas while performing each release. For the reasons highlighted in this chain and on that issue, one jar inclusion could be quite detrimental and cause some difficult issues to those that have created extensions. On Mon, Oct 31, 2016 at 9:39 PM, Joe Witt <[email protected]> wrote: > So...I might be a bit of a flip flopper... > > Had some off-list discussions with a couple people and they really > disliked this idea. They, rightfully, pointed out how this > establishes a problematic precedent, how it is really only being done > to save space, and that while perhaps not able to pair down to 200MB > we can likely do more analysis of the dependency graph to improve the > situation. The real issue that kicked this off was the finding that > bcprov (and commons-lang3) had made their way into the root of our > classloader (which impacts all classloaders) and this was introduced > with the important NIFI-1831 work. So I've created [1] to address > that and return the application classloader to the minimal set of jars > necessary which should basically be the runtime application launcher, > the logging libraries, and the nifi-api. > > The ultimate solution to addressing the distribution size concerns > will be to address it via the extension registry. > > [1] https://issues.apache.org/jira/browse/NIFI-2971 > > Thanks > Joe > > On Mon, Oct 31, 2016 at 8:51 PM, Andre <[email protected]> wrote: > > +1 as well. > > > > On Tue, Nov 1, 2016 at 2:41 AM, Joe Witt <[email protected]> wrote: > > > >> Team, > >> > >> In the 1.x line bcprov made its way into the root of the classpath/lib > >> folder to support the encrypted sensitive properties features. This > >> can be reworked to isolate it a bit more if we need to. > >> > >> However, in [1] it has been observed that we have bcprov dependencies > >> in about 53 places with each instance taking about 4MB of space for a > >> total hit of about 200MB of bcprov. > >> > >> [1] proposes to make bcprov-jdk15on and bcpkix-jdk15on part of the > >> core provided list of things right along side the > >> logback/slf4j/logging interfaces. This means all things will have > >> access to these going forward and it means it impacts our version > >> compatibility. Given that it would be a standard/provided thing if we > >> want to upgrade versions or swap it out with something else we could > >> break extensions that then chose to depend on it. We are not in > >> control of the bcprov api just like we're not in control of the > >> logging APIs. If there was some important security related fix we > >> needed from bcprov but changing also pulled in api changes for them it > >> could break our extensions. > >> > >> Even with all this said, given the nature, importance, and size > >> benefit, I am in favor of NIFI-2954. But, would like to highlight > >> this in case others have perspective they'd like to share. > >> > >> [1] https://issues.apache.org/jira/browse/NIFI-2954 > >> > >> Thanks > >> Joe > >> >
