I would vote for doing the reorganization after this release because on its own it’s doesn’t really solver this specific problem. We would be moving all the NARs to another git repo, but still need a way to distribute them. Some of the options would be...
A) Pick and choose only some of them to be included in the standard distribution (same thing we are doing now and doesn’t require reorganization). B) Create a couple of different NAR assemblies that each on their own don’t exceed the size limit, and could easily be dropped into the base distribution (In previous discussions no one could agree on how to do this). C) Release each NAR independently to a central extension registry to be easily installed to anyone’s NiFi (ultimate goal but doesn’t exist yet). On Wed, Aug 21, 2019 at 10:13 PM Jeff <[email protected]> wrote: > Realistically we'll have to do the reorganization and removal of optional > NARs. Regardless of the reorganization, the convenience binary has grown > too large. Given that, it probably makes the most sense to pull out an > initial set of optional NARs from the convenience binary this release, and > implement the reorganization in the following release. > > On Wed, Aug 21, 2019 at 10:06 PM Mike Thomsen <[email protected]> > wrote: > > > My impression was that the reorganization initiative has enough steps > that > > it might actually be wise for us to at least take on a chunk of them now > so > > we don't end up with a release that suddenly feels to NiFi users the way > > Java 9 felt to Java 7 and 8 users. > > > > On Wed, Aug 21, 2019 at 9:59 PM Jeff <[email protected]> wrote: > > > > > How motivated could we be to do the reorganization of the NiFi > repository > > > before the 1.10.0 release? It sounds like we have a few paths to get > the > > > resulting convenience binary down below the size limit. If we don't do > > the > > > reorganization for this upcoming release, we should make it a top > > priority > > > for the following release. > > > > > > On Wed, Aug 21, 2019 at 6:22 PM Mike Thomsen <[email protected]> > > > wrote: > > > > > > > Another factor on why that would be a good idea: I might soon have to > > > pivot > > > > and do the R&D on adding dgraph support to graph bundle. So it's not > > > > altogether unlikely that it might need to be refactored to make room > > for > > > > other graph tech. > > > > > > > > On Wed, Aug 21, 2019 at 6:20 PM Mike Thomsen <[email protected] > > > > > > wrote: > > > > > > > > > Go ahead and remove the whole graph bundle from the assembly. I > would > > > > > recommend cutting a release of it separately and putting it up on > > > > GitHub's > > > > > releases listing if that's a possibility for us/INFRA w/ GitHub. > Most > > > of > > > > > our potential graph users are savvy enough that if add a few > steps, I > > > > don't > > > > > see it causing any grief on them getting it stood up and giving us > > > > feedback. > > > > > > > > > > Might be a good idea also to add a "full-build" profile to the > > assembly > > > > so > > > > > that we can throw the whole kitchen sink into an unofficial build > if > > we > > > > > build it ourselves for someone else. > > > > > > > > > > On Wed, Aug 21, 2019 at 3:09 PM Joe Witt <[email protected]> > wrote: > > > > > > > > > >> Bryan > > > > >> > > > > >> I agree with all of that. What does that get us to? > > > > >> > > > > >> Thanks > > > > >> > > > > >> On Wed, Aug 21, 2019 at 3:03 PM Bryan Bende <[email protected]> > > wrote: > > > > >> > > > > >> > I would vote to make nifi-flume-nar optional, and it looks like > > > > >> > nifi-other-graph-services-nar might be new since last release, > so > > > > >> > since that is in the top 10 and not released yet, it might also > > be a > > > > >> > good candidate (not downplaying the usefulness of anything in > that > > > > >> > NAR). > > > > >> > > > > > >> > I would also think we could consider the nifi-kafka-0-8-nar > since > > > > >> > Kafka 0.8 is quite old at this point, and we already have other > > > Kafka > > > > >> > NARs for 0.9, 0.10, 0.11, 1.0, and 2.0. Might even consider > > > dropping a > > > > >> > few more versions from default assembly. > > > > >> > > > > > >> > On Wed, Aug 21, 2019 at 2:45 PM Aldrin Piri <[email protected]> > > > > wrote: > > > > >> > > > > > > >> > > Hi folks, > > > > >> > > > > > > >> > > Doing a recent PR review and build, it seems that master has > > > amassed > > > > >> some > > > > >> > > additional size since our 1.9.2 release approaching 200MB. > > > > >> > > > > > > >> > > Unfortunately, this is problematic and needs to be addressed > in > > > > >> advance > > > > >> > of > > > > >> > > our 1.10 release. INFRA has been more than helpful making one > > off > > > > >> > > exceptions [1][2] for the larger assembly to get published to > > the > > > > ASF > > > > >> > > repository and its associated mirrors, but another release > that > > is > > > > >> even > > > > >> > > larger is not something we can allow. In a Linux environment, > > the > > > > >> master > > > > >> > > build reports in at 1575671276 which puts us over the hard > limit > > > > >> > > highlighted in [2]. > > > > >> > > > > > > >> > > We had a prior community discussion [3] about splitting the > > > > framework > > > > >> and > > > > >> > > extension repos and I am hoping to revive that discussion, in > > > part. > > > > >> We > > > > >> > > certainly know what our longer term goals and ambitions are > but > > > > need a > > > > >> > fix > > > > >> > > in the interim. In the current state, we will not be able to > > make > > > > our > > > > >> > > convenience binaries available at the conclusion of the > release > > > > >> process. > > > > >> > > > > > > >> > > At minimum we should evaluate which bundles are eligible to > get > > > > >> treated > > > > >> > as > > > > >> > > optional dependencies and only enabled via profile, much like > > the > > > > work > > > > >> > that > > > > >> > > has occurred surrounding some of our other, hefty NARs. [4] A > > > > listing > > > > >> of > > > > >> > > the top 50 largest NARs, excluding framework and standard, is > > > > >> available > > > > >> > in > > > > >> > > a gist [5]. The nifi-media-nar looks to be a good initial > > > candidate > > > > >> for > > > > >> > > exclusion. > > > > >> > > > > > > >> > > Thanks for your consideration! > > > > >> > > > > > > >> > > --aldrin > > > > >> > > > > > > >> > > [1] https://issues.apache.org/jira/browse/INFRA-11252 > > > > >> > > [2] https://issues.apache.org/jira/browse/INFRA-15816 > > > > >> > > [3] > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > https://lists.apache.org/thread.html/939a7630a2e32594cd10444e48b7a1321fd9ce51834d911a8c04b6a9@ > > > > >> > <dev.nifi.apache.org> > > > > >> > > [4] > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > https://github.com/apache/nifi/blob/master/nifi-assembly/pom.xml#L807-L875 > > > > >> > > [5] > > > https://gist.github.com/apiri/4d9a02f9f6b46867b601956df83b6d8c > > > > >> > > > > > >> > > > > > > > > > > > > > > > -- Sent from Gmail Mobile
