Realistically we'll have to do the reorganization and removal of optional NARs. Regardless of the reorganization, the convenience binary has grown too large. Given that, it probably makes the most sense to pull out an initial set of optional NARs from the convenience binary this release, and implement the reorganization in the following release.
On Wed, Aug 21, 2019 at 10:06 PM Mike Thomsen <[email protected]> wrote: > My impression was that the reorganization initiative has enough steps that > it might actually be wise for us to at least take on a chunk of them now so > we don't end up with a release that suddenly feels to NiFi users the way > Java 9 felt to Java 7 and 8 users. > > On Wed, Aug 21, 2019 at 9:59 PM Jeff <[email protected]> wrote: > > > How motivated could we be to do the reorganization of the NiFi repository > > before the 1.10.0 release? It sounds like we have a few paths to get the > > resulting convenience binary down below the size limit. If we don't do > the > > reorganization for this upcoming release, we should make it a top > priority > > for the following release. > > > > On Wed, Aug 21, 2019 at 6:22 PM Mike Thomsen <[email protected]> > > wrote: > > > > > Another factor on why that would be a good idea: I might soon have to > > pivot > > > and do the R&D on adding dgraph support to graph bundle. So it's not > > > altogether unlikely that it might need to be refactored to make room > for > > > other graph tech. > > > > > > On Wed, Aug 21, 2019 at 6:20 PM Mike Thomsen <[email protected]> > > > wrote: > > > > > > > Go ahead and remove the whole graph bundle from the assembly. I would > > > > recommend cutting a release of it separately and putting it up on > > > GitHub's > > > > releases listing if that's a possibility for us/INFRA w/ GitHub. Most > > of > > > > our potential graph users are savvy enough that if add a few steps, I > > > don't > > > > see it causing any grief on them getting it stood up and giving us > > > feedback. > > > > > > > > Might be a good idea also to add a "full-build" profile to the > assembly > > > so > > > > that we can throw the whole kitchen sink into an unofficial build if > we > > > > build it ourselves for someone else. > > > > > > > > On Wed, Aug 21, 2019 at 3:09 PM Joe Witt <[email protected]> wrote: > > > > > > > >> Bryan > > > >> > > > >> I agree with all of that. What does that get us to? > > > >> > > > >> Thanks > > > >> > > > >> On Wed, Aug 21, 2019 at 3:03 PM Bryan Bende <[email protected]> > wrote: > > > >> > > > >> > I would vote to make nifi-flume-nar optional, and it looks like > > > >> > nifi-other-graph-services-nar might be new since last release, so > > > >> > since that is in the top 10 and not released yet, it might also > be a > > > >> > good candidate (not downplaying the usefulness of anything in that > > > >> > NAR). > > > >> > > > > >> > I would also think we could consider the nifi-kafka-0-8-nar since > > > >> > Kafka 0.8 is quite old at this point, and we already have other > > Kafka > > > >> > NARs for 0.9, 0.10, 0.11, 1.0, and 2.0. Might even consider > > dropping a > > > >> > few more versions from default assembly. > > > >> > > > > >> > On Wed, Aug 21, 2019 at 2:45 PM Aldrin Piri <[email protected]> > > > wrote: > > > >> > > > > > >> > > Hi folks, > > > >> > > > > > >> > > Doing a recent PR review and build, it seems that master has > > amassed > > > >> some > > > >> > > additional size since our 1.9.2 release approaching 200MB. > > > >> > > > > > >> > > Unfortunately, this is problematic and needs to be addressed in > > > >> advance > > > >> > of > > > >> > > our 1.10 release. INFRA has been more than helpful making one > off > > > >> > > exceptions [1][2] for the larger assembly to get published to > the > > > ASF > > > >> > > repository and its associated mirrors, but another release that > is > > > >> even > > > >> > > larger is not something we can allow. In a Linux environment, > the > > > >> master > > > >> > > build reports in at 1575671276 which puts us over the hard limit > > > >> > > highlighted in [2]. > > > >> > > > > > >> > > We had a prior community discussion [3] about splitting the > > > framework > > > >> and > > > >> > > extension repos and I am hoping to revive that discussion, in > > part. > > > >> We > > > >> > > certainly know what our longer term goals and ambitions are but > > > need a > > > >> > fix > > > >> > > in the interim. In the current state, we will not be able to > make > > > our > > > >> > > convenience binaries available at the conclusion of the release > > > >> process. > > > >> > > > > > >> > > At minimum we should evaluate which bundles are eligible to get > > > >> treated > > > >> > as > > > >> > > optional dependencies and only enabled via profile, much like > the > > > work > > > >> > that > > > >> > > has occurred surrounding some of our other, hefty NARs. [4] A > > > listing > > > >> of > > > >> > > the top 50 largest NARs, excluding framework and standard, is > > > >> available > > > >> > in > > > >> > > a gist [5]. The nifi-media-nar looks to be a good initial > > candidate > > > >> for > > > >> > > exclusion. > > > >> > > > > > >> > > Thanks for your consideration! > > > >> > > > > > >> > > --aldrin > > > >> > > > > > >> > > [1] https://issues.apache.org/jira/browse/INFRA-11252 > > > >> > > [2] https://issues.apache.org/jira/browse/INFRA-15816 > > > >> > > [3] > > > >> > > > > > >> > > > > >> > > > > > > https://lists.apache.org/thread.html/939a7630a2e32594cd10444e48b7a1321fd9ce51834d911a8c04b6a9@ > > > >> > <dev.nifi.apache.org> > > > >> > > [4] > > > >> > > > > > >> > > > > >> > > > > > > https://github.com/apache/nifi/blob/master/nifi-assembly/pom.xml#L807-L875 > > > >> > > [5] > > https://gist.github.com/apiri/4d9a02f9f6b46867b601956df83b6d8c > > > >> > > > > >> > > > > > > > > > >
