Apologies for my ignorance in this effort -- I haven't been following it particularly closely -- but has there been any discussion in integrating with existing solutions for retrieving artifacts, versus rolling out yet another artifact repository? In particular, supporting Maven repository spec would seem to make a lot of sense here.
On Fri, Sep 27, 2019 at 9:41 AM Joseph Thweatt <[email protected]> wrote: > Hi Erik, this tool currently works by looking at an existing flow and > seeing which nars are being used by it. So I think if you are using > something like a PRODUCTION nar in your flow, then this would try to pull > that version in. > > On Fri, Sep 27, 2019 at 12:33 PM Erik Anderson <[email protected]> wrote: > > > This is important for our usages too. We dont want to redeploy NiFi > > containers just because we added a new internal systems NAR file. > > > > Hopefully you will allow tags/versioning on the NAR, like LATEST or > > PRODUCTION vs DEVELOPMENT, v1.0 or v2.1.43. > > > > This way we can test new(er) NAR's without using separate NiFi systems. > > Even test 2 identical flows with different NAR versions. > > > > Erik Anderson > > Bloomberg > > > > On Fri, Sep 27, 2019, at 12:07 PM, Joe Witt wrote: > > > Joseph > > > > > > Cool - this is precisely a key part of the motivation behind the NiFi > Reg > > > effort and all the work to make extensions sourced from the registry > > > on-demand. As a result we'll no longer package most (perhaps even any) > > > nars with a nifi distribution going forward. We've been steadily > making > > > progress on this front. It would be good for you to collaborate in > those > > > efforts if you're interested. The driver here is not docker but rather > > all > > > forms of nifi consumption/deployment models but the docker world would > > > certainly benefit. > > > > > > Thanks > > > > > > On Fri, Sep 27, 2019 at 12:01 PM Joseph Thweatt < > [email protected] > > > > > > wrote: > > > > > > > tl:dr I made this to make the NiFi docker image smaller, let me know > if > > > > it's useful: https://github.com/josephthweatt/skinifi > > > > > > > > Hello everyone, > > > > > > > > My team has been been using NiFi for a couple months now and we have > > > > recently begun adding the docker image into a stack. I noticed that > > NiFi's > > > > docker image is pretty big (1.91 GB) and takes a while to download > as a > > > > result. Adding to that, the image itself is only a part of the > > multistage > > > > build. After adding custom processors we were actually looking at > > something > > > > closer to 2.8 GB, making itesting/building/downloading a nightmare. > > > > > > > > The main reason the image is so large is that the lib is full of nar > > files, > > > > the majority of which are never used. Those same nars are then > > duplicated > > > > to the work directory when nifi is running, making it even larger. > > > > > > > > My solution to this was the github project above. Basically if you > > provide > > > > it a Flow from a registry or a template it will find which nars are > > needed > > > > to run nifi and pack them into a smaller image. A bare-bones image > > (NiFi > > > > with only the processors needed to run without breaking) is about 680 > > MB > > > > and in my team's case we got the file size reduced to 1.2 GB. > > Integration > > > > tests for our stack also halved as a result, from 35 mins down to 17. > > > > > > > > I'll probably continue to add some useful features that I feel we > > need, but > > > > I figured I should put this out there if anyone else wanted to try > it. > > > > > > > > Suggestions are also appreciated! > > > > > > > > > >
