Hi Jon, To clarify, I'm not actually an active developer on nifi itself. At present this tool is something I was doing on my own in order to speed up my team development process.
I actually have an issue on the repo right now to turn this into a maven plugin. Personally, I like the idea of integrating this project into something managed by the larger nifi community if possible. ᐧ On Fri, Sep 27, 2019 at 12:56 PM Jon Logan <[email protected]> wrote: > Apologies for my ignorance in this effort -- I haven't been following it > particularly closely -- but has there been any discussion in integrating > with existing solutions for retrieving artifacts, versus rolling out yet > another artifact repository? In particular, supporting Maven repository > spec would seem to make a lot of sense here. > > > On Fri, Sep 27, 2019 at 9:41 AM Joseph Thweatt <[email protected]> > wrote: > > > Hi Erik, this tool currently works by looking at an existing flow and > > seeing which nars are being used by it. So I think if you are using > > something like a PRODUCTION nar in your flow, then this would try to pull > > that version in. > > > > On Fri, Sep 27, 2019 at 12:33 PM Erik Anderson <[email protected]> > wrote: > > > > > This is important for our usages too. We dont want to redeploy NiFi > > > containers just because we added a new internal systems NAR file. > > > > > > Hopefully you will allow tags/versioning on the NAR, like LATEST or > > > PRODUCTION vs DEVELOPMENT, v1.0 or v2.1.43. > > > > > > This way we can test new(er) NAR's without using separate NiFi systems. > > > Even test 2 identical flows with different NAR versions. > > > > > > Erik Anderson > > > Bloomberg > > > > > > On Fri, Sep 27, 2019, at 12:07 PM, Joe Witt wrote: > > > > Joseph > > > > > > > > Cool - this is precisely a key part of the motivation behind the NiFi > > Reg > > > > effort and all the work to make extensions sourced from the registry > > > > on-demand. As a result we'll no longer package most (perhaps even > any) > > > > nars with a nifi distribution going forward. We've been steadily > > making > > > > progress on this front. It would be good for you to collaborate in > > those > > > > efforts if you're interested. The driver here is not docker but > rather > > > all > > > > forms of nifi consumption/deployment models but the docker world > would > > > > certainly benefit. > > > > > > > > Thanks > > > > > > > > On Fri, Sep 27, 2019 at 12:01 PM Joseph Thweatt < > > [email protected] > > > > > > > > wrote: > > > > > > > > > tl:dr I made this to make the NiFi docker image smaller, let me > know > > if > > > > > it's useful: https://github.com/josephthweatt/skinifi > > > > > > > > > > Hello everyone, > > > > > > > > > > My team has been been using NiFi for a couple months now and we > have > > > > > recently begun adding the docker image into a stack. I noticed that > > > NiFi's > > > > > docker image is pretty big (1.91 GB) and takes a while to download > > as a > > > > > result. Adding to that, the image itself is only a part of the > > > multistage > > > > > build. After adding custom processors we were actually looking at > > > something > > > > > closer to 2.8 GB, making itesting/building/downloading a nightmare. > > > > > > > > > > The main reason the image is so large is that the lib is full of > nar > > > files, > > > > > the majority of which are never used. Those same nars are then > > > duplicated > > > > > to the work directory when nifi is running, making it even larger. > > > > > > > > > > My solution to this was the github project above. Basically if you > > > provide > > > > > it a Flow from a registry or a template it will find which nars are > > > needed > > > > > to run nifi and pack them into a smaller image. A bare-bones image > > > (NiFi > > > > > with only the processors needed to run without breaking) is about > 680 > > > MB > > > > > and in my team's case we got the file size reduced to 1.2 GB. > > > Integration > > > > > tests for our stack also halved as a result, from 35 mins down to > 17. > > > > > > > > > > I'll probably continue to add some useful features that I feel we > > > need, but > > > > > I figured I should put this out there if anyone else wanted to try > > it. > > > > > > > > > > Suggestions are also appreciated! > > > > > > > > > > > > > > >
