It seems like the separate release process is the one of preference. I have created a JIRA issue (https://issues.apache.org/jira/browse/NIFI-4425) to capture some of the sentiment from this thread and to subsequently track the efforts once we start executing upon them. Feel free to add any additional comments/thoughts to that ticket.
Thanks! --aldrin On Thu, Sep 21, 2017 at 1:44 PM, Adam Taft <[email protected]> wrote: > Aldrin, > > +1 to separate repository (bullet #2). The basic premise that Docker > releases should happen separate from the main distribution is spot on. I > think a separate repository would help keep this separation. > > I tend to believe that the future of NiFi distributions will be via > containerization. Making the Docker components somewhat a standalone > initiative will help drive changes and innovation in this area. I'd like > to help see Docker become a first-class citizen for distributing, running > and upgrading NiFi. > > Thanks, > > Adam > > > > On Thu, Sep 21, 2017 at 9:11 AM, Aldrin Piri <[email protected]> wrote: > > > Hey folks, > > > > ** This message turned out to be more detailed than anticipated. In > > summary, I propose consolidating Docker/container work with a separate > > release process outside of the repository they are packaging. Full > > thoughts and background follow. Any input would be appreciated! > > > > --- > > > > I've been working through providing some additional Docker capabilities > for > > the project and wanted to share some thoughts as well as possible plans > to > > help us be a bit more nimble and responsive to curating Dockerfiles and > > their respective images on DockerHub. > > > > As a bit of context, we currently have the core NiFi project captured in > > two Dockerfiles, one that is used in conjunction with a Maven plugin for > > creating an image during the NiFi build (dockermaven), and another that > is > > used for building tagged releases on Docker Hub (dockerhub). Both of > these > > artifacts, currently, reside in a nifi-docker project and are activated > via > > Maven profile, (-P docker). > > > > We've seen at times that this is a very coupled process and limits our > > flexibility. For instance, we had an ill-placed 'chown' which caused a > > duplicating layer and causes our image to be doubly large. While this > has > > been remedied, given current release processes, this is included with the > > core nifi release and we have been unable to rectify that issue. > > > > Another issue is a very specific sequence of actions that needs to happen > > with the current release for artifacts to be triggered correctly in > Docker > > Hub. This can be seen in Section 6 of the release guide [1]. While > there > > are ways to rectify this if the timing isn't quite right and/or an error > is > > made, it can impose an additional burden on the INFRA team to facilitate > > these requests as there currently is no capability for PMCs to manage > their > > Docker repositories directly. > > > > Ultimately, I think we should consider a separate release process for > NiFi > > Docker, and any associated efforts that may relate to those files. In > this > > context, NiFi is encompassing of all projects/efforts in the project. > > Additional efforts could comprise of examples of configuring NiFi to be > > secured or clustered, receive data from MiNiFi instances, or using Docker > > Compose or other orchestration frameworks. I have also noticed a number > of > > different areas across our work that are using Docker for integration > > testing purposes. With some planning and coordination, we could likely > > consolidate many of these core resources/templates to allow us to reuse > > them across efforts. > > > > I believe there are two approaches from an organizational standpoint that > > let us execute on the separate release process effectively: > > > > 1.) Condense all Docker artifacts into the current NiFi repository [2]. > We > > update our release for NiFi to exclude the Docker subtree to carry out > our > > normal release flow and provide the build/tooling for the Docker subtree > to > > be released on its own. > > > > 2.) Establish a new git repository to handle Docker and any other > > containerization efforts and migrate all existing resources into a file > > structure that makes sense. > > > > My inclination is toward (2). > > > > Regardless of path chosen above, this frees us to handle updates and > > improvements to container efforts when needed. Any time we wanted to > > release updates to Docker images, we could perform a separate release on > > either the subtree of (1) or the repository of (2) and reference the > > associated latest artifacts of NiFi. > > > > If you've made it this far, thanks for working through the wall of text > and > > would appreciate any thoughts or comments. > > > > [1] http://nifi.apache.org/release-guide.html > > [2] https://git-wip-us.apache.org/repos/asf?p=nifi.git > > >
