Hi, following up on my last email I've implemented both changes I was referring to.
First I've added the the same integration tests, that dockermaven already uses to cover the dockerhub image as well. Unfortunately for this I had to use the latest public artifact (1.6.0) since only that is available publicly. I've added it to the existing pull request which I look forward to be merged very soon, to get into 1.7.0: https://github.com/apache/nifi/pull/2747/commits/0c4f9a0ba4b64e258386ef3526c7b44b1f4ba9f7 Sidenote: with the latest snapshot build NiFi seems to bind on all available interfaces instead of using only the primary by default, as it did in 1.6.0 which I'm not sure is intentional or not. If not I need to look deeper what's happening. Second is what I mentioned as a theoretical option, to host the locally built snapshot with it's checksum for the Docker build to consume. This way we can use the dockerhub Dockerfile with locally built NiFi artifacts easily with all it's advantages, but currently it only works on Docker for Mac and Docker for Windows 18.03 onwards. I've implemented it using a simple busybox container that is started before the Docker build and hosts the content of the target folder locally where the NiFi artifact and checksum are located: https://github.com/pepov/nifi/commit/a1d9d8020fb579a50be51e67d98224cc86d985cc I did not add this to the previous pull request, because I don't want to hold it back. However it adds very little complexity to the Dockerfile itself (one extra build arg), instead some tooling around the package and integration-test phases of the docker profile in the nifi-docker module. I know it's not ideal that portability is not guaranteed currently (bash, curl, docker all can have their issues), but I hope it is only temporary and as more people start to use it, it will converge to a fully multiplatform solution. Until then the dockermaven modul can stay of course to be available for legacy usage. Any feedback would be appreciated! Cheers, Peter On Thu, Jun 7, 2018 at 4:25 PM Peter Wilcsinszky <peterwilcsins...@gmail.com> wrote: > Hi, > > Disclaimer: I've been working with the nifi-docker module for only a > couple of weeks now so I'm relatively new to this. > Unfortunately there's no way I'm aware of right now using a pure > Dockerfile to implement a build time switch for cases like this where you > have a a) locally built binary b) remote artifact. > There is the ADD [1] instruction that combines something like that, but it > falls short when used with a custom user and a big artifact, like in our > case. The problem with that is that even if you provide the nifi user with > --chown flag, docker uses that only to set the ownership of the downloaded > archive, but then the extraction happens and ownership info will be used > based on whatever was set in the archive [2]. > Given that, you will need one extra instruction to set the ownership of > the extracted files and boom, you now have double the size of your image > [3]. This is why it is wise to do the download and chown in one > instruction, like in the dockerhub Dockerfile [4] > But back to scenario a) You have the files built with maven locally, but > you can't use curl to download it, because it is not hosted anywhere. While > I'm writing this I can think of a theoretical option to implement something > to host the file locally for the docker container to consume. With that we > could avoid the separate maven Dockerfile, but until then we are left with > the ADD instruction, following with a chown that makes that image almost > 3G... > > For now I'm adding some integration tests for both scenarios [5], but > would be glad to hear others feedback as well! > > [1] https://docs.docker.com/engine/reference/builder/#add > [2] https://github.com/moby/moby/issues/35525#issuecomment-344908030 > [3] https://github.com/moby/moby/issues/22810#issuecomment-220021580 > [4] > https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/Dockerfile#L45 > [5] https://github.com/apache/nifi/pull/2747 > > On Thu, Jun 7, 2018 at 3:37 PM Jeremy Dyer <jdy...@gmail.com> wrote: > >> Sivaprasanna - They are very similar but different. Exists to primarily >> assist with speeding up a developers dev cycle. Its purpose is to allow >> you >> to make source code changes to NiFi, build with maven (including docker >> image), and then run the container with your latest changes. So in a sense >> you can think of the docker maven module as providing you with the latest >> from master. The dockerhub module on the other hand is used to build an >> image for a stable Apache validated release. The current dockerhub build >> will pull down Apache NiFi 1.6 from the Apache mirrors. >> >> TLDR: In cases outside of development you should use the dockerhub one. >> >> Thanks, >> Jeremy Dyer >> >> On Thu, Jun 7, 2018 at 9:09 AM, Sivaprasanna <sivaprasanna...@gmail.com> >> wrote: >> >> > Team, >> > >> > In the project's nifi-docker >> > <https://github.com/apache/nifi/tree/master/nifi-docker> module, we >> have >> > two things: dockerhub and dockermaven. Are they one and the same or they >> > are there for different purposes? I have to warn you I don't know much >> > about 'dockerfile-maven-plugin' so some light on this might help me >> > understand the differences (if at all). >> > >> > Thanks, >> > Sivaprasanna >> > >> >
