James, (adding comments as requested - but from here it looks like there is enough momentum from the team to resolve the core issue)
Firstly, it looks like a custom role can be added to the repo to allow GitHub Actions (ie Jib) to push new images to DockerHub. Some new GitHub Action code will have to be added to do the "push", but that is trivial. Both of these tasks can be done quickly and easily, and solves the short term problem of getting current Fineract images to DockerHub. This is an immediate win and resolves the most immediate issue. The second issue that Fineract may wish to address is whether or not to include a Dockerfile in the public repo. The docker-compose file (pointing to DockerHub) that is currently included in the Fineract repo *will work*, allowing new users to get setup quickly. However users will have no way to create new local versions of the Fineract container image locally without installing the complete java (Jib) toolchain. This may not be an issue at all if most users already have a local java environment. For casual users looking to contribute to Fineract, this may present a bit of friction. Including a Dockerfile in the repo can be done *without replacing Jib*. However, having a visible Dockerfile that is NOT used with the non-visible Jib build process will be confusing. Keeping the Dockerfile up to date will also be harder, since the file will not be the single source of truth. This may or not be an issue for the team. Lastly (and probably the most contentious option) would be to replace the Jib image build tooling with the more common Dockerfile build mechanism. This represents more work from the team, but also increases the transparency from the outside user. This may not be an issue for the team (especially if the number of outside contributors is small). -Todd On Sun, May 5, 2024 at 1:51 PM James Dailey <jamespdai...@gmail.com> wrote: > Victor - thank you for your PR and Zoltan for your comments. > > On Fri, May 3, 2024 at 10:31 AM VICTOR MANUEL ROMERO RODRIGUEZ < > victor.rom...@fintecheando.mx> wrote: > >> James, >> >> I think that this option is also viable: >> >> "For Github Actions we can use a role account and attach the secrets to >> your repository" >> >> At this point there are two options >> >> 1. Apache Infra has to add the secrets to the Apache Fineract repository >> and then we can create/modify github actions for using the existing Jib >> library. >> > This approach maintains the same consistent build but is a bit > non-standard? The problem with option #2 is that there will be effectively > two "builds" from the same source, and they won't match because the Jib > library is in use by active dev teams on the project today. > > >> 2. Merge the https://github.com/apache/fineract/pull/3879 to the develop >> branch in order to allow Dockerhub to build/publish a new docker image. >> > This approach returns the project to a more standard approach? It does > not require special action by Infra but creates a separate and potentially > inconsistent build. > >> >> Any other option or discussion about the solution for having an updated >> Apache Fineract image published on Dockerhub is welcome. >> > agreed - last call for comments on which of these. I'm leaning toward the > second because it requires no action by Infra and is the shortest path to > getting a new image at DockerHUB. > > Todd- can you comment? > > >> >> Best regards >> >> Victor >> >> Regards >> >> Víctor Romero >> >> El vie, 3 may 2024 a las 8:56, James Dailey (<jdai...@apache.org>) >> escribió: >> >>> As this relates to this thread but was over on infra users... >>> >>> >>> ---------- Forwarded message --------- >>> From: Gavin McDonald <gmcdon...@apache.org> >>> Date: Sun, Feb 18, 2024 at 12:24 AM >>> Subject: Re: Docker help >>> To: James Dailey <jdai...@apache.org> >>> Cc: Users <us...@infra.apache.org> >>> >>> >>> Hi James. >>> >>> >>> >>> On Sun, Feb 18, 2024 at 3:00 AM James Dailey <jdai...@apache.org> wrote: >>> >>>> Infra - >>>> >>>> Can you confirm that we can use other processes to push to >>>> apache DockerHUB? >>>> >>> >>> Current supported methods are via Github Actions or Jenkins or locally >>> via your own credentials. >>> >>> For Github Actions we can use a role account and attach the secrets to >>> your repository, or you >>> can provide your own secrets for us to add to your repository >>> >>> For Jenkins we have a role account that we provide access to push to >>> your repository. >>> >>> Committers could also use a settings.xml with this plugin and use their >>> own credentials, we just need >>> to ensure they have push access to Dockerhub. >>> >>> There may also be other methods not explored. >>> >>> See also: >>> https://github.com/GoogleContainerTools/jib/tree/master/jib-maven-plugin#authentication-methods >>> >>> HTH >>> >>>> >>>> When I opened a ticket about this, I was told we need a dockerfile at >>>> the root. >>>> >>>> Can we use "jib-maven-plugin to publish the image to Dockerhub". ? >>>> Can we get credentials ? >>>> >>>> James >>>> >>>> >>>> ---------- Forwarded message --------- >>>> From: Arnold Galovics <arn...@apache.org> >>>> Date: Sun, Feb 11, 2024 at 10:45 PM >>>> Subject: Re: Docker help >>>> To: <dev@fineract.apache.org> >>>> >>>> >>>> James, >>>> >>>> This is the out-of-the box solution from DockerHub which definitely >>>> won't work without a Dockerfile. Though that doesn't mean it's the only way >>>> to build a docker image; as I stated in my previous email. >>>> >>>> Best, >>>> Arnold >>>> >>>> On Mon, Feb 12, 2024 at 7:43 AM James Dailey <jamespdai...@gmail.com> >>>> wrote: >>>> >>>>> On DockerHUB the build fails because there is no dockerfile. >>>>> https://hub.docker.com/r/apache/fineract >>>>> >>>>> 2024-02-08T13:12:27Z Building in Docker Cloud's infrastructure... >>>>> 2024-02-08T13:12:28Z Cloning into '.'... >>>>> 2024-02-08T13:12:28Z Warning: Permanently added the RSA host key for >>>>> IP address '140.82.114.4' to the list of known hosts. >>>>> 2024-02-08T13:12:48Z Reset branch 'develop' >>>>> 2024-02-08T13:12:48Z Your branch is up to date with 'origin/develop'. >>>>> 2024-02-08T13:12:48Z Dockerfile not found at ./Dockerfile >>>>> >>>>> >>>>> Let's discuss on slack and revert back here. >>>>> >>>>> My intention is to either DELETE the DockerHUB repo or to get this >>>>> working. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Sun, Feb 11, 2024 at 10:14 PM Arnold Galovics <arn...@apache.org> >>>>> wrote: >>>>> >>>>>> Hi Zoltan, James, >>>>>> >>>>>> Just to reflect on your points: >>>>>> 1) Let's not do such a radical change unless we absolutely need to >>>>>> 2) I'm not sure what's the issue here, please explain. We already >>>>>> have docker builds in our pipeline via GitHub Actions (using their >>>>>> runners), the only missing piece is to do a docker push. >>>>>> >>>>>> We need the credentials to be able to do a docker push, alter the >>>>>> pipeline and that's all. >>>>>> >>>>>> If the only thing preventing us from doing this is to keep asking the >>>>>> infra team for the creds, let's pursue them instead of making such an >>>>>> unnecessary change. >>>>>> >>>>>> Arnold >>>>>> >>>>>> On Mon, Feb 12, 2024 at 3:30 AM James Dailey <jamespdai...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Thanks Zoltan >>>>>>> >>>>>>> Micheal - can you please comment on this discussion? As this >>>>>>> relates to the Google deployment that you put in place? Question! >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sun, Feb 11, 2024 at 6:27 PM Zoltan Mezei <zoltan.me...@zz-it.hu> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I think the real issue here is that we use GoogleContainerTools's >>>>>>>> Jib as the build mechanism. It works entirely without a Dockerfile. And >>>>>>>> unfortunately Dockerhub's Automated Builds doesn't support building >>>>>>>> without >>>>>>>> a Dockerfile. :-( >>>>>>>> >>>>>>>> We have two ways to move forward: >>>>>>>> >>>>>>>> 1. Replace the Jib build with a more traditional, Dockerfile-based >>>>>>>> approach. This would be a quite large change of how Fineract is built >>>>>>>> and >>>>>>>> the consequences need to be explored - but it's definitely doable. >>>>>>>> 2. Stick with the Jib build, but don't rely on >>>>>>>> Dockerhub's Automated Builds, but some other build tools like >>>>>>>> jib-maven-plugin to publish the image to Dockerhub. This could also >>>>>>>> work, >>>>>>>> but it requires a build server that I'm not sure we have. >>>>>>>> >>>>>>>> I can try to create a traditional Dockerfile, but it will be >>>>>>>> different from what Jib can produce, so this might lead to regressions. >>>>>>>> >>>>>>>> Want me to try this approach next week? >>>>>>>> >>>>>>>> Kind regards, >>>>>>>> Zoltan >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Feb 11, 2024 at 8:16 AM James Dailey < >>>>>>>> jamespdai...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Victor - my read of the docs is that the default “build rule “ >>>>>>>>> points to master or main but we can also use dev. In fact that’s what >>>>>>>>> is >>>>>>>>> already there in dockerHUB for our project. >>>>>>>>> >>>>>>>>> I think a proper dockerfile in dev branch should be fine. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> James >>>>>>>>> >>>>>>>>> On Fri, Feb 9, 2024 at 7:47 PM VICTOR MANUEL ROMERO RODRIGUEZ < >>>>>>>>> victor.rom...@fintecheando.mx> wrote: >>>>>>>>> >>>>>>>>>> Reading the dockerhub docs, I think we can do the following: >>>>>>>>>> >>>>>>>>>> 1. Create a master branch from develop branch >>>>>>>>>> 2. Add the Dockerfile (and some scripting on it for handling the >>>>>>>>>> versions) on master branch >>>>>>>>>> 3. Dockerhub will use the dockerfile (and its scripts) from the >>>>>>>>>> master branch >>>>>>>>>> 4. Create github action for keeping in sync develop with master, >>>>>>>>>> so then it will push the changes to the master branch everytime the >>>>>>>>>> develop >>>>>>>>>> branch has a commit on it, then the dockerhub will publish it as the >>>>>>>>>> latest >>>>>>>>>> version. >>>>>>>>>> >>>>>>>>>> Or... we can be more standard >>>>>>>>>> >>>>>>>>>> 1. Rename develop to master >>>>>>>>>> 2. Add a Dockerfile template (and some scripting on it for >>>>>>>>>> handling the versions) on master branch >>>>>>>>>> 3. Dockerhub will use the dockerfile (and its scripts) from the >>>>>>>>>> master branch >>>>>>>>>> 4. Everytime a new commit or tag is created, the dockerhub will >>>>>>>>>> publish it as the latest/specific version. >>>>>>>>>> >>>>>>>>>> What do you think? >>>>>>>>>> >>>>>>>>>> Dockerhub automated builds info: >>>>>>>>>> https://docs.docker.com/docker-hub/builds >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> El vie, 9 feb 2024 a las 20:34, James Dailey (< >>>>>>>>>> jamespdai...@gmail.com>) escribió: >>>>>>>>>> >>>>>>>>>>> Victor - I was trying to go down that path as well, as that is >>>>>>>>>>> the error thrown and the suggestion at DockerHUB. However, to add >>>>>>>>>>> the key >>>>>>>>>>> to the git hub requires access and the git is controlled by Apache >>>>>>>>>>> Infra. >>>>>>>>>>> I asked infra@a.o. about that since, again, that is what >>>>>>>>>>> DockerHUB had documented. Unfortunately, I think infra has it >>>>>>>>>>> setup a >>>>>>>>>>> specific way to allow all of the projects to publish to the Apache >>>>>>>>>>> DockerHUB so that route would appear to be blocked. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Feb 9, 2024 at 4:04 PM VICTOR MANUEL ROMERO RODRIGUEZ < >>>>>>>>>>> victor.rom...@fintecheando.mx> wrote: >>>>>>>>>>> >>>>>>>>>>>> For making it work without a Dockerfile the credentials of the >>>>>>>>>>>> docker hub account are requiered. >>>>>>>>>>>> >>>>>>>>>>>> If they are set in the git repository, a github action can be >>>>>>>>>>>> enabled for this task. >>>>>>>>>>>> >>>>>>>>>>>> Regards >>>>>>>>>>>> >>>>>>>>>>>> El vie., 9 de febrero de 2024 4:45 p. m., < >>>>>>>>>>>> jamespdai...@gmail.com> escribió: >>>>>>>>>>>> >>>>>>>>>>>>> I've re-opened >>>>>>>>>>>>> https://issues.apache.org/jira/browse/FINERACT-1164 >>>>>>>>>>>>> >>>>>>>>>>>>> This ticket is to enable the build at DockerHUB to work. For >>>>>>>>>>>>> the past two years ++ the Build has failed. >>>>>>>>>>>>> >>>>>>>>>>>>> https://hub.docker.com/r/apache/fineract >>>>>>>>>>>>> This docker account is held by Apache and the Fineract project >>>>>>>>>>>>> is responsible for the content. >>>>>>>>>>>>> >>>>>>>>>>>>> The dockerHUB has an "auto build" concept so that every >>>>>>>>>>>>> committed change on Dev leads to a new deployment. >>>>>>>>>>>>> >>>>>>>>>>>>> The build is actually failing or not running because we >>>>>>>>>>>>> have removed the dockerbuild file from the root. That is as far >>>>>>>>>>>>> as I've >>>>>>>>>>>>> gotten. I suspect we had good reasons for that at the time. >>>>>>>>>>>>> >>>>>>>>>>>>> Anyway, I would also say that if we cannot get the Docker >>>>>>>>>>>>> build to work THEN we should take this down. Our standard is to >>>>>>>>>>>>> only >>>>>>>>>>>>> support and distribute publicly the last two releases. This build >>>>>>>>>>>>> is really >>>>>>>>>>>>> old, has unfixed CVEs, and is being downloaded in large numbers. >>>>>>>>>>>>> (no idea >>>>>>>>>>>>> why) >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> James >>>>>>>>>>>>> >>>>>>>>>>>> >>> >>> -- >>> >>> >>> *Gavin McDonald - * >>> Systems Administrator, ASF Infrastructure Team >>> V.P Travel Assistance Committee >>> >>> https://tac.apache.org - Applications now open for Community Over Code >>> 2024 >>> in Bratislava, Slovakia. Don't delay, apply today! >>> >>>
