[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004389#comment-16004389 ] Patrick Lucas commented on FLINK-3026: -- We now have [official Docker images|https://hub.docker.com/_/flink/]! Any remaining discussion from this issue should be moved to a new one, as this one is now resolved. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[
https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923956#comment-15923956
]
Patrick Lucas commented on FLINK-3026:
--
Personally I like the idea of having as little code as possible in the
docker-entrypoint.sh script. I think it should be designed to basically work in
all Flink versions going forward. I suppose that would mean that we should have
some scripts on the PATH named 'jobmanager' and 'taskmanager' so that
docker-entrypoint.sh can effectively just be:
{code}
#!/bin/bash -e
# Drop root privs code here
exec "$@"
{code}
And absolutely these builds will be automated—see my note in README.md about
generating the .travis.yml config. I just built some images as a proof of
concept and to think about how tagging will work.
We should include a script like the generate-stackbrew-library.sh one that many
official Dockerfile repos include to generate the file that is referenced by
the
[docker-library/official-images|https://github.com/docker-library/official-images/]
repo to actually build and tag all the right permutations.
And regarding this repo living my personal account, I consider that just a
temporary arrangement until we can get these included in docker-library. You're
welcome to clone the repo if you want to experiment with kicking off builds.
Finally, could you expound a little more about your opinion about the
relationship between a Dockerfile included in apache/flink and these official
images? I'm not sure if I see the benefit of trying to keep it around if we
have official, publicly available, well-documented reference Dockerfiles and
images available. The one argument I see is building a Docker image from your
work tree if you're a developer, but in that case I think it's a lot less
important to maintain it very actively. It could even just be a Dockerfile that
bases itself on the official image and then replaces the Flink release within
with your work tree!
> Publish the flink docker container to the docker registry
> -
>
> Key: FLINK-3026
> URL: https://issues.apache.org/jira/browse/FLINK-3026
> Project: Flink
> Issue Type: Task
> Components: Build System, Docker
>Reporter: Omer Katz
>Assignee: Patrick Lucas
> Labels: Deployment, Docker
>
> There's a dockerfile that can be used to build a docker container already in
> the repository. It'd be awesome to just be able to pull it instead of
> building it ourselves.
> The dockerfile can be found at
> https://github.com/apache/flink/tree/master/flink-contrib/docker-flink
> It also doesn't point to the latest version of Flink which I fixed in
> https://github.com/apache/flink/pull/1366
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906515#comment-15906515 ] Ismaël Mejía commented on FLINK-3026: - I saw the images you built, probably it is a better idea to enable automated builds for the repo so those are rebuilt once we push the changes in github, have you tried this? And, I have another proposition to make, maybe it is a better idea to create a github organization for this, and move the repo there, to end up with something like (docker-flink/docker-flink) instead of your personal account or the dataartisans one, that way we can share the maintainance, and I or other future maintainer can trigger the builds too from docker hub if he has the permissions. Currently I can’t trigger the builds of docker-flink because they only let your own account and the organizations you belong too. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906514#comment-15906514 ] Ismaël Mejía commented on FLINK-3026: - One thing we need to discuss is the inclusion of the start-foreground scripts (btw, sorry if I removed that entrypoint from the docker-flink repo), I think we must not change any file of the release, because this can become a maintainance burden in the future, I prefer to do the least amount of changes possible, and concretely for the start-foreground case I prefer to include this only if it is part of the official distribution, e.g. by backporting this to flink 1.2.1, or start supporting this once it is released in flink 1.3. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906513#comment-15906513 ] Ismaël Mejía commented on FLINK-3026: - Ok, we are going progress fast, that’s great. I agree with you I am 100% convinced now that those should not live in the Apache repo, but I still want to guarantee that the official image does not differ radically from the reference one in the apache repo. My proposition more concretely is that we (the docker image maintainers) guarantee that the order of changes in the future come from the apache/flink repo towards the docker-flink repo, if you see my refactorings in the docker-flink repo go in this direction, they are mostly to make this job easier (but currently they come from the future with your unmerged changes and my non-alpine version). I agree with you, in the future these will be minor changes, even if you look at the history of the image, we have had changes like each release and some small fixes in between. So changing the template should be rare. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906494#comment-15906494 ] Patrick Lucas commented on FLINK-3026: -- I've pushed some of the images built from the Dockerfiles in that repo @[aafb1e4b|https://github.com/patricklucas/docker-flink/commit/aafb1e4bc4a1d6d1fe706cbe457376385e3f8d02] (after which were some changes by [~iemejia] that I want to discuss) to the Docker Hub repo [plucas/flink|https://hub.docker.com/r/plucas/flink/tags/], along with the relevant tags. Specifically, I pushed: * 1.1.4-hadoop27-scala_2.11-alpine * 1.2.0-hadoop26-scala_2.10-alpine * 1.2.0-hadoop26-scala_2.11-alpine * 1.2.0-hadoop27-scala_2.10-alpine * 1.2.0-hadoop27-scala_2.11-alpine to demonstrate various tag permutations, e.g.: * latest-alpine -> 1.2.0-hadoop27-scala_2.11-alpine * 1.1-alpine -> 1.1.4-hadoop27-scala_2.11-alpine * 1.2-hadoop26-alpine -> 1.2.0-hadoop26-scala_2.11-alpine * 1.2-scala_2.10-alpine -> 1.2.0-hadoop27-scala_2.10-alpine > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903317#comment-15903317 ] Patrick Lucas commented on FLINK-3026: -- Yes, I gave you contributor permissions. This repo is really just a stop-gap until we find a permanent home for these scripts, such as in docker-library. I'm fairly convinced that these should be versioned independently of Flink (in a separate repo), as they are updated as a consequence of Flink releases and are not themselves part of a given release. Moreover, the scripts herein may relate to many different "stable" branches of Flink. And I don't really see how it would be feasible to make the contents of this repo somehow "downstream" from the Docker support included in apache/flink as I think they serve different purposes. In apache/flink, having a Dockerfile to package up the current tree would be useful for development, and as a baseline if a user wanted to create their own images from scratch. The docker-flink repo meanwhile encodes the generation of all Dockerfile variants for all "stable" versions of Flink. Finally, I think a tenet of this work (Docker support) going forward should be that the Dockerfile templates really should not change much over time. Instead, we should try to make any changes in functionality in Flink itself, and keep the Dockerfiles (and docker-entrypoint.sh scripts) as simple as possible. If we're doing things right, this code should very rarely change, except when we need to generate Dockerfiles for a new Flink release. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903272#comment-15903272 ] Ismaël Mejía commented on FLINK-3026: - Great, taking a look right now, I will rebase my PR adding the fix I see there, can I push directly in the docker-flink repo ? > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903162#comment-15903162 ] Patrick Lucas commented on FLINK-3026: -- The new [docker-flink|https://github.com/patricklucas/docker-flink/] repo is up. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903127#comment-15903127 ] Ismaël Mejía commented on FLINK-3026: - I suppose most projects do this to avoid having a plethora of repeated dockerfiles where the only difference is the release version number, which makes sense, also because this solves the evolution problem that we will have now (with the start-foreground scripts that are coming in 1.3.0). The only thing that is lost with this approach of having an external repo is the community awareness about the docker images. [~plucas] I like your plan, I would like to have FLINK-6003 merged first (I think this one apart of adding the new non-alpine image it fixes important issues like the resolution of the taskmanagers in the docker-compose file. Afterwards just tell me the exact address of the repo and I will jump in to prepare everything to put the official images in place, > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903128#comment-15903128 ] Ismaël Mejía commented on FLINK-3026: - I wonder still how could we address the issue of community awareness, I have thought about two options: 1. We guarantee that the image in the apache/flink repo is always equivalent to the latest version supported in the docker-flink repo (like this the community is aware and can continue to improve it), so the maintenance of 'latest' will continue in apache. 2. We just remove everything from the docker contrib folder and mention somewhere in the main README that the docker image is maintained in the docker-flink repo. I am obviously pro 1, What do you think ? > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902792#comment-15902792 ] Patrick Lucas commented on FLINK-3026: -- Following from discussion on [#3494|https://github.com/apache/flink/pull/3494], it's apparent that more work is required to provide official Flink Docker images than simply improving the Dockerfile in the Flink repo. I explored the official Docker Hub repos and looked at how the first five Apache projects I saw, [httpd|https://hub.docker.com/_/httpd/], [tomcat|https://hub.docker.com/_/tomcat/], [cassandra|https://hub.docker.com/_/cassandra/], [maven|https://hub.docker.com/_/maven/], and [solr|https://hub.docker.com/_/solr/], have their Dockerfiles hosted. Common between all of them is that they all have a dedicated git repo to host the Dockerfiles, and none are hosted by the Apache project. Three ([httpd|https://github.com/docker-library/httpd], [tomcat|https://github.com/docker-library/tomcat], and [cassandra|https://github.com/docker-library/cassandra]) are hosted within the [docker-library|https://github.com/docker-library] GitHub org, one ([solr|https://github.com/docker-solr/docker-solr]) has a dedicated GitHub org ([docker-solr|https://github.com/docker-solr]), and one ([maven|https://github.com/carlossg/docker-maven]) is hosted within a Maven contributor and ASF member's GitHub account ([carlossg|https://github.com/carlossg]). Adding Flink to the docker-library repo is thus the route most inline with these projects, though there may be some implications I'm not aware of. In the mean time, I will create a repo under [my own GitHub account|https://github.com/patricklucas] using the Apache license called docker-flink and grant [~iemejia] commit access so we can work on the Dockerfiles and supporting scripts. Does this sound like a good starting point [~iemejia] & [~jgrier]? > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Patrick Lucas > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902108#comment-15902108 ] ASF GitHub Bot commented on FLINK-3026: --- Github user iemejia closed the pull request at: https://github.com/apache/flink/pull/3493 > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902107#comment-15902107 ] ASF GitHub Bot commented on FLINK-3026: --- Github user iemejia commented on the issue: https://github.com/apache/flink/pull/3493 I am closing this to create a new one based on the new JIRA > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901677#comment-15901677 ] ASF GitHub Bot commented on FLINK-3026: --- Github user iemejia commented on the issue: https://github.com/apache/flink/pull/3493 I oups not the ubuntu, the non-alpine one :) > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901673#comment-15901673 ] ASF GitHub Bot commented on FLINK-3026: --- Github user iemejia commented on the issue: https://github.com/apache/flink/pull/3493 I will, and add the commits for the ubuntu image, probably I will create a JIRA for this. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901665#comment-15901665 ] ASF GitHub Bot commented on FLINK-3026: --- Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3493 Since this PR is no longer directly related to publishing official Docker images, could you change its title? (See my reply to your comment on #3494) > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901120#comment-15901120 ] ASF GitHub Bot commented on FLINK-3026: --- Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3493 Aha, I see. I didn't follow the chain up further. :) Then yes, I'm generally on board with following this convention. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901102#comment-15901102 ] ASF GitHub Bot commented on FLINK-3026: --- Github user iemejia commented on the issue: https://github.com/apache/flink/pull/3493 Yes, maybe I was not clear enough, my goal is to support the two variants of openjdk that you mention. I was just mentioning ubuntu/debian because this is the base of the non-alpine base container for openjdk. https://github.com/docker-library/openjdk/blob/445f8b8d18d7c61e2ae7fda76d8883b5d51ae0a5/8-jre So we will need some small tweaks and an extra Dockerfile to support both. For other images that support both openjdk base and the alpine version. https://hub.docker.com/_/solr/ https://hub.docker.com/_/jenkins/ https://hub.docker.com/_/tomcat/ > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901099#comment-15901099 ] ASF GitHub Bot commented on FLINK-3026: --- Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3493 I've opened my PR, #3494. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901090#comment-15901090 ] ASF GitHub Bot commented on FLINK-3026: --- Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3493 I took a look through the [top official Docker](https://hub.docker.com/explore/) repos to see what variants they provide, and the first-listed Java-based project is [Elasticsearch](https://hub.docker.com/_/elasticsearch/). They have two variants for each of their supported version, alpine and non-alpine which are based on openjdk:8-jre and openjdk:8-jre-alpine respectively. Can you point me at other official repos basing on debian/ubuntu images? > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901071#comment-15901071 ] ASF GitHub Bot commented on FLINK-3026: --- Github user iemejia commented on the issue: https://github.com/apache/flink/pull/3493 Cool ! Just ping me when you are done and I will rebase to finish the clean ups, I am going to start contacting the docker people to see what else do we need and continue to work in the ubuntu version meanwhile (I think having both is worth even if it was me who slimmed the thing into alpine). > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901052#comment-15901052 ] ASF GitHub Bot commented on FLINK-3026: --- Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3493 Thanks @iemejia. I have a branch building on @jgrier's commits from #3205 which I hope to push before any further changes are made. Hopefully I can get it done today. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901042#comment-15901042 ] ASF GitHub Bot commented on FLINK-3026: --- Github user iemejia commented on the issue: https://github.com/apache/flink/pull/3493 R: @greghogan cc: @jgrier @patricklucas > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898438#comment-15898438 ] Jamie Grier commented on FLINK-3026: [~iemejia] [~plucas] I'm definitely also for the #2 option above. We should definitely create an Official Flink Docker Image. I also agree with getting rid of the bluemix files but I'm not so sure about having multiple base images. Is that truly a common practice? I also think your labelling scheme looks reasonable - a full spec plus some defaults to make it simple to just grab the latest Flink image. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897066#comment-15897066 ] Ismaël Mejía commented on FLINK-3026: - Other thing that we have to decide afterwards is the tags for the images, what do you think about something like (for example a current default would be): docker pull flink:1.2.0-hadoop27-scala_2.11-alpine And then some defaults (for example if we default to the latest versions): 1.2-alpine -> 1.2.0-hadoop27-alpine -> 1.2.0-hadoop27-scala_2.11-alpine 1.2 -> 1.2.0-hadoop27 -> 1.2.0-hadoop27-scala_2.11 1.1-alpine -> 1.1.4-hadoop27-alpine -> 1.1.4-hadoop27-scala_2.11-alpine ... > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897040#comment-15897040 ] Ismaël Mejía commented on FLINK-3026: - There are other pending issues if we go with 2: 1. We need to get rid of those bluemix files, official repos ask for no extras. 2. We need to rebase the image on openjdk, because the java one is deprecated, see https://hub.docker.com/_/java/ 3. We probably need to re-introduce an ubuntu/debian based image since this is a common practice in the official registry (to have both an ubuntu and an alpine image, this is not mandatory but a common practice). If you agree I can go create JIRAs for those and fix them. Other thing we will have to define is w > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897030#comment-15897030 ] Ismaël Mejía commented on FLINK-3026: - [~jgrier] I have been following the progress on FLINK-5634 and FLINK-5635 so I can retake this one once they are merged (good work btw), so maybe this is a good moment to restart discussing. Also since Max is not around at this moment maybe you are the appropriate person I should discuss around this. We have two alternatives: 1. Once is to ask infra and get the image in the apache account (users will do docker pull apache/flink), this can have some annoyances since I have the impression we will depend on infra for some things, but not sure 100%. 2. Try to have a docker official image. This one is nice because it has more visability (and users will do docker pull flink). https://docs.docker.com/docker-hub/official_repos/ I am really pro (2), what do you think ? > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task > Components: Build System, Docker >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-3026) Publish the flink docker container to the docker registry
[ https://issues.apache.org/jira/browse/FLINK-3026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410906#comment-15410906 ] Ismaël Mejía commented on FLINK-3026: - I assign myself this issue following the previous discussion with [~aljoscha]. > Publish the flink docker container to the docker registry > - > > Key: FLINK-3026 > URL: https://issues.apache.org/jira/browse/FLINK-3026 > Project: Flink > Issue Type: Task >Reporter: Omer Katz >Assignee: Ismaël Mejía > Labels: Deployment, Docker > > There's a dockerfile that can be used to build a docker container already in > the repository. It'd be awesome to just be able to pull it instead of > building it ourselves. > The dockerfile can be found at > https://github.com/apache/flink/tree/master/flink-contrib/docker-flink > It also doesn't point to the latest version of Flink which I fixed in > https://github.com/apache/flink/pull/1366 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
