Hi Manikumar, Thanks for the feedback!
1. We propose the addition of a new directory named "docker" at the root of the repository, where all Docker-related code will be stored. A detailed directory structure has been added in the KIP. 2. We request the creation of an Apache Kafka repository (apache/kafka) on DockerHub, to be administered under the The Apache Software Foundation <https://hub.docker.com/u/apache>. The PMC members should have the necessary permissions for pushing updates to the docker repo. Thanks and regards, Vedarth On Fri, Oct 20, 2023 at 2:44 PM Manikumar <manikumar.re...@gmail.com> wrote: > Hi Krishna, Vedarth, > > Thanks for the KIP. > > 1. Can we add directory structure of Docker Image related files in Kafka > repo. > > 2. > Steps for the Docker image release will be included in the Release > Process doc of Apache Kafka > > Can we list down the requirements (repos, accounts) for releasing images to > docker hub. I am mainly asking because PMC needs to request docker hub > access/repos. > I can help in getting required repos/accounts. > https://infra.apache.org/docker-hub-policy.html > > > Thanks, > Manikumar > > On Thu, Oct 19, 2023 at 8:22 PM Krishna Agarwal < > krishna0608agar...@gmail.com> wrote: > > > Hi Viktor, > > > > I've noticed there are two types of custom jar configurations: > > > > 1. *Type 1*: In this case, only the class name is required(e.g > > *authorizer.class.name > > <http://authorizer.class.name>**)* This can be configured by the > > following steps: > > - Mount the jar in the container. > > - Configure the *CLASSPATH* environment variable (used by > > *kafka-run-class.sh*) by providing the mounted path to it. This can > > be passed as an environment variable to the docker container. > > 2. *Type 2*: Here, in addition to the class name, classpath can also > be > > configured (eg *remote.log.metadata.manager.class.name > > <http://remote.log.metadata.manager.class.name> *and > > *remote.log.metadata.manager.class.path*). This can be configured by > the > > following steps: > > - Mount the jar in the container. > > - Configure the respective *class.path* property. > > > > Regards, > > Krishna > > > > On Mon, Sep 25, 2023 at 11:41 PM Krishna Agarwal < > > krishna0608agar...@gmail.com> wrote: > > > > > Hi Viktor, > > > Thanks for the questions. > > > > > > 1. While the docker image outlined in KIP-975 is designed for > > > production environments, it is equally suitable for development and > > testing > > > purposes. We will furnish the docker image, allowing users the > > flexibility > > > to employ it according to their specific needs. > > > 2. The configs will be injected into the docker container through > > > environment variables. These environment variables will have a > prefix > > > allowing for efficient parsing to extract the relevant > > properties.(Will add > > > this implementation in the KIP as well once we converge on this.) > > > 3. Regarding this question, I'll conduct a test on my end after > > > gaining a better understanding, and then provide you with a > response. > > > > > > Regards, > > > Krishna > > > > > > > > > On Tue, Sep 19, 2023 at 3:42 PM Viktor Somogyi-Vass > > > <viktor.somo...@cloudera.com.invalid> wrote: > > > > > >> Hi Ismael, > > >> > > >> I'm not trying to advocate against the docker image, I just pointed > out > > >> that the current scoping of the KIP may be a bit too generic and > thought > > >> that KIP-974 and KIP-975 were aiming for mostly the same thing and can > > be > > >> discussed under one umbrella. Apologies if this was rooted in a > > >> misunderstanding. > > >> > > >> Kirshna, > > >> > > >> I think we need to refine the KIP a bit more. I think there are some > > >> interfaces that we need to include in the KIP as Kafka has plugins in > > >> certain cases where users are expected to provide implementation and I > > >> think it's worth discussing this in the KIP as they're kind of > > interfaces > > >> for users. Here are my questions in order: > > >> 1. In what environments do you want the image to be used? As I > > understand > > >> it would replace the current testing image and serve as a basis for > > >> development, but would it aim at production use cases too > > (docker-compose, > > >> Kubernetes, etc.)? > > >> 2. How do you plan to forward configs to the broker? Do we expect a > > >> populated server.properties file placed in a certain location or > should > > >> the > > >> docker image create this file based on some input (like env vars)? > > >> 3. Certain parts can be pluggable, like metric reporters or remote log > > >> implementations that were just introduced by KIP-405. These manifest > in > > >> jar > > >> files that must be put on the classpath of Kafka while certain > > classnames > > >> have to be configured. How do you plan to implement this, how do we > > >> allow users to configure such things? > > >> > > >> Thanks, > > >> Viktor > > >> > > >> > > >> > > >> > > >> On Thu, Sep 14, 2023 at 4:59 PM Kenneth Eversole > > >> <kevers...@cloudflare.com.invalid> wrote: > > >> > > >> > Hello, > > >> > > > >> > I think this would be a wonderful improvement to the ecosystem. > While > > >> > Viktor is correct that most Docker pipelines eventually lead to a > > >> > kubernetes deployment, that should not stop us from creating an > > >> > Official Docker Image. Creating a Docker image would allow us to > > ensure > > >> a > > >> > level of quality and support for people who want to deploy Kafka as > a > > >> > container on baremetal machines, it could allow us to create > > >> > a sandbox/developer environment for new contributors and developers > to > > >> test > > >> > and have a single agreed upon environment that kafka works in for > > future > > >> > KIPs and would most likely spawn more contributions from people > > wanting > > >> to > > >> > optimize kafka for k8s. > > >> > > > >> > > > >> > I am 100% for this and will gladly help if approved. > > >> > > > >> > Kenneth > > >> > > > >> > On Thu, Sep 14, 2023 at 5:47 AM Ismael Juma <m...@ismaeljuma.com> > > wrote: > > >> > > > >> > > Hi Viktor, > > >> > > > > >> > > I disagree. Docker is a very popular deployment tool and it's not > > only > > >> > used > > >> > > with Kubernetes. > > >> > > > > >> > > Ismael > > >> > > > > >> > > On Thu, Sep 14, 2023, 1:14 AM Viktor Somogyi-Vass > > >> > > <viktor.somo...@cloudera.com.invalid> wrote: > > >> > > > > >> > > > Hi Krishna, > > >> > > > > > >> > > > I think you should merge this KIP and KIP-974 > > >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974> as > there > > >> are > > >> > overlaps as > > >> > > > Federico pointed out on KIP-974 > > >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974>. I > think > > >> you > > >> > should keep that one as it > > >> > > > has well defined goals (improve tests) while I feel this one is > > too > > >> > > > generic. Docker is usually just a tool for either testing or > > >> > Kubernetes, > > >> > > so > > >> > > > they have very well defined use-cases. In the case of Flink for > > >> > instance > > >> > > > the image is used for its kubernetes operator. The use case > would > > >> > > determine > > >> > > > a lot of things and I think a generic image would likely not fit > > the > > >> > > needs > > >> > > > of all use-cases. > > >> > > > > > >> > > > Best, > > >> > > > Viktor > > >> > > > > > >> > > > On Fri, Sep 8, 2023 at 9:58 AM Krishna Agarwal < > > >> > > > krishna0608agar...@gmail.com> > > >> > > > wrote: > > >> > > > > > >> > > > > Hi, > > >> > > > > Apache Kafka does not have an official docker image currently. > > >> > > > > I want to submit a KIP to publish a docker image for Apache > > Kafka. > > >> > > > > > > >> > > > > KIP-975 < > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-975>: > > >> > Docker Image for Apache Kafka > > >> > > > > < > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka > > >> > > > > > > > >> > > > > > > >> > > > > Regards, > > >> > > > > Krishna > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > >
