Thanks for providing feedback and helping in improving this KIP. We have done a very thorough discussion on this KIP and I feel it's now ready for the voting process.
Thanks and regards, Vedarth On Tue, 14 May 2024 at 20:43, Chris Egerton <chr...@aiven.io.invalid> wrote: > Hi Vedarth, > > This looks great, thank you for helping me thoroughly understand the > motivation and benefits for the KIP. Looks good to me. > > Regarding public interface for the images--everything in the "Public > Interface" section in KIP-975 does qualify as public interface for the > images IMO, but I don't think it's comprehensive. If we were asked to, for > example, change the port in the EXPOSE directive in our Dockerfile, that > would probably qualify as a change to public interface too. With the strict > language in the latest draft of this KIP that ensures that any functional > changes to our Docker images go through another follow-up KIP, we should be > fine without having to identify a comprehensive list of everything that > constitutes public interface for our images. > > Cheers, and thanks again for the KIP, > > Chris > > On Mon, May 13, 2024 at 3:07 PM Vedarth Sharma <vedarth.sha...@gmail.com> > wrote: > > > Hey Chris, > > > > Once we provide the definitions to docker, they should take care of > > everything from there. They mentioned here > > < > > > https://github.com/docker-library/official-images?tab=readme-ov-file#library-definition-files > > > > > that > > the image will be rebuilt when the base image is updated. Hence active > > rebuilds won't require any changes from our side. > > If we are packaging something which may contain a CVE, like some jar, > then > > the onus will be on us to patch it, but it will be upto us whether we > > consider the threat severe enough to fix and when we want to provide the > > fixed version. Having Docker Official Image will not impact the frequency > > of our releases. It will be the Apache Kafka community's call on when a > > release goes and Docker Official Image will be released accordingly as > per > > the KIP. source > > <https://github.com/docker-library/faq?tab=readme-ov-file#image-building > > > > > > As mentioned in Docker's documentation as well "In essence we strive to > > heed upstream's recommendations on how they intend for their software to > be > > consumed." source > > < > > > https://github.com/docker-library/official-images?tab=readme-ov-file#what-are-official-images > > > > > Docker Official Image will rely on upstream's recommendation for > > functionality. But I do agree that since Docker's stance on this might > > change in future it makes sense to put a safeguard that will not allow > any > > functionality changes get incorporated as part of the vetting process. I > > have updated the KIP to reflect the same. > > > > KIP-975 has a well defined public interface based on how configs can be > > supplied and how it can be used. I am not sure if we put that label on it > > during discussions. I am happy to have a separate email thread on it to > > iron things out. > > > > I hope this addresses all of your concerns! > > > > Thanks and regards, > > Vedarth > > > > On Mon, May 13, 2024 at 10:55 PM Chris Egerton <chr...@aiven.io.invalid> > > wrote: > > > > > Thanks both for your responses! Friendly reminder: again, better to > > provide > > > a quote instead of just a link :) > > > > > > I've seen a bit about image rebuilding to handle CVEs but I'm a little > > > unclear on how this would work in practice, and I couldn't find any > > > concrete details in any of the links. Does Docker do this automatically > > for > > > DOIs? Or will the onus be on us to put out patched images? Would this > > lead > > > to us putting out images more quickly than we put out standard > releases? > > As > > > a plus, it does look like DOIs get the benefit of Docker Scout [1] for > > > free, which is nice, but it's still unclear who'd be doing the rest of > > the > > > work on that front. > > > > > > As far as this point from Vedarth goes: > > > > > > > By incorporating the source code of the Docker Official Image into > our > > > > AK ecosystem, we gain control over its functionality, ensuring > > alignment > > > > with the OSS Docker image. This ensures a seamless experience for > users > > > who > > > > may need to transition between these images. > > > > > > This captures my concern with the KIP pretty well. If there's any > > > significant divergence in behavior (not just build methodology) between > > the > > > apache/kafka image and what Docker requires for a Kafka DOI, how are we > > > going to vet these changes moving forward? Under the "Post Release > > Process > > > - if Dockerhub folks suggest changes to the Dockerfiles:" header, this > > KIP > > > proposes that we port all suggested changes for the DOI to > > > the docker/jvm/Dockerfile image, but this seems a bit too permissive. > As > > an > > > alternative, we could state that all build-related changes can be done > > with > > > a PR on the apache/kafka GitHub repo (which will require approval from > a > > > single committer), but any functional changes will require a KIP. > > > > > > Finally, during KIP-975 was there discussion on what we would count as > > the > > > public interface for the apache/kafka image? If not, it'd be nice to > get > > > that ironed out since it may make future discussions around our Docker > > > images quicker, but I don't think this is necessary for KIP-1028. > > > > > > [1] - https://www.docker.com/products/docker-scout/ > > > > > > On Mon, May 13, 2024 at 4:37 AM Prabha Manepalli > > > <mpra...@confluent.io.invalid> wrote: > > > > > > > Hi Chris, > > > > > > > > Sharing the requested links explaining why Docker Official images are > > > > considered more secure - > > > > > > > > > > > > > > https://www.docker.com/blog/enhancing-security-and-transparency-with-docker-official-images/ > > > > and > > > > > > > > > > > > > > https://github.com/docker-library/faq#why-does-my-security-scanner-show-that-an-image-has-cves > > > > > > > > I hope these links help you understand why we need Docker Official > > images > > > > for organisations with stringent security compliance requirements for > > > their > > > > Kafka workloads. > > > > > > > > Thank you. > > > > > > > > > > > > > > > > On Sun, May 12, 2024 at 3:33 PM Vedarth Sharma < > > vedarth.sha...@gmail.com > > > > > > > > wrote: > > > > > > > > > Hey Chris! > > > > > > > > > > Functionality wise, we don't intend to have any differences between > > OSS > > > > > Docker Image and Docker Official Image. > > > > > The Docker Official Image will be the recommended one. > > > > > Since the Docker Official Image might be delayed due to review done > > by > > > > > Docker, images on apache/kafka (OSS Docker Image) can be used by > > users. > > > > > > > > > > 1) I read https://docs.docker.com/trusted-content/official-images/ > > and > > > > > > didn't find anything on that page or immediately around it that > > > > explains > > > > > > what compliance requirements might be satisfied by a DOI that > > > couldn't > > > > be > > > > > > satisfied by the existing apache/kafka image. Can you elaborate? > > Feel > > > > > free > > > > > > to provide another link, but please also quote the relevant > > sections > > > > from > > > > > > it (as StackOverflow likes to say, links can grow stale over > time). > > > > > > > > > > > > > > > - If a user's selection is confined solely to Docker Official > > > Images, > > > > > this Docker Image will ensure their access to Kafka. > > > > > - Details on specific advantages of Docker Official Images can > be > > > > found > > > > > at > > > > > > > > > > > > > > > > > > > > https://github.com/docker-library/official-images?tab=readme-ov-file#what-are-official-images > > > > > . > > > > > - The Docker Official Image will be actively rebuilt for updates > > and > > > > > security fixes. > > > > > - It's true we can provide exactly the same benefits in the OSS > > > Docker > > > > > Image as well. But it won't have the Docker Official Image badge > > and > > > > it > > > > > will add additional overhead for Apache Kafka community. > > > > > - The fact that it will have Docker Official Image badge will > set > > it > > > > > apart from the OSS Docker Image. > > > > > - Also the ability to do just docker pull kafka to get the kafka > > > > docker > > > > > image will only be possible with Docker Official Image. > > > > > > > > > > > > > > > 2) It would be great to see a brief summary of the differences in > > these > > > > > > images included in the KIP, in order to try to gauge how this > would > > > > look > > > > > to > > > > > > our users. > > > > > > > > > > > > > > > - Functionally, both Docker images will remain identical. > > > > > - The variance lies primarily in the methodologies of building > and > > > > > validation, as outlined in the updated KIP. > > > > > > > > > > > > > > > 3) What I suggested last time was not a separate > apache/apache-docker > > > > > > repository, but a repository controlled entirely by Docker. The > DOI > > > > docs > > > > > > [1] state that "While it's preferable to have upstream software > > > authors > > > > > > maintaining their Docker Official Images, this isn't a strict > > > > > requirement", > > > > > > which I take to mean that it's not required for an Apache Kafka > DOI > > > to > > > > > live > > > > > > under the apache organization on GitHub. It also seems like > there's > > > > > > precedent for this: images for MySQL [2] and PHP [3] already > exist > > > > under > > > > > > the control of Docker. The reason I think this is worth > considering > > > is > > > > > that > > > > > > Docker can arbitrarily change the eligibility requirements for > > their > > > > > > official images at any time, and it doesn't seem like there's a > > clear > > > > > > process in the KIP for judging how we should respond to these > > changes > > > > (in > > > > > > fact, it seems like the idea in the KIP is that we should make > any > > > > change > > > > > > required with no further vetting beyond possibly a pull request > on > > > > > > apache/kafka, which would require approval by a committer). By > > > hosting > > > > > the > > > > > > DOI definitions ourselves (either in apache/kafka, or in a > > > theoretical > > > > > > apache/docker-kafka repository), we take responsibility for the > > > image, > > > > > even > > > > > > if the owner on Docker Hub is Docker, not Apache. If the code > lives > > > > > > elsewhere, then (as long as basic trademark and possibly security > > > > > > guidelines are respected) Apache doesn't have to concern itself > at > > > all > > > > > with > > > > > > the image and the maintainers are free to make whatever changes > > they > > > > want > > > > > > to it in order to meet Docker's requirements. > > > > > > > > > > > > > > > - By incorporating the source code of the Docker Official Image > > into > > > > our > > > > > AK ecosystem, we gain control over its functionality, ensuring > > > > alignment > > > > > with the OSS Docker image. This ensures a seamless experience > for > > > > users > > > > > who > > > > > may need to transition between these images. > > > > > - Maintaining both images within the same community facilitates > > ease > > > > of > > > > > management and fosters a singular source of truth. > > > > > - While Apache may not retain ownership of the hosted Docker > > > Official > > > > > Image, we are, in essence, providing Docker with a foundation > that > > > > > aligns > > > > > with their established guidelines as well as remains consistent > > with > > > > OSS > > > > > Docker Image apache/kafka. > > > > > - Any future alterations to the functionality can be seamlessly > > > > > propagated across both the OSS and Official Docker Images. > > > > > > > > > > I think these reasons must be why a lot of the Apache projects > choose > > > to > > > > > host the docker definitions themselves. The responsibility of > owning > > > the > > > > > definitions comes with benefits as well that we should also > consider. > > > > > > > > > > Let us know if you have any further questions! > > > > > > > > > > Thanks and regards, > > > > > Vedarth > > > > > > > > > > On Fri, May 10, 2024 at 7:56 PM Chris Egerton > > <chr...@aiven.io.invalid > > > > > > > > > wrote: > > > > > > > > > > > Hi Krish and Prabha, > > > > > > > > > > > > Thanks for your replies. I still have some follow-up questions: > > > > > > > > > > > > 1) I read > https://docs.docker.com/trusted-content/official-images/ > > > and > > > > > > didn't find anything on that page or immediately around it that > > > > explains > > > > > > what compliance requirements might be satisfied by a DOI that > > > couldn't > > > > be > > > > > > satisfied by the existing apache/kafka image. Can you elaborate? > > Feel > > > > > free > > > > > > to provide another link, but please also quote the relevant > > sections > > > > from > > > > > > it (as StackOverflow likes to say, links can grow stale over > time). > > > > > > > > > > > > 2) It would be great to see a brief summary of the differences in > > > these > > > > > > images included in the KIP, in order to try to gauge how this > would > > > > look > > > > > to > > > > > > our users. > > > > > > > > > > > > 3) What I suggested last time was not a separate > > apache/apache-docker > > > > > > repository, but a repository controlled entirely by Docker. The > DOI > > > > docs > > > > > > [1] state that "While it's preferable to have upstream software > > > authors > > > > > > maintaining their Docker Official Images, this isn't a strict > > > > > requirement", > > > > > > which I take to mean that it's not required for an Apache Kafka > DOI > > > to > > > > > live > > > > > > under the apache organization on GitHub. It also seems like > there's > > > > > > precedent for this: images for MySQL [2] and PHP [3] already > exist > > > > under > > > > > > the control of Docker. The reason I think this is worth > considering > > > is > > > > > that > > > > > > Docker can arbitrarily change the eligibility requirements for > > their > > > > > > official images at any time, and it doesn't seem like there's a > > clear > > > > > > process in the KIP for judging how we should respond to these > > changes > > > > (in > > > > > > fact, it seems like the idea in the KIP is that we should make > any > > > > change > > > > > > required with no further vetting beyond possibly a pull request > on > > > > > > apache/kafka, which would require approval by a committer). By > > > hosting > > > > > the > > > > > > DOI definitions ourselves (either in apache/kafka, or in a > > > theoretical > > > > > > apache/docker-kafka repository), we take responsibility for the > > > image, > > > > > even > > > > > > if the owner on Docker Hub is Docker, not Apache. If the code > lives > > > > > > elsewhere, then (as long as basic trademark and possibly security > > > > > > guidelines are respected) Apache doesn't have to concern itself > at > > > all > > > > > with > > > > > > the image and the maintainers are free to make whatever changes > > they > > > > want > > > > > > to it in order to meet Docker's requirements. > > > > > > > > > > > > [1] - > > > > > > > > > https://docs.docker.com/trusted-content/official-images/contributing/, > > > > > > second paragraph under "Contributing to Docker Official Images" > > > > > > [2] - https://github.com/docker-library/mysql > > > > > > [3] - https://github.com/docker-library/php > > > > > > > > > > > > Cheers, > > > > > > > > > > > > Chris > > > > > > > > > > > > On Fri, May 10, 2024 at 7:46 AM Krish Vora < > krishvor...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > Hey Chris, > > > > > > > > > > > > > > We have responded to the initial round of queries. And as you > > > > mentioned > > > > > > in > > > > > > > your comment, you may have more questions related to this KIP. > > > Please > > > > > let > > > > > > > us know if you have any. > > > > > > > We want to start the voting for this KIP soon, as we intend to > > > > include > > > > > it > > > > > > > in the 3.8.0 release. > > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > Krish. > > > > > > > > > > > > > > On Wed, May 8, 2024 at 6:19 PM Krish Vora < > krishvor...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Chris. Thanks for the questions. > > > > > > > > > > > > > > > > 3. Would a separate Docker-owned repository be out of the > > > question? > > > > > I'm > > > > > > > >> guessing there are some trademark issues that might get in > the > > > > way, > > > > > > but > > > > > > > >> it's worth exploring since the entire purpose of this KIP > > seems > > > to > > > > > be > > > > > > to > > > > > > > >> provide images that are vetted and designed by Docker more > > than > > > by > > > > > the > > > > > > > >> Apache Kafka contributors/committers/PMC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > - The process for introducing a Docker Official Image > > involves > > > > > > > > - Hosting the Dockerfile in the Apache Kafka repository > > and > > > > > > > > - Providing the path to this Dockerfile to Docker Hub > in > > > > Docker > > > > > > > > Hub’s own repo > > > > > > > > < > > > > > > > > > > > > https://github.com/docker-library/official-images/tree/master/library> > > > > > > > > . > > > > > > > > - This ensures that any updates to the Dockerfile in the > AK > > > > > > repository > > > > > > > > are directly applicable to the docker official images > > > available > > > > on > > > > > > > Docker > > > > > > > > Hub. > > > > > > > > > > > > > > > > > > > > > > > > - We also did not find any added advantage to create a > > > separate > > > > > > > > repository named apache-docker within the Apache GitHub > > > > > > organization. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Krish. > > > > > > > > > > > > > > > > On Wed, May 8, 2024 at 6:05 PM Prabha Manepalli > > > > > > > > <mpra...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > >> Hi Chris, I would like to add more context to this KIP's > > > > > motivation. > > > > > > > >> Vedarth and Krish, please weigh in with your inputs. > > > > > > > >> > > > > > > > >> In the motivation section it's stated that "Several other > > Apache > > > > > > > projects, > > > > > > > >> > like Flink, Spark, Solr, have already released Docker > > Official > > > > > > Images, > > > > > > > >> with > > > > > > > >> > download figures ranging from 50 million to over 1 > billion. > > > > These > > > > > > > >> numbers > > > > > > > >> > highlight the significant demand among users." But then > > > > > immediately > > > > > > > >> > afterwards, we learn that "Also the Docker Official Images > > are > > > > > > always > > > > > > > >> the > > > > > > > >> > top 1 search result, irrespective of the number of > > downloads." > > > > > > > Wouldn't > > > > > > > >> a > > > > > > > >> > high number of downloads for an image naturally follow > from > > > > being > > > > > > the > > > > > > > >> top > > > > > > > >> > search result? It seems like we can't necessarily assume > > that > > > > > Docker > > > > > > > >> > Official Images are inherently more desirable for users > > based > > > > > solely > > > > > > > on > > > > > > > >> > download statistics. > > > > > > > >> > > > > > > > > >> > > > > > > > >> *My thoughts: *Unlike the Sponsored OSS image, the Docker > > > Official > > > > > > image > > > > > > > >> is > > > > > > > >> more desirable for workloads that have stringent compliance > > > > > > > requirements. > > > > > > > >> More details on why official images are more trusted are > > > > documented > > > > > > here > > > > > > > >> <https://docs.docker.com/trusted-content/official-images/>. > > The > > > > > > Docker > > > > > > > >> Official image would also help an absolutely new Kafka > > beginner > > > > who > > > > > > > might > > > > > > > >> not know about Apache or the concept of Sponsored images. We > > > want > > > > to > > > > > > > make > > > > > > > >> it easier for Kafka beginners to discover the Kafka image > > > through > > > > > > > >> DockerHub. > > > > > > > >> > > > > > > > >> > > > > > > > >> Can you elaborate on the value that these new images would > add > > > > from > > > > > a > > > > > > > >> > user's perspective? I'm hesitant to introduce another > image, > > > > since > > > > > > it > > > > > > > >> adds > > > > > > > >> > to the cognitive burden of people who will inevitably have > > to > > > > > answer > > > > > > > the > > > > > > > >> > question of "What are the differences between all of the > > > > available > > > > > > > >> images > > > > > > > >> > and which one is best for my use case?" > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> *My thoughts: *This is a valid concern to address. The > > response > > > to > > > > > the > > > > > > > >> above question addresses the value-add this new Docker > > Official > > > > > image > > > > > > > >> would > > > > > > > >> provide. I also agree we need a clear distinction between > each > > > of > > > > > > these > > > > > > > >> images to be well documented. We plan to update the AK > website > > > > with > > > > > > > >> details > > > > > > > >> on how, why, and when a developer would want to use each of > > > these > > > > > > > >> particular images(KIP-974,975,1028). > > > > > > > >> > > > > > > > >> Thanks, > > > > > > > >> Prabha. > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> On Tue, Apr 30, 2024 at 9:41 PM Chris Egerton > > > > > <chr...@aiven.io.invalid > > > > > > > > > > > > > > >> wrote: > > > > > > > >> > > > > > > > >> > Hi Vedarth and Krish, > > > > > > > >> > > > > > > > > >> > Thanks for the KIP! I have to admit I'm a little > skeptical; > > > > > > hopefully > > > > > > > >> you > > > > > > > >> > can help me understand the need for these additional > images. > > > > > > > >> > > > > > > > > >> > 1) In the motivation section it's stated that "Several > other > > > > > Apache > > > > > > > >> > projects, like Flink, Spark, Solr, have already released > > > Docker > > > > > > > Official > > > > > > > >> > Images, with download figures ranging from 50 million to > > over > > > 1 > > > > > > > billion. > > > > > > > >> > These numbers highlight the significant demand among > users." > > > But > > > > > > then > > > > > > > >> > immediately afterwards, we learn that "Also the Docker > > > Official > > > > > > Images > > > > > > > >> are > > > > > > > >> > always the top 1 search result, irrespective of the number > > of > > > > > > > >> downloads." > > > > > > > >> > Wouldn't a high number of downloads for an image naturally > > > > follow > > > > > > from > > > > > > > >> > being the top search result? It seems like we can't > > > necessarily > > > > > > assume > > > > > > > >> that > > > > > > > >> > Docker Official Images are inherently more desirable for > > users > > > > > based > > > > > > > >> solely > > > > > > > >> > on download statistics. > > > > > > > >> > > > > > > > > >> > 2) Can you elaborate on the value that these new images > > would > > > > add > > > > > > > from a > > > > > > > >> > user's perspective? I'm hesitant to introduce another > image, > > > > since > > > > > > it > > > > > > > >> adds > > > > > > > >> > to the cognitive burden of people who will inevitably have > > to > > > > > answer > > > > > > > the > > > > > > > >> > question of "What are the differences between all of the > > > > available > > > > > > > >> images > > > > > > > >> > and which one is best for my use case?" > > > > > > > >> > > > > > > > > >> > 3) Would a separate Docker-owned repository be out of the > > > > > question? > > > > > > > I'm > > > > > > > >> > guessing there are some trademark issues that might get in > > the > > > > > way, > > > > > > > but > > > > > > > >> > it's worth exploring since the entire purpose of this KIP > > > seems > > > > to > > > > > > be > > > > > > > to > > > > > > > >> > provide images that are vetted and designed by Docker more > > > than > > > > by > > > > > > the > > > > > > > >> > Apache Kafka contributors/committers/PMC. > > > > > > > >> > > > > > > > > >> > I may have more questions later but wanted to get this > > initial > > > > > round > > > > > > > out > > > > > > > >> > now without trying to list everything first. > > > > > > > >> > > > > > > > > >> > Looking forward to your thoughts! > > > > > > > >> > > > > > > > > >> > Cheers, > > > > > > > >> > > > > > > > > >> > Chris > > > > > > > >> > > > > > > > > >> > On Mon, Apr 22, 2024 at 2:14 PM Vedarth Sharma < > > > > > > > >> vedarth.sha...@gmail.com> > > > > > > > >> > wrote: > > > > > > > >> > > > > > > > > >> > > Hey folks, > > > > > > > >> > > > > > > > > > >> > > Thanks a lot for reviewing the KIP and providing > feedback. > > > > > > > >> > > The discussion thread seems resolved and KIP has been > > > updated > > > > > > > >> > accordingly. > > > > > > > >> > > We will be starting the voting thread for this KIP in > the > > > next > > > > > few > > > > > > > >> days. > > > > > > > >> > > Please take a look at the KIP and let us know if any > > further > > > > > > > >> discussion > > > > > > > >> > > is needed. > > > > > > > >> > > > > > > > > > >> > > Thanks and regards, > > > > > > > >> > > Vedarth > > > > > > > >> > > > > > > > > > >> > > On Fri, Apr 19, 2024 at 1:33 PM Manikumar < > > > > > > > manikumar.re...@gmail.com> > > > > > > > >> > > wrote: > > > > > > > >> > > > > > > > > > >> > > > Thanks Krish. KIP looks good to me. > > > > > > > >> > > > > > > > > > > >> > > > On Wed, Apr 17, 2024 at 1:38 PM Krish Vora < > > > > > > krishvor...@gmail.com > > > > > > > > > > > > > > > >> > > wrote: > > > > > > > >> > > > > > > > > > > > >> > > > > Hi Manikumar, > > > > > > > >> > > > > > > > > > > > >> > > > > Thanks for the comments. > > > > > > > >> > > > > > > > > > > > >> > > > > Maybe as part of the release process, RM can create > a > > > JIRA > > > > > for > > > > > > > >> this > > > > > > > >> > > > > > task. This can be taken by RM or any comitter or > any > > > > > > > contributor > > > > > > > >> > > (with > > > > > > > >> > > > > > some help from commiters to run "Docker Image > > > > Preparation > > > > > > via > > > > > > > >> > GitHub > > > > > > > >> > > > > > Actions:" > > > > > > > >> > > > > > > > > > > > >> > > > > This sounds like a good idea. This step would be > > > > beneficial. > > > > > > By > > > > > > > >> > > creating > > > > > > > >> > > > a > > > > > > > >> > > > > JIRA ticket, it will also serve as a reminder to > > > complete > > > > > the > > > > > > > >> > > > post-release > > > > > > > >> > > > > steps for the Docker official images. Have updated > the > > > KIP > > > > > > with > > > > > > > >> this > > > > > > > >> > > > step. > > > > > > > >> > > > > > > > > > > > >> > > > > Is this using GitHub Actions workflow? or manual > > > testing? > > > > > > > >> > > > > > > > > > > > >> > > > > This will be done by a Github Actions workflow, > which > > > will > > > > > > test > > > > > > > >> the > > > > > > > >> > > > static > > > > > > > >> > > > > Docker Official Image assets for a specific release > > > > version. > > > > > > > >> > > > > > > > > > > > >> > > > > Is it mandatory for RM/comitters to raise the PR to > > > Docker > > > > > > Hub’s > > > > > > > >> > > > > > official images repository (or) can it be done by > > any > > > > > > > >> contributor. > > > > > > > >> > > > > > > > > > > > >> > > > > I believe that it can be done by any contributor > (ref: > > > > This > > > > > > link > > > > > > > >> > > > > < > > > > > > > >> > > > > > > > > > > https://docs.docker.com/trusted-content/official-images/contributing/ > > > > > > > >> > > > > > > > > > > >> > > > > quotes "*Anyone can provide feedback, contribute > code, > > > > > suggest > > > > > > > >> > process > > > > > > > >> > > > > changes, or even propose a new Official Image.*") > > > > > > > >> > > > > > > > > > > > >> > > > > Also I was thinking, once the KIP gets voted, we > > should > > > > try > > > > > to > > > > > > > >> > release > > > > > > > >> > > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This > > > will > > > > > help > > > > > > > us > > > > > > > >> to > > > > > > > >> > > > > > validate the process and allow us to fix any > changes > > > > > > suggested > > > > > > > >> by > > > > > > > >> > > > > > Dockerhub before the 3.8.0 release. > > > > > > > >> > > > > > > > > > > > >> > > > > This sounds like a great idea. This KIP proposes > > release > > > > of > > > > > > DOI > > > > > > > >> as a > > > > > > > >> > > > > post-release process, which can be done anytime post > > > > > release. > > > > > > > >> Since > > > > > > > >> > > 3.7.0 > > > > > > > >> > > > > is already released, we can perform these steps for > > that > > > > > > release > > > > > > > >> too. > > > > > > > >> > > By > > > > > > > >> > > > > the time the KIP gets implemented, if 3.7.1 is > > released, > > > > we > > > > > > > could > > > > > > > >> do > > > > > > > >> > > > these > > > > > > > >> > > > > steps for 3.7.1, instead of 3.7.0. This would allow > us > > > to > > > > > make > > > > > > > >> > changes > > > > > > > >> > > to > > > > > > > >> > > > > the Dockerfiles and other assets based on feedback > > from > > > > > Docker > > > > > > > Hub > > > > > > > >> > > before > > > > > > > >> > > > > the release of version 3.8.0. > > > > > > > >> > > > > > > > > > > > >> > > > > Thanks, > > > > > > > >> > > > > Krish. > > > > > > > >> > > > > > > > > > > > >> > > > > On Mon, Apr 15, 2024 at 12:59 PM Manikumar < > > > > > > > >> > manikumar.re...@gmail.com> > > > > > > > >> > > > > wrote: > > > > > > > >> > > > > > > > > > > > >> > > > > > Hi Krish, > > > > > > > >> > > > > > > > > > > > > >> > > > > > Thanks for the updated KIP. a few comments below. > > > > > > > >> > > > > > > > > > > > > >> > > > > > > "These actions can be carried out by the RM or > any > > > > > > > contributor > > > > > > > >> > post > > > > > > > >> > > > the > > > > > > > >> > > > > > release process." > > > > > > > >> > > > > > Maybe as part of the release process, RM can > create > > a > > > > JIRA > > > > > > for > > > > > > > >> this > > > > > > > >> > > > > > task. This can be taken by RM or any comitter or > any > > > > > > > contributor > > > > > > > >> > > (with > > > > > > > >> > > > > > some help from commiters to run "Docker Image > > > > Preparation > > > > > > via > > > > > > > >> > GitHub > > > > > > > >> > > > > > Actions:" > > > > > > > >> > > > > > > > > > > > > >> > > > > > > "Perform Docker build tests to ensure image > > > integrity" > > > > > > > >> > > > > > Is this using GitHub Actions workflow? or manual > > > > testing? > > > > > > > >> > > > > > > > > > > > > >> > > > > > > "The RM will manually raise the final PR to > Docker > > > > Hub’s > > > > > > > >> official > > > > > > > >> > > > images > > > > > > > >> > > > > > repository using the contents of the generated > file" > > > > > > > >> > > > > > Is it mandatory for RM/comitters to raise the PR > to > > > > > Docker > > > > > > > >> Hub’s > > > > > > > >> > > > > > official images repository (or) can it be done by > > any > > > > > > > >> contributor. > > > > > > > >> > > > > > > > > > > > > >> > > > > > Also I was thinking, once the KIP gets voted, we > > > should > > > > > try > > > > > > to > > > > > > > >> > > release > > > > > > > >> > > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This > > > will > > > > > help > > > > > > > us > > > > > > > >> to > > > > > > > >> > > > > > validate the process and allow us to fix any > changes > > > > > > suggested > > > > > > > >> by > > > > > > > >> > > > > > Dockerhub before the 3.8.0 release. > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > Thanks, > > > > > > > >> > > > > > > > > > > > > >> > > > > > On Mon, Apr 8, 2024 at 2:33 PM Krish Vora < > > > > > > > >> krishvor...@gmail.com> > > > > > > > >> > > > wrote: > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > Hi Manikumar and Luke. > > > > > > > >> > > > > > > Thanks for the questions. > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > 1. No, the Docker inventory files and > > configurations > > > > > will > > > > > > > not > > > > > > > >> be > > > > > > > >> > > the > > > > > > > >> > > > same > > > > > > > >> > > > > > > for Open Source Software (OSS) Images and Docker > > > > > Official > > > > > > > >> Images > > > > > > > >> > > > (DOI). > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > For OSS images, the Dockerfile located in > > > > > > > >> docker/jvm/dockerfile > > > > > > > >> > is > > > > > > > >> > > > > > > utilized. This process is integrated with the > > > existing > > > > > > > release > > > > > > > >> > > > pipeline > > > > > > > >> > > > > > as > > > > > > > >> > > > > > > outlined in KIP-975 > > > > > > > >> > > > > > > < > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka#KIP975:DockerImageforApacheKafka-Status > > > > > > > >> > > > > > >, > > > > > > > >> > > > > > > where the Kafka URL is provided as a build > > argument. > > > > > This > > > > > > > >> method > > > > > > > >> > > > allows > > > > > > > >> > > > > > for > > > > > > > >> > > > > > > building, testing, and releasing OSS images > > > > dynamically. > > > > > > The > > > > > > > >> OSS > > > > > > > >> > > > images > > > > > > > >> > > > > > > will continue to be released under the standard > > > > release > > > > > > > >> process . > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > In contrast, the release process for DOIs > requires > > > > > > providing > > > > > > > >> the > > > > > > > >> > > > Docker > > > > > > > >> > > > > > Hub > > > > > > > >> > > > > > > team with a specific directory for each version > > > > release > > > > > > that > > > > > > > >> > > > contains a > > > > > > > >> > > > > > > standalone Dockerfile. These Dockerfiles are > > > designed > > > > to > > > > > > be > > > > > > > >> > > > > > > self-sufficient, hence require hardcoded values > > > > instead > > > > > of > > > > > > > >> > relying > > > > > > > >> > > on > > > > > > > >> > > > > > build > > > > > > > >> > > > > > > arguments. To accommodate this, in our proposed > > > > > approach, > > > > > > a > > > > > > > >> new > > > > > > > >> > > > directory > > > > > > > >> > > > > > > named docker_official_images has been created. > > This > > > > > > > directory > > > > > > > >> > > > contains > > > > > > > >> > > > > > > version-specific directories, having Dockerfiles > > > with > > > > > > > >> hardcoded > > > > > > > >> > > > > > > configurations for each release, acting as the > > > source > > > > of > > > > > > > truth > > > > > > > >> > for > > > > > > > >> > > > DOI > > > > > > > >> > > > > > > releases. The hardcoded dockerfiles will be > > created > > > > > using > > > > > > > the > > > > > > > >> > > > > > > docker/jvm/dockerfile as a template. Thus, as > part > > > of > > > > > post > > > > > > > >> > release > > > > > > > >> > > we > > > > > > > >> > > > > > will > > > > > > > >> > > > > > > be creating a Dockerfile that will be reviewed > by > > > the > > > > > > > >> Dockerhub > > > > > > > >> > > > community > > > > > > > >> > > > > > > and might need changes as per their review. This > > > > > approach > > > > > > > >> ensures > > > > > > > >> > > > that > > > > > > > >> > > > > > DOIs > > > > > > > >> > > > > > > are built consistently and meet the specific > > > > > requirements > > > > > > > set > > > > > > > >> by > > > > > > > >> > > > Docker > > > > > > > >> > > > > > Hub. > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > 2. Yes Manikumar, transitioning the release of > > > Docker > > > > > > > Official > > > > > > > >> > > Images > > > > > > > >> > > > > > (DOI) > > > > > > > >> > > > > > > to a post-release activity does address the > > concerns > > > > > about > > > > > > > >> > > > complicating > > > > > > > >> > > > > > the > > > > > > > >> > > > > > > release process. Initially, we considered > > > > incorporating > > > > > > DOI > > > > > > > >> > release > > > > > > > >> > > > > > > directly into Kafka's release workflow. However, > > > this > > > > > > > approach > > > > > > > >> > > > > > > significantly increased the RMs workload due to > > the > > > > > > addition > > > > > > > >> of > > > > > > > >> > > > numerous > > > > > > > >> > > > > > > steps, complicating the process. By designating > > the > > > > DOI > > > > > > > >> release > > > > > > > >> > as > > > > > > > >> > > a > > > > > > > >> > > > > > > post-release task, we maintain the original > > release > > > > > > process. > > > > > > > >> This > > > > > > > >> > > > > > > adjustment allows for the DOI release to be done > > > after > > > > > the > > > > > > > >> main > > > > > > > >> > > > release. > > > > > > > >> > > > > > We > > > > > > > >> > > > > > > have revised the KIP to reflect that DOI > releases > > > will > > > > > now > > > > > > > >> occur > > > > > > > >> > > > after > > > > > > > >> > > > > > the > > > > > > > >> > > > > > > main release phase. Please review the updated > > > document > > > > > and > > > > > > > >> > provide > > > > > > > >> > > > any > > > > > > > >> > > > > > > feedback you might have. > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > Thanks, > > > > > > > >> > > > > > > Krish. > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > On Wed, Apr 3, 2024 at 3:35 PM Luke Chen < > > > > > > show...@gmail.com > > > > > > > > > > > > > > > >> > > wrote: > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > Hi Krishna, > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > I also have the same question as Manikumar > > raised: > > > > > > > >> > > > > > > > 1. Will the Docker inventory files/etc are the > > > same > > > > > for > > > > > > > OSS > > > > > > > >> > Image > > > > > > > >> > > > and > > > > > > > >> > > > > > > > Docker Official Images? > > > > > > > >> > > > > > > > If no, then why not? Could we make them > > identical > > > so > > > > > > that > > > > > > > we > > > > > > > >> > > don't > > > > > > > >> > > > > > have to > > > > > > > >> > > > > > > > build 2 images for each release? > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > Thank you. > > > > > > > >> > > > > > > > Luke > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > On Wed, Apr 3, 2024 at 12:41 AM Manikumar < > > > > > > > >> > > > manikumar.re...@gmail.com> > > > > > > > >> > > > > > > > wrote: > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > Hi Krishna, > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > Thanks for the KIP. > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > I think Docker Official Images will be > > > beneficial > > > > to > > > > > > the > > > > > > > >> > Kafka > > > > > > > >> > > > > > community. > > > > > > > >> > > > > > > > > Few queries below. > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > 1. Will the Docker inventory files/etc are > the > > > > same > > > > > > for > > > > > > > >> OSS > > > > > > > >> > > > Image and > > > > > > > >> > > > > > > > > Docker Official Images > > > > > > > >> > > > > > > > > 2. I am a bit worried about the new steps to > > the > > > > > > release > > > > > > > >> > > process. > > > > > > > >> > > > > > Maybe > > > > > > > >> > > > > > > > we > > > > > > > >> > > > > > > > > should consider Docker Official Images > release > > > as > > > > > > > >> > Post-Release > > > > > > > >> > > > > > activity. > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > Thanks, > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > On Fri, Mar 22, 2024 at 3:29 PM Krish Vora < > > > > > > > >> > > > krishvor...@gmail.com> > > > > > > > >> > > > > > > > wrote: > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > Hi Hector, > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > Thanks for reaching out. This KIP builds > on > > > top > > > > of > > > > > > > >> KIP-975 > > > > > > > >> > > > > > > > > > < > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > and > > > > > > > >> > > > > > > > > > aims to introduce a JVM-based Docker > > Official > > > > > Image > > > > > > > (DOI > > > > > > > >> > > > > > > > > > < > > > > > > > >> https://docs.docker.com/trusted-content/official-images/ > > > > > > > >> > >) > > > > > > > >> > > > for > > > > > > > >> > > > > > Apache > > > > > > > >> > > > > > > > > > Kafka that will be visible under Docker > > > Official > > > > > > > Images > > > > > > > >> > > > > > > > > > < > > > > > > > https://hub.docker.com/search?image_filter=official&q= > > > > > > > >> >. > > > > > > > >> > > Once > > > > > > > >> > > > > > > > > implemented > > > > > > > >> > > > > > > > > > for Apache Kafka, for each release, there > > will > > > > be > > > > > > one > > > > > > > >> more > > > > > > > >> > > > > > JVM-based > > > > > > > >> > > > > > > > > Docker > > > > > > > >> > > > > > > > > > image available to users. > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > Currently, we already have an OSS > sponsored > > > > image, > > > > > > > which > > > > > > > >> > was > > > > > > > >> > > > > > introduced > > > > > > > >> > > > > > > > > via > > > > > > > >> > > > > > > > > > KIP-975 (apache/kafka < > > > > > > > >> > > > https://hub.docker.com/r/apache/kafka/tags > > > > > > > >> > > > > > >) > > > > > > > >> > > > > > > > > which > > > > > > > >> > > > > > > > > > comes under The Apache Software > Foundation < > > > > > > > >> > > > > > > > > > https://hub.docker.com/u/apache> in > > > > > > > >> > > > > > > > > > Docker Hub. The new Docker Image is the > > Docker > > > > > > > Official > > > > > > > >> > Image > > > > > > > >> > > > > > (DOI), > > > > > > > >> > > > > > > > > which > > > > > > > >> > > > > > > > > > will be built and maintained by Docker > > > > Community. > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > For example, for a release version like > > 3.8.0 > > > we > > > > > > will > > > > > > > >> have > > > > > > > >> > > two > > > > > > > >> > > > JVM > > > > > > > >> > > > > > > > based > > > > > > > >> > > > > > > > > > docker images:- > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > - apache/kafka:3.8.0 (OSS sponsored > > image) > > > > > > > >> > > > > > > > > > - kafka:3.8.0 (Docker Official image) > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > I have added the same in the KIP too for > > > > > everyone's > > > > > > > >> > > reference. > > > > > > > >> > > > > > > > > > Thanks, > > > > > > > >> > > > > > > > > > Krish. > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > On Fri, Mar 22, 2024 at 2:50 AM Hector > > > Geraldino > > > > > > > >> > (BLOOMBERG/ > > > > > > > >> > > > 919 > > > > > > > >> > > > > > 3RD > > > > > > > >> > > > > > > > A) < > > > > > > > >> > > > > > > > > > hgerald...@bloomberg.net> wrote: > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > Hi, > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > What is the difference between this KIP > > and > > > > > > KIP-975: > > > > > > > >> > Docker > > > > > > > >> > > > > > Image for > > > > > > > >> > > > > > > > > > > Apache Kafka? > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > From: dev@kafka.apache.org At: 03/21/24 > > > > > 07:30:07 > > > > > > > >> > > UTC-4:00To: > > > > > > > >> > > > > > > > > > > dev@kafka.apache.org > > > > > > > >> > > > > > > > > > > Subject: [DISCUSS] KIP-1028: Docker > > Official > > > > > Image > > > > > > > for > > > > > > > >> > > Apache > > > > > > > >> > > > > > Kafka > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > Hi everyone, > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > I would like to start the discussion on > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Im > > > > > > > >> > > > > > > > > > > age+for+Apache+Kafka > > > > > > > >> > > > > > > > > > > . > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > This KIP aims to introduce JVM based > > Docker > > > > > > Official > > > > > > > >> > Image > > > > > > > >> > > > (DOI) > > > > > > > >> > > > > > for > > > > > > > >> > > > > > > > > > Apache > > > > > > > >> > > > > > > > > > > Kafka. > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > Regards, > > > > > > > >> > > > > > > > > > > Krish. > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> -- > > > > > > > >> > > > > > > > >> [image: Confluent] <https://www.confluent.io> > > > > > > > >> Prabha Manepalli > > > > > > > >> Product Manager for Confluent Platform Security, Docker > > > > > > > >> linkedin.com/in/prabhamanepalli > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > [image: Confluent] <https://www.confluent.io> > > > > Prabha Manepalli > > > > Product Manager for Confluent Platform Security, Docker Containers > > > > linkedin.com/in/prabhamanepalli > > > > > > > > > >