Let's start a new discussion thread about publishing docker images. I see several existing threads on the same topic already.
On Thu, Feb 19, 2026 at 9:20 AM Robin Moffatt via dev < [email protected]> wrote: > Yay, great news. Thank you for everyone's help with this. > We discussed the idea of publishing the new docker image - can someone > guide me on what the process would look like here for that? I'm happy to > have a go at the PR. > > thanks, Robin. > > On Thu, 19 Feb 2026 at 16:22, Kevin Liu <[email protected]> wrote: > >> https://github.com/apache/iceberg/pull/15062 is merged >> The Flink Quickstart is live at >> https://iceberg.apache.org/docs/nightly/flink/ >> >> Thanks everyone! >> >> On Wed, Feb 18, 2026 at 9:38 AM Jean-Baptiste Onofré <[email protected]> >> wrote: >> >>> Hi >>> >>> I did a new pass, thanks ! >>> >>> Regards >>> JB >>> >>> On Wed, Feb 18, 2026 at 5:14 PM Kevin Liu <[email protected]> wrote: >>> >>>> Here's the latest version of the Flink Quickstart PR, >>>> https://github.com/apache/iceberg/pull/15062 >>>> Please take a look! >>>> >>>> Let's start a new thread to follow up on publishing the Docker image >>>> (for Flink and Spark) >>>> >>>> >>>> >>>> On Wed, Feb 18, 2026 at 4:54 AM Maximilian Michels <[email protected]> >>>> wrote: >>>> >>>>> Thanks Robin! Really great work. >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>> On Sun, Feb 15, 2026 at 9:07 PM Kevin Liu <[email protected]> >>>>> wrote: >>>>> > >>>>> > Thanks again for working on this. I took another look at #15124. >>>>> Let's try to get this merged, it's very self contained. >>>>> > >>>>> > Best, >>>>> > Kevin Liu >>>>> > >>>>> > On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev < >>>>> [email protected]> wrote: >>>>> >> >>>>> >> (I realised I mistakenly sent this only to Péter instead of the >>>>> whole list) >>>>> >> >>>>> >> Could I get a review on >>>>> https://github.com/apache/iceberg/pull/15124 please. >>>>> >> Once that's sorted, I'll update the quickstart docs PR (15062) for >>>>> the necessary changes. >>>>> >> >>>>> >> thanks, Robin. >>>>> >> >>>>> >> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <[email protected]> >>>>> wrote: >>>>> >>> >>>>> >>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for >>>>> review please. Once that's sorted, I'll update the quickstart docs PR >>>>> (15062) for the necessary changes. >>>>> >>> >>>>> >>> thanks, Robin. >>>>> >>> >>>>> >>> On Fri, 6 Feb 2026 at 11:20, Péter Váry < >>>>> [email protected]> wrote: >>>>> >>>> >>>>> >>>> In there are no objections then I would prefer it in the >>>>> docker/iceberg-flink-quickstart >>>>> >>>> >>>>> >>>> Robin Moffatt via dev <[email protected]> ezt írta >>>>> (időpont: 2026. febr. 6., P, 11:13): >>>>> >>>>> >>>>> >>>>> Hi Peter, >>>>> >>>>> >>>>> >>>>> Thanks for the direction. I'll remove the publish step so that >>>>> we can get the quickstart published, and then work on the publishing >>>>> subsequently. >>>>> >>>>> >>>>> >>>>> Do you think the Dockerfile is best kept in flink/quickstart, or >>>>> docker/iceberg-flink-quickstart ? >>>>> >>>>> >>>>> >>>>> thanks, Robin >>>>> >>>>> >>>>> >>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry < >>>>> [email protected]> wrote: >>>>> >>>>>> >>>>> >>>>>> I think we have two options: >>>>> >>>>>> >>>>> >>>>>> Remove the image publication from this PR ( >>>>> https://github.com/apache/iceberg/pull/15124) for now, and proceed >>>>> with adding the Docker image and updating the documentation. >>>>> >>>>>> Alternatively, we could discuss publishing the Flink quickstart >>>>> image at the next Iceberg Community Sync and use that as an opportunity to >>>>> simplify both the documentation and the overall user experience. >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> Robin Moffatt via dev <[email protected]> ezt írta >>>>> (időpont: 2026. febr. 4., Sze, 18:52): >>>>> >>>>>>> >>>>> >>>>>>> Hi, >>>>> >>>>>>> >>>>> >>>>>>> I have perhaps managed to deadlock this process :) I'd >>>>> appreciate some help untangling it. The recap is in my previous email >>>>> (below). >>>>> >>>>>>> >>>>> >>>>>>> thanks, Robin. >>>>> >>>>>>> >>>>> >>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt < >>>>> [email protected]> wrote: >>>>> >>>>>>>> >>>>> >>>>>>>> Hi Kevin, >>>>> >>>>>>>> >>>>> >>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :) >>>>> >>>>>>>> I have two related PRs: >>>>> >>>>>>>> >>>>> >>>>>>>> #15124: Add Flink Quickstart docker image >>>>> >>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile >>>>> too) >>>>> >>>>>>>> >>>>> >>>>>>>> I can see a few routes forward: >>>>> >>>>>>>> >>>>> >>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy >>>>> with the publish script (I've not seen anything raised about it yet tho?) >>>>> >>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still >>>>> relying on local image build (not sure what this would achieve vs the >>>>> option above tho?) >>>>> >>>>>>>> 3. Merge #15124 including publish script, then #15062 using >>>>> the published image not the local build >>>>> >>>>>>>> >>>>> >>>>>>>> Either way, one thing that needs resolving is the Dockerfile >>>>> location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart >>>>> (#15124). >>>>> >>>>>>>> >>>>> >>>>>>>> LMK if I've missed an angle here. >>>>> >>>>>>>> >>>>> >>>>>>>> thanks, Robin >>>>> >>>>>>>> >>>>> >>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu < >>>>> [email protected]> wrote: >>>>> >>>>>>>>> >>>>> >>>>>>>>> Thanks for working on this, Robin! It looks like the >>>>> complexity here is publishing the docker image. What do you think about >>>>> isolating that part? (Just move the publish script out of #15124) We can >>>>> start with the Dockerfile definition, which allows us to build locally. >>>>> This should unblock us from merging the getting started docs in #15062 >>>>> >>>>>>>>> Thoughts? >>>>> >>>>>>>>> >>>>> >>>>>>>>> Best, >>>>> >>>>>>>>> Kevin Liu >>>>> >>>>>>>>> >>>>> >>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev < >>>>> [email protected]> wrote: >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> Hi, >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> Thanks for the discussion and input. >>>>> >>>>>>>>>> It sounds like there are no major blockers. Could someone >>>>> please review https://github.com/apache/iceberg/pull/15124 ? >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> thanks, >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> Robin. >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu < >>>>> [email protected]> wrote: >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>> Hey folks, >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses >>>>> the Spark base image and installs all the necessary jars. This is used for >>>>> our integration test setup [2] and is inspired by >>>>> databricks/docker-spark-iceberg [3]. We've made many improvements such as >>>>> upgrading to Spark 4, supporting Spark Connect, and better image build >>>>> caching. >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>> This is already self-contained and can be reused by other >>>>> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep >>>>> them in sync. >>>>> >>>>>>>>>>> I think it would be beneficial for the project to publish >>>>> this image and something similar for Flink. >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>> Let me know what you think. >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>> Best, >>>>> >>>>>>>>>>> Kevin Liu >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>> [1] >>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile >>>>> >>>>>>>>>>> [2] >>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21 >>>>> >>>>>>>>>>> [3] >>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile >>>>> >>>>>>>>>>> [4] >>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu < >>>>> [email protected]> wrote: >>>>> >>>>>>>>>>>> >>>>> >>>>>>>>>>>> > Since the integration code for both Spark and Flink >>>>> lives in our repository, it might make sense to also store the Docker >>>>> images and the corresponding scripts there. >>>>> >>>>>>>>>>>> >>>>> >>>>>>>>>>>> I agree with Peter here. >>>>> >>>>>>>>>>>> >>>>> >>>>>>>>>>>> The previous thread has some concerns if the Iceberg >>>>> project should host those docker images. Not sure if the opinions have >>>>> changed. >>>>> >>>>>>>>>>>> >>>>> >>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev < >>>>> [email protected]> wrote: >>>>> >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread. >>>>> >>>>>>>>>>>>> Having looked at it, am I understanding the view to be >>>>> that ideally Flink would publish a Docker image that included the Iceberg >>>>> dependencies? >>>>> >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> However we do this, I feel that the user coming to run >>>>> the Flink quickstart should not have to build their own Docker image; this >>>>> adds unnecessary friction that is easily alleviated. >>>>> >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> If I've understood the situation correctly, then I'm >>>>> happy to discuss this idea with the Flink community; please let me know >>>>> before I do so. >>>>> >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> thanks, Robin. >>>>> >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat < >>>>> [email protected]> wrote: >>>>> >>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>> Hi Robin and Peter, >>>>> >>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>> I discussed community-maintained Docker images >>>>> previously: >>>>> >>>>>>>>>>>>>> >>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq >>>>> >>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>> The consensus was to publish only the REST fixture >>>>> Docker image (now at 100K+ total downloads) and use Docker images >>>>> published >>>>> by the main engines in the quickstart, instead of maintaining these images >>>>> ourselves. >>>>> >>>>>>>>>>>>>> See the thread above for more details. >>>>> >>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in >>>>> favor of adding it and relying on the Docker images provided by Flink >>>>> rather than maintaining our own images. >>>>> >>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>> - Ajantha >>>>> >>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry < >>>>> [email protected]> wrote: >>>>> >>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>> Hi Robin, >>>>> >>>>>>>>>>>>>>> It would be nice to separate them. I expect that we >>>>> will have some extra stuff to do with the docker image. For example make >>>>> sure that we have ci in place to build it. >>>>> >>>>>>>>>>>>>>> Thanks, >>>>> >>>>>>>>>>>>>>> Peter >>>>> >>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev < >>>>> [email protected]> wrote: >>>>> >>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>> Thanks for the positive reception of this idea. >>>>> >>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :) >>>>> >>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as >>>>> separate PRs, or combine them? >>>>> >>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>> thanks, Robin. >>>>> >>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124 >>>>> >>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062 >>>>> >>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré < >>>>> [email protected]> wrote: >>>>> >>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>> Hi, >>>>> >>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>> This is a great idea. >>>>> >>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker >>>>> image published by the project, we must ensure it is fully compliant with >>>>> ASF requirements regarding LICENSE/NOTICE files, etc. While this may seem >>>>> straightforward, it is a detail that is often overlooked. >>>>> >>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>> I would be happy to help with this process. >>>>> >>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>> Regards, >>>>> >>>>>>>>>>>>>>>>> JB >>>>> >>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels < >>>>> [email protected]> wrote: >>>>> >>>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>>> Hey Robin, >>>>> >>>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful >>>>> for new users to get >>>>> >>>>>>>>>>>>>>>>>> all the dependencies in the right place. >>>>> >>>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker >>>>> images: >>>>> >>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink >>>>> >>>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>>> -Max >>>>> >>>>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry < >>>>> [email protected]> wrote: >>>>> >>>>>>>>>>>>>>>>>> > >>>>> >>>>>>>>>>>>>>>>>> > Hi Robin, >>>>> >>>>>>>>>>>>>>>>>> > >>>>> >>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in >>>>> the Iceberg repo. >>>>> >>>>>>>>>>>>>>>>>> > >>>>> >>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side: >>>>> https://github.com/apache/iceberg/issues/13519 >>>>> >>>>>>>>>>>>>>>>>> > The conclusion was: >>>>> >>>>>>>>>>>>>>>>>> >> >>>>> >>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and >>>>> bring this back up to date. My preference would be to rely on the Spark >>>>> image <https://hub.docker.com/r/apache/spark> provided by the Apache >>>>> Spark project, similar to what we do for the Hive < >>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be >>>>> able to load all the Iceberg-specific JARs through the spark.jars.packages >>>>> configuration <https://spark.apache.org/docs/3.5.1/configuration.html >>>>> >. >>>>> >>>>>>>>>>>>>>>>>> > >>>>> >>>>>>>>>>>>>>>>>> > >>>>> >>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list >>>>> thread: >>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4 >>>>> >>>>>>>>>>>>>>>>>> > >>>>> >>>>>>>>>>>>>>>>>> > Thanks for working on this, >>>>> >>>>>>>>>>>>>>>>>> > Peter >>>>> >>>>>>>>>>>>>>>>>> > >>>>> >>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <[email protected]> >>>>> ezt írta (időpont: 2026. jan. 22., Cs, 19:23): >>>>> >>>>>>>>>>>>>>>>>> >> >>>>> >>>>>>>>>>>>>>>>>> >> Hi, >>>>> >>>>>>>>>>>>>>>>>> >> >>>>> >>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR >>>>> [1], what do people think about adding an official quickstart Docker image >>>>> for Flink to the project? >>>>> >>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses >>>>> tabulario/spark-iceberg so perhaps that could be brought into the project >>>>> too. >>>>> >>>>>>>>>>>>>>>>>> >> >>>>> >>>>>>>>>>>>>>>>>> >> thanks, Robin. >>>>> >>>>>>>>>>>>>>>>>> >> >>>>> >>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062 >>>>> >>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> >>>>>>>>>> >>>>> >>>>>>>> >>>>> >>>>>>>> >>>>> >>>>>>> >>>>> >>>>> >>>>> >>> >>>>> >>> >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> Robin Moffatt >>>>> >> Sr. Principal Advisor, Streaming Data Technologies >>>>> >>>> > > -- > *Robin Moffatt* > *Sr. Principal Advisor, Streaming Data Technologies* >
