Here's the latest version of the Flink Quickstart PR, https://github.com/apache/iceberg/pull/15062 Please take a look!
Let's start a new thread to follow up on publishing the Docker image (for Flink and Spark) On Wed, Feb 18, 2026 at 4:54 AM Maximilian Michels <[email protected]> wrote: > Thanks Robin! Really great work. > > Cheers, > Max > > On Sun, Feb 15, 2026 at 9:07 PM Kevin Liu <[email protected]> wrote: > > > > Thanks again for working on this. I took another look at #15124. Let's > try to get this merged, it's very self contained. > > > > Best, > > Kevin Liu > > > > On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev < > [email protected]> wrote: > >> > >> (I realised I mistakenly sent this only to Péter instead of the whole > list) > >> > >> Could I get a review on https://github.com/apache/iceberg/pull/15124 > please. > >> Once that's sorted, I'll update the quickstart docs PR (15062) for the > necessary changes. > >> > >> thanks, Robin. > >> > >> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <[email protected]> wrote: > >>> > >>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for > review please. Once that's sorted, I'll update the quickstart docs PR > (15062) for the necessary changes. > >>> > >>> thanks, Robin. > >>> > >>> On Fri, 6 Feb 2026 at 11:20, Péter Váry <[email protected]> > wrote: > >>>> > >>>> In there are no objections then I would prefer it in the > docker/iceberg-flink-quickstart > >>>> > >>>> Robin Moffatt via dev <[email protected]> ezt írta (időpont: > 2026. febr. 6., P, 11:13): > >>>>> > >>>>> Hi Peter, > >>>>> > >>>>> Thanks for the direction. I'll remove the publish step so that we > can get the quickstart published, and then work on the publishing > subsequently. > >>>>> > >>>>> Do you think the Dockerfile is best kept in flink/quickstart, or > docker/iceberg-flink-quickstart ? > >>>>> > >>>>> thanks, Robin > >>>>> > >>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <[email protected]> > wrote: > >>>>>> > >>>>>> I think we have two options: > >>>>>> > >>>>>> Remove the image publication from this PR ( > https://github.com/apache/iceberg/pull/15124) for now, and proceed with > adding the Docker image and updating the documentation. > >>>>>> Alternatively, we could discuss publishing the Flink quickstart > image at the next Iceberg Community Sync and use that as an opportunity to > simplify both the documentation and the overall user experience. > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> Robin Moffatt via dev <[email protected]> ezt írta (időpont: > 2026. febr. 4., Sze, 18:52): > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> I have perhaps managed to deadlock this process :) I'd appreciate > some help untangling it. The recap is in my previous email (below). > >>>>>>> > >>>>>>> thanks, Robin. > >>>>>>> > >>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <[email protected]> > wrote: > >>>>>>>> > >>>>>>>> Hi Kevin, > >>>>>>>> > >>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :) > >>>>>>>> I have two related PRs: > >>>>>>>> > >>>>>>>> #15124: Add Flink Quickstart docker image > >>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile too) > >>>>>>>> > >>>>>>>> I can see a few routes forward: > >>>>>>>> > >>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with > the publish script (I've not seen anything raised about it yet tho?) > >>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still > relying on local image build (not sure what this would achieve vs the > option above tho?) > >>>>>>>> 3. Merge #15124 including publish script, then #15062 using the > published image not the local build > >>>>>>>> > >>>>>>>> Either way, one thing that needs resolving is the Dockerfile > location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart > (#15124). > >>>>>>>> > >>>>>>>> LMK if I've missed an angle here. > >>>>>>>> > >>>>>>>> thanks, Robin > >>>>>>>> > >>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <[email protected]> > wrote: > >>>>>>>>> > >>>>>>>>> Thanks for working on this, Robin! It looks like the complexity > here is publishing the docker image. What do you think about isolating that > part? (Just move the publish script out of #15124) We can start with the > Dockerfile definition, which allows us to build locally. This should > unblock us from merging the getting started docs in #15062 > >>>>>>>>> Thoughts? > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> Kevin Liu > >>>>>>>>> > >>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev < > [email protected]> wrote: > >>>>>>>>>> > >>>>>>>>>> Hi, > >>>>>>>>>> > >>>>>>>>>> Thanks for the discussion and input. > >>>>>>>>>> It sounds like there are no major blockers. Could someone > please review https://github.com/apache/iceberg/pull/15124 ? > >>>>>>>>>> > >>>>>>>>>> thanks, > >>>>>>>>>> > >>>>>>>>>> Robin. > >>>>>>>>>> > >>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <[email protected]> > wrote: > >>>>>>>>>>> > >>>>>>>>>>> Hey folks, > >>>>>>>>>>> > >>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the > Spark base image and installs all the necessary jars. This is used for our > integration test setup [2] and is inspired by > databricks/docker-spark-iceberg [3]. We've made many improvements such as > upgrading to Spark 4, supporting Spark Connect, and better image build > caching. > >>>>>>>>>>> > >>>>>>>>>>> This is already self-contained and can be reused by other > subprojects. In fact, iceberg-rust already uses it [4] and I try to keep > them in sync. > >>>>>>>>>>> I think it would be beneficial for the project to publish this > image and something similar for Flink. > >>>>>>>>>>> > >>>>>>>>>>> Let me know what you think. > >>>>>>>>>>> > >>>>>>>>>>> Best, > >>>>>>>>>>> Kevin Liu > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> [1] > https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile > >>>>>>>>>>> [2] > https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21 > >>>>>>>>>>> [3] > https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile > >>>>>>>>>>> [4] > https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile > >>>>>>>>>>> > >>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu < > [email protected]> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> > Since the integration code for both Spark and Flink lives > in our repository, it might make sense to also store the Docker images and > the corresponding scripts there. > >>>>>>>>>>>> > >>>>>>>>>>>> I agree with Peter here. > >>>>>>>>>>>> > >>>>>>>>>>>> The previous thread has some concerns if the Iceberg project > should host those docker images. Not sure if the opinions have changed. > >>>>>>>>>>>> > >>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev < > [email protected]> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread. > >>>>>>>>>>>>> Having looked at it, am I understanding the view to be that > ideally Flink would publish a Docker image that included the Iceberg > dependencies? > >>>>>>>>>>>>> > >>>>>>>>>>>>> However we do this, I feel that the user coming to run the > Flink quickstart should not have to build their own Docker image; this adds > unnecessary friction that is easily alleviated. > >>>>>>>>>>>>> > >>>>>>>>>>>>> If I've understood the situation correctly, then I'm happy > to discuss this idea with the Flink community; please let me know before I > do so. > >>>>>>>>>>>>> > >>>>>>>>>>>>> thanks, Robin. > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat < > [email protected]> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi Robin and Peter, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I discussed community-maintained Docker images previously: > >>>>>>>>>>>>>> > https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> The consensus was to publish only the REST fixture Docker > image (now at 100K+ total downloads) and use Docker images published by the > main engines in the quickstart, instead of maintaining these images > ourselves. > >>>>>>>>>>>>>> See the thread above for more details. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in > favor of adding it and relying on the Docker images provided by Flink > rather than maintaining our own images. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> - Ajantha > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry < > [email protected]> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi Robin, > >>>>>>>>>>>>>>> It would be nice to separate them. I expect that we will > have some extra stuff to do with the docker image. For example make sure > that we have ci in place to build it. > >>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>> Peter > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev < > [email protected]> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks for the positive reception of this idea. > >>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :) > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as > separate PRs, or combine them? > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> thanks, Robin. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124 > >>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré < > [email protected]> wrote: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> This is a great idea. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker image > published by the project, we must ensure it is fully compliant with ASF > requirements regarding LICENSE/NOTICE files, etc. While this may seem > straightforward, it is a detail that is often overlooked. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> I would be happy to help with this process. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Regards, > >>>>>>>>>>>>>>>>> JB > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels < > [email protected]> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Hey Robin, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for > new users to get > >>>>>>>>>>>>>>>>>> all the dependencies in the right place. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images: > >>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> -Max > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry < > [email protected]> wrote: > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > Hi Robin, > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in the > Iceberg repo. > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side: > https://github.com/apache/iceberg/issues/13519 > >>>>>>>>>>>>>>>>>> > The conclusion was: > >>>>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and bring > this back up to date. My preference would be to rely on the Spark image < > https://hub.docker.com/r/apache/spark> provided by the Apache Spark > project, similar to what we do for the Hive < > https://iceberg.apache.org/hive-quickstart/> quickstart. We should be > able to load all the Iceberg-specific JARs through the spark.jars.packages > configuration <https://spark.apache.org/docs/3.5.1/configuration.html>. > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list > thread: https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4 > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > Thanks for working on this, > >>>>>>>>>>>>>>>>>> > Peter > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <[email protected]> ezt > írta (időpont: 2026. jan. 22., Cs, 19:23): > >>>>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>>>> >> Hi, > >>>>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], > what do people think about adding an official quickstart Docker image for > Flink to the project? > >>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses > tabulario/spark-iceberg so perhaps that could be brought into the project > too. > >>>>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>>>> >> thanks, Robin. > >>>>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>> > >>> > >> > >> > >> -- > >> Robin Moffatt > >> Sr. Principal Advisor, Streaming Data Technologies >
