Thanks Robin! Really great work. Cheers, Max
On Sun, Feb 15, 2026 at 9:07 PM Kevin Liu <[email protected]> wrote: > > Thanks again for working on this. I took another look at #15124. Let's try to > get this merged, it's very self contained. > > Best, > Kevin Liu > > On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev > <[email protected]> wrote: >> >> (I realised I mistakenly sent this only to Péter instead of the whole list) >> >> Could I get a review on https://github.com/apache/iceberg/pull/15124 please. >> Once that's sorted, I'll update the quickstart docs PR (15062) for the >> necessary changes. >> >> thanks, Robin. >> >> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <[email protected]> wrote: >>> >>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for review >>> please. Once that's sorted, I'll update the quickstart docs PR (15062) for >>> the necessary changes. >>> >>> thanks, Robin. >>> >>> On Fri, 6 Feb 2026 at 11:20, Péter Váry <[email protected]> wrote: >>>> >>>> In there are no objections then I would prefer it in the >>>> docker/iceberg-flink-quickstart >>>> >>>> Robin Moffatt via dev <[email protected]> ezt írta (időpont: 2026. >>>> febr. 6., P, 11:13): >>>>> >>>>> Hi Peter, >>>>> >>>>> Thanks for the direction. I'll remove the publish step so that we can get >>>>> the quickstart published, and then work on the publishing subsequently. >>>>> >>>>> Do you think the Dockerfile is best kept in flink/quickstart, or >>>>> docker/iceberg-flink-quickstart ? >>>>> >>>>> thanks, Robin >>>>> >>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <[email protected]> >>>>> wrote: >>>>>> >>>>>> I think we have two options: >>>>>> >>>>>> Remove the image publication from this PR >>>>>> (https://github.com/apache/iceberg/pull/15124) for now, and proceed with >>>>>> adding the Docker image and updating the documentation. >>>>>> Alternatively, we could discuss publishing the Flink quickstart image at >>>>>> the next Iceberg Community Sync and use that as an opportunity to >>>>>> simplify both the documentation and the overall user experience. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Robin Moffatt via dev <[email protected]> ezt írta (időpont: 2026. >>>>>> febr. 4., Sze, 18:52): >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I have perhaps managed to deadlock this process :) I'd appreciate some >>>>>>> help untangling it. The recap is in my previous email (below). >>>>>>> >>>>>>> thanks, Robin. >>>>>>> >>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <[email protected]> wrote: >>>>>>>> >>>>>>>> Hi Kevin, >>>>>>>> >>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :) >>>>>>>> I have two related PRs: >>>>>>>> >>>>>>>> #15124: Add Flink Quickstart docker image >>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile too) >>>>>>>> >>>>>>>> I can see a few routes forward: >>>>>>>> >>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with the >>>>>>>> publish script (I've not seen anything raised about it yet tho?) >>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still relying on >>>>>>>> local image build (not sure what this would achieve vs the option >>>>>>>> above tho?) >>>>>>>> 3. Merge #15124 including publish script, then #15062 using the >>>>>>>> published image not the local build >>>>>>>> >>>>>>>> Either way, one thing that needs resolving is the Dockerfile location: >>>>>>>> flink/quickstart (#15062) vs docker/iceberg-flink-quickstart (#15124). >>>>>>>> >>>>>>>> LMK if I've missed an angle here. >>>>>>>> >>>>>>>> thanks, Robin >>>>>>>> >>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <[email protected]> wrote: >>>>>>>>> >>>>>>>>> Thanks for working on this, Robin! It looks like the complexity here >>>>>>>>> is publishing the docker image. What do you think about isolating >>>>>>>>> that part? (Just move the publish script out of #15124) We can start >>>>>>>>> with the Dockerfile definition, which allows us to build locally. >>>>>>>>> This should unblock us from merging the getting started docs in #15062 >>>>>>>>> Thoughts? >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Kevin Liu >>>>>>>>> >>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev >>>>>>>>> <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Thanks for the discussion and input. >>>>>>>>>> It sounds like there are no major blockers. Could someone please >>>>>>>>>> review https://github.com/apache/iceberg/pull/15124 ? >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> >>>>>>>>>> Robin. >>>>>>>>>> >>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hey folks, >>>>>>>>>>> >>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark >>>>>>>>>>> base image and installs all the necessary jars. This is used for >>>>>>>>>>> our integration test setup [2] and is inspired by >>>>>>>>>>> databricks/docker-spark-iceberg [3]. We've made many improvements >>>>>>>>>>> such as upgrading to Spark 4, supporting Spark Connect, and better >>>>>>>>>>> image build caching. >>>>>>>>>>> >>>>>>>>>>> This is already self-contained and can be reused by other >>>>>>>>>>> subprojects. In fact, iceberg-rust already uses it [4] and I try to >>>>>>>>>>> keep them in sync. >>>>>>>>>>> I think it would be beneficial for the project to publish this >>>>>>>>>>> image and something similar for Flink. >>>>>>>>>>> >>>>>>>>>>> Let me know what you think. >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Kevin Liu >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [1] >>>>>>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile >>>>>>>>>>> [2] >>>>>>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21 >>>>>>>>>>> [3] >>>>>>>>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile >>>>>>>>>>> [4] >>>>>>>>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile >>>>>>>>>>> >>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> > Since the integration code for both Spark and Flink lives in our >>>>>>>>>>>> > repository, it might make sense to also store the Docker images >>>>>>>>>>>> > and the corresponding scripts there. >>>>>>>>>>>> >>>>>>>>>>>> I agree with Peter here. >>>>>>>>>>>> >>>>>>>>>>>> The previous thread has some concerns if the Iceberg project >>>>>>>>>>>> should host those docker images. Not sure if the opinions have >>>>>>>>>>>> changed. >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread. >>>>>>>>>>>>> Having looked at it, am I understanding the view to be that >>>>>>>>>>>>> ideally Flink would publish a Docker image that included the >>>>>>>>>>>>> Iceberg dependencies? >>>>>>>>>>>>> >>>>>>>>>>>>> However we do this, I feel that the user coming to run the Flink >>>>>>>>>>>>> quickstart should not have to build their own Docker image; this >>>>>>>>>>>>> adds unnecessary friction that is easily alleviated. >>>>>>>>>>>>> >>>>>>>>>>>>> If I've understood the situation correctly, then I'm happy to >>>>>>>>>>>>> discuss this idea with the Flink community; please let me know >>>>>>>>>>>>> before I do so. >>>>>>>>>>>>> >>>>>>>>>>>>> thanks, Robin. >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat >>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Robin and Peter, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I discussed community-maintained Docker images previously: >>>>>>>>>>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq >>>>>>>>>>>>>> >>>>>>>>>>>>>> The consensus was to publish only the REST fixture Docker image >>>>>>>>>>>>>> (now at 100K+ total downloads) and use Docker images published >>>>>>>>>>>>>> by the main engines in the quickstart, instead of maintaining >>>>>>>>>>>>>> these images ourselves. >>>>>>>>>>>>>> See the thread above for more details. >>>>>>>>>>>>>> >>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in favor of >>>>>>>>>>>>>> adding it and relying on the Docker images provided by Flink >>>>>>>>>>>>>> rather than maintaining our own images. >>>>>>>>>>>>>> >>>>>>>>>>>>>> - Ajantha >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry >>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Robin, >>>>>>>>>>>>>>> It would be nice to separate them. I expect that we will have >>>>>>>>>>>>>>> some extra stuff to do with the docker image. For example make >>>>>>>>>>>>>>> sure that we have ci in place to build it. >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Peter >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev >>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for the positive reception of this idea. >>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as separate >>>>>>>>>>>>>>>> PRs, or combine them? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> thanks, Robin. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124 >>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré >>>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This is a great idea. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker image >>>>>>>>>>>>>>>>> published by the project, we must ensure it is fully >>>>>>>>>>>>>>>>> compliant with ASF requirements regarding LICENSE/NOTICE >>>>>>>>>>>>>>>>> files, etc. While this may seem straightforward, it is a >>>>>>>>>>>>>>>>> detail that is often overlooked. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I would be happy to help with this process. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>> JB >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels >>>>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hey Robin, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for new >>>>>>>>>>>>>>>>>> users to get >>>>>>>>>>>>>>>>>> all the dependencies in the right place. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images: >>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -Max >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry >>>>>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > Hi Robin, >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in the >>>>>>>>>>>>>>>>>> > Iceberg repo. >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side: >>>>>>>>>>>>>>>>>> > https://github.com/apache/iceberg/issues/13519 >>>>>>>>>>>>>>>>>> > The conclusion was: >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and bring this >>>>>>>>>>>>>>>>>> >> back up to date. My preference would be to rely on the >>>>>>>>>>>>>>>>>> >> Spark image <https://hub.docker.com/r/apache/spark> >>>>>>>>>>>>>>>>>> >> provided by the Apache Spark project, similar to what we >>>>>>>>>>>>>>>>>> >> do for the Hive >>>>>>>>>>>>>>>>>> >> <https://iceberg.apache.org/hive-quickstart/> quickstart. >>>>>>>>>>>>>>>>>> >> We should be able to load all the Iceberg-specific JARs >>>>>>>>>>>>>>>>>> >> through the spark.jars.packages configuration >>>>>>>>>>>>>>>>>> >> <https://spark.apache.org/docs/3.5.1/configuration.html>. >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list thread: >>>>>>>>>>>>>>>>>> > https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4 >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > Thanks for working on this, >>>>>>>>>>>>>>>>>> > Peter >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <[email protected]> ezt írta >>>>>>>>>>>>>>>>>> > (időpont: 2026. jan. 22., Cs, 19:23): >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> Hi, >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], what >>>>>>>>>>>>>>>>>> >> do people think about adding an official quickstart >>>>>>>>>>>>>>>>>> >> Docker image for Flink to the project? >>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses >>>>>>>>>>>>>>>>>> >> tabulario/spark-iceberg so perhaps that could be brought >>>>>>>>>>>>>>>>>> >> into the project too. >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> thanks, Robin. >>>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> >>> >> >> >> -- >> Robin Moffatt >> Sr. Principal Advisor, Streaming Data Technologies
