Where are we in this process? We have some Kafka connect integration test failures in the CI build that seem related to the minio problem. e.g. https://github.com/apache/iceberg/actions/runs/36037765015/job/107788134738?pr=18213 https://github.com/apache/iceberg/actions/runs/36047551910/job/107794733276?pr=18248
Docker compose loads the MinIO service. https://github.com/apache/iceberg/blob/main/kafka-connect/kafka-connect-runtime/docker/docker-compose.yml#L23 On Mon, Sep 14, 2026 at 11:12 AM Neelesh Salian <[email protected]> wrote: > I know in another thread there were some CVE concerns raised about RustFS > (mailing list thread titled: "[DISCUSS] Replacing minio with RustFS in > quick start demo.") > I see a blog referencing them is here: > https://rustfs.com/blog/patch-release-rustfs-1-0-0-alpha-79-released/ > Ok with either choice as long as there is long term support for RustFs and > for security patches with prompt fixes when CVEs show up. > > On Mon, Sep 14, 2026 at 11:02 AM Szehon Ho <[email protected]> > wrote: > >> It also sounds good to me, and also don't have any other strong >> preference. >> >> Thanks >> Szehon >> >> On Mon, Sep 14, 2026 at 9:03 AM Xuanwo <[email protected]> wrote: >> >>> RustFS LGTM. >>> >>> It's just for CI and should be easy to change in the future. I agree >>> with Kevin that let's just move. >>> >>> On Mon, Sep 14, 2026, at 23:55, Kevin Liu wrote: >>> >>> Thanks for the PRs to fix the MinIO issue blocking CI. I think the fixes >>> have now been merged across all the affected Iceberg repos. >>> >>> +1 on moving away from MinIO. Last time we discussed this, I think there >>> was general agreement on moving to another object store, but we didn’t >>> settle on which one. >>> >>> SeaweedFS seems fine, but PyIceberg has already moved to RustFS and >>> Polaris is also using it. I don’t have a strong preference between the >>> different options, so I think using RustFS across the Iceberg repos for >>> consistency would be the simplest option, unless there’s something we >>> specifically need from SeaweedFS. >>> >>> I think we should just go ahead and make the change. I can help review >>> the PR. >>> >>> On Fri, Sep 11, 2026 at 6:21 PM Sreesh Maheshwar < >>> [email protected]> wrote: >>> >>> Hi all, >>> >>> Wanted to revive this question - today, we saw some CI blockages [1] >>> across Iceberg projects due to MinIO images being removed from Docker Hub, >>> which adds another data point to moving away from MinIO. >>> >>> Worth noting that have been similar threads to this one in the past [2], >>> [3] - also, Polaris now uses RustFS for its quickstart, with RustFS and >>> Floci for integration tests (separating those concerns) [4], and PyIceberg >>> has moved to RustFS [5]. >>> >>> Wondering what the community now thinks about moving away from MinIO for >>> quickstarts and integration tests? >>> >>> [1] Iceberg Java: https://github.com/apache/iceberg/pull/18071 >>> [2] https://lists.apache.org/thread/vnw9jonmfcsz6bwojhfch1nmywyl50h3 >>> [3] https://lists.apache.org/thread/prmpbfy6v20spsmts19hs8gbkkxo16vv >>> [4] https://lists.apache.org/thread/8o31ly7cd8ov70opjbtg630qlhrfl5yh >>> [5] https://github.com/apache/iceberg-python/pull/3928 >>> >>> >>> Thanks, >>> >>> Sreesh >>> >>> On Fri, Mar 20, 2026 at 5:38 AM Chris Lu <[email protected]> wrote: >>> >>> >>> Hi everyone, >>> >>> I’ve been following the recent discussions around S3-compatible backends >>> for local testing and CI, especially the challenges around S3 compatibility >>> and credential-related testing. >>> >>> I’d like to propose an approach to help address: >>> https://github.com/apache/iceberg/issues/14638 >>> >>> I’ve opened an initial PR that switches the test backend from MinIO to >>> SeaweedFS: >>> https://github.com/apache/iceberg/pull/15577 >>> >>> The goal is to improve test reliability and better support scenarios >>> that are currently difficult to validate. >>> >>> SeaweedFS is an S3-compatible storage system with support for IAM-style >>> access control and STS flows. One reason I explored this direction is that >>> it may help with testing credential-related scenarios (e.g., temporary >>> credentials / vended credentials), which have been discussed recently. >>> >>> There are also a couple of areas that could potentially expand test >>> coverage over time: >>> >>> - Table-oriented bucket layouts could make it easier to exercise >>> Iceberg-specific storage patterns that are currently hard to simulate in CI >>> >>> - Support for IAM and STS flows could allow better validation of >>> credential-related behaviors used by some catalog integrations >>> >>> The current PR is intentionally minimal and focuses only on getting the >>> existing tests running. If this direction is useful, I’m planning to follow >>> up with additional integration tests (e.g., around table bucket behaviors). >>> >>> I want to emphasize that this is mainly to address #14638 and improve >>> coverage. If replacing the backend is too disruptive, I’m also happy to >>> explore alternatives such as running multiple backends or limiting usage to >>> specific scenarios. >>> >>> I’d really appreciate feedback on: >>> >>> - Whether switching the backend is a reasonable approach for #14638 >>> - Any concerns around CI stability or maintenance >>> - Whether running multiple backends would be preferable >>> - Any specific scenarios we should prioritize testing >>> >>> For context, I’m the author of SeaweedFS. SeaweedFS is open source under >>> the Apache 2.0 license. >>> >>> Thanks for your time and guidance! >>> >>> Chris >>> >>> Xuanwo >>> >>> https://xuanwo.io/ >>> >>>
