This is now done across all the iceberg repos (see https://github.com/apache/iceberg/issues/18246). Thank you Sreesh!
We should be unblocked for the RC On Thu, Sep 24, 2026 at 2:57 PM Neelesh Salian <[email protected]> wrote: > Tracking the issue of the integ tests failures here: > https://github.com/apache/iceberg/issues/18246 > There is an open PR to get CI unblocked for now: > https://github.com/apache/iceberg/pull/18247 > > On Thu, Sep 24, 2026 at 1:24 PM Steven Wu <[email protected]> wrote: > >> Where are we in this process? >> >> We have some Kafka connect integration test failures in the CI build that >> seem related to the minio problem. e.g. >> >> https://github.com/apache/iceberg/actions/runs/36037765015/job/107788134738?pr=18213 >> >> https://github.com/apache/iceberg/actions/runs/36047551910/job/107794733276?pr=18248 >> >> Docker compose loads the MinIO service. >> >> https://github.com/apache/iceberg/blob/main/kafka-connect/kafka-connect-runtime/docker/docker-compose.yml#L23 >> >> >> On Mon, Sep 14, 2026 at 11:12 AM Neelesh Salian <[email protected]> >> wrote: >> >>> I know in another thread there were some CVE concerns raised about >>> RustFS (mailing list thread titled: "[DISCUSS] Replacing minio with RustFS >>> in quick start demo.") >>> I see a blog referencing them is here: >>> https://rustfs.com/blog/patch-release-rustfs-1-0-0-alpha-79-released/ >>> Ok with either choice as long as there is long term support for RustFs >>> and for security patches with prompt fixes when CVEs show up. >>> >>> On Mon, Sep 14, 2026 at 11:02 AM Szehon Ho <[email protected]> >>> wrote: >>> >>>> It also sounds good to me, and also don't have any other strong >>>> preference. >>>> >>>> Thanks >>>> Szehon >>>> >>>> On Mon, Sep 14, 2026 at 9:03 AM Xuanwo <[email protected]> wrote: >>>> >>>>> RustFS LGTM. >>>>> >>>>> It's just for CI and should be easy to change in the future. I agree >>>>> with Kevin that let's just move. >>>>> >>>>> On Mon, Sep 14, 2026, at 23:55, Kevin Liu wrote: >>>>> >>>>> Thanks for the PRs to fix the MinIO issue blocking CI. I think the >>>>> fixes have now been merged across all the affected Iceberg repos. >>>>> >>>>> +1 on moving away from MinIO. Last time we discussed this, I think >>>>> there was general agreement on moving to another object store, but we >>>>> didn’t settle on which one. >>>>> >>>>> SeaweedFS seems fine, but PyIceberg has already moved to RustFS and >>>>> Polaris is also using it. I don’t have a strong preference between the >>>>> different options, so I think using RustFS across the Iceberg repos for >>>>> consistency would be the simplest option, unless there’s something we >>>>> specifically need from SeaweedFS. >>>>> >>>>> I think we should just go ahead and make the change. I can help review >>>>> the PR. >>>>> >>>>> On Fri, Sep 11, 2026 at 6:21 PM Sreesh Maheshwar < >>>>> [email protected]> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> Wanted to revive this question - today, we saw some CI blockages [1] >>>>> across Iceberg projects due to MinIO images being removed from Docker Hub, >>>>> which adds another data point to moving away from MinIO. >>>>> >>>>> Worth noting that have been similar threads to this one in the past >>>>> [2], [3] - also, Polaris now uses RustFS for its quickstart, with RustFS >>>>> and Floci for integration tests (separating those concerns) [4], and >>>>> PyIceberg has moved to RustFS [5]. >>>>> >>>>> Wondering what the community now thinks about moving away from MinIO >>>>> for quickstarts and integration tests? >>>>> >>>>> [1] Iceberg Java: https://github.com/apache/iceberg/pull/18071 >>>>> [2] https://lists.apache.org/thread/vnw9jonmfcsz6bwojhfch1nmywyl50h3 >>>>> [3] https://lists.apache.org/thread/prmpbfy6v20spsmts19hs8gbkkxo16vv >>>>> [4] https://lists.apache.org/thread/8o31ly7cd8ov70opjbtg630qlhrfl5yh >>>>> [5] https://github.com/apache/iceberg-python/pull/3928 >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Sreesh >>>>> >>>>> On Fri, Mar 20, 2026 at 5:38 AM Chris Lu <[email protected]> wrote: >>>>> >>>>> >>>>> Hi everyone, >>>>> >>>>> I’ve been following the recent discussions around S3-compatible >>>>> backends for local testing and CI, especially the challenges around S3 >>>>> compatibility and credential-related testing. >>>>> >>>>> I’d like to propose an approach to help address: >>>>> https://github.com/apache/iceberg/issues/14638 >>>>> >>>>> I’ve opened an initial PR that switches the test backend from MinIO to >>>>> SeaweedFS: >>>>> https://github.com/apache/iceberg/pull/15577 >>>>> >>>>> The goal is to improve test reliability and better support scenarios >>>>> that are currently difficult to validate. >>>>> >>>>> SeaweedFS is an S3-compatible storage system with support for >>>>> IAM-style access control and STS flows. One reason I explored this >>>>> direction is that it may help with testing credential-related scenarios >>>>> (e.g., temporary credentials / vended credentials), which have been >>>>> discussed recently. >>>>> >>>>> There are also a couple of areas that could potentially expand test >>>>> coverage over time: >>>>> >>>>> - Table-oriented bucket layouts could make it easier to exercise >>>>> Iceberg-specific storage patterns that are currently hard to simulate in >>>>> CI >>>>> >>>>> - Support for IAM and STS flows could allow better validation of >>>>> credential-related behaviors used by some catalog integrations >>>>> >>>>> The current PR is intentionally minimal and focuses only on getting >>>>> the existing tests running. If this direction is useful, I’m planning to >>>>> follow up with additional integration tests (e.g., around table bucket >>>>> behaviors). >>>>> >>>>> I want to emphasize that this is mainly to address #14638 and improve >>>>> coverage. If replacing the backend is too disruptive, I’m also happy to >>>>> explore alternatives such as running multiple backends or limiting usage >>>>> to >>>>> specific scenarios. >>>>> >>>>> I’d really appreciate feedback on: >>>>> >>>>> - Whether switching the backend is a reasonable approach for #14638 >>>>> - Any concerns around CI stability or maintenance >>>>> - Whether running multiple backends would be preferable >>>>> - Any specific scenarios we should prioritize testing >>>>> >>>>> For context, I’m the author of SeaweedFS. SeaweedFS is open source >>>>> under the Apache 2.0 license. >>>>> >>>>> Thanks for your time and guidance! >>>>> >>>>> Chris >>>>> >>>>> Xuanwo >>>>> >>>>> https://xuanwo.io/ >>>>> >>>>>
