RustFS LGTM. It's just for CI and should be easy to change in the future. I agree with Kevin that let's just move.
On Mon, Sep 14, 2026, at 23:55, Kevin Liu wrote: > Thanks for the PRs to fix the MinIO issue blocking CI. I think the fixes have > now been merged across all the affected Iceberg repos. > > +1 on moving away from MinIO. Last time we discussed this, I think there was > general agreement on moving to another object store, but we didn’t settle on > which one. > > SeaweedFS seems fine, but PyIceberg has already moved to RustFS and Polaris > is also using it. I don’t have a strong preference between the different > options, so I think using RustFS across the Iceberg repos for consistency > would be the simplest option, unless there’s something we specifically need > from SeaweedFS. > > I think we should just go ahead and make the change. I can help review the PR. > > On Fri, Sep 11, 2026 at 6:21 PM Sreesh Maheshwar <[email protected]> > wrote: >> Hi all, >> >> Wanted to revive this question - today, we saw some CI blockages [1] across >> Iceberg projects due to MinIO images being removed from Docker Hub, which >> adds another data point to moving away from MinIO. >> >> Worth noting that have been similar threads to this one in the past [2], [3] >> - also, Polaris now uses RustFS for its quickstart, with RustFS and Floci >> for integration tests (separating those concerns) [4], and PyIceberg has >> moved to RustFS [5]. >> >> Wondering what the community now thinks about moving away from MinIO for >> quickstarts and integration tests? >> >> [1] Iceberg Java: https://github.com/apache/iceberg/pull/18071 >> [2] https://lists.apache.org/thread/vnw9jonmfcsz6bwojhfch1nmywyl50h3 >> [3] https://lists.apache.org/thread/prmpbfy6v20spsmts19hs8gbkkxo16vv >> [4] https://lists.apache.org/thread/8o31ly7cd8ov70opjbtg630qlhrfl5yh >> [5] https://github.com/apache/iceberg-python/pull/3928 >> >> >> >> Thanks, >> >> Sreesh >> >> >> On Fri, Mar 20, 2026 at 5:38 AM Chris Lu <[email protected]> wrote: >>> >>> Hi everyone, >>> >>> I’ve been following the recent discussions around S3-compatible backends >>> for local testing and CI, especially the challenges around S3 compatibility >>> and credential-related testing. >>> >>> I’d like to propose an approach to help address: >>> https://github.com/apache/iceberg/issues/14638 >>> >>> I’ve opened an initial PR that switches the test backend from MinIO to >>> SeaweedFS: >>> https://github.com/apache/iceberg/pull/15577 >>> >>> The goal is to improve test reliability and better support scenarios that >>> are currently difficult to validate. >>> >>> SeaweedFS is an S3-compatible storage system with support for IAM-style >>> access control and STS flows. One reason I explored this direction is that >>> it may help with testing credential-related scenarios (e.g., temporary >>> credentials / vended credentials), which have been discussed recently. >>> >>> There are also a couple of areas that could potentially expand test >>> coverage over time: >>> >>> - Table-oriented bucket layouts could make it easier to exercise >>> Iceberg-specific storage patterns that are currently hard to simulate in CI >>> >>> - Support for IAM and STS flows could allow better validation of >>> credential-related behaviors used by some catalog integrations >>> >>> The current PR is intentionally minimal and focuses only on getting the >>> existing tests running. If this direction is useful, I’m planning to follow >>> up with additional integration tests (e.g., around table bucket behaviors). >>> >>> I want to emphasize that this is mainly to address #14638 and improve >>> coverage. If replacing the backend is too disruptive, I’m also happy to >>> explore alternatives such as running multiple backends or limiting usage to >>> specific scenarios. >>> >>> I’d really appreciate feedback on: >>> >>> - Whether switching the backend is a reasonable approach for #14638 >>> - Any concerns around CI stability or maintenance >>> - Whether running multiple backends would be preferable >>> - Any specific scenarios we should prioritize testing >>> >>> For context, I’m the author of SeaweedFS. SeaweedFS is open source under >>> the Apache 2.0 license. >>> >>> Thanks for your time and guidance! >>> >>> Chris >>> Xuanwo https://xuanwo.io/
