nevzheng opened a new pull request, #13155: URL: https://github.com/apache/gravitino/pull/13155
### What changes were proposed in this pull request? Replace the shared MinIO fixture with RustFS **1.0.0-rc.6** for fileset connection testing and Iceberg credential vending. The fixture consumes one version-and-digest pin from a packaged Docker Compose resource, waits for signed S3 readiness, and creates buckets through the AWS SDK. Preserve the Hadoop and Spark/S3FileIO tests, strengthen Spark write-denial assertions to require `403 AccessDenied`, and add focused checks of Gravitino's generated inline STS policy: read-only sessions, object and bucket boundaries, both listing APIs, Hadoop's deliberate bare-prefix behavior, deletion, and multipart upload. Image index: `rustfs/rustfs:1.0.0-rc.6@sha256:97171b3d72cd47dc81000f92ea84de25608bfc35a94c965501afaeb5d99f6035`. Published `linux/amd64` and `linux/arm64` manifests were verified. ### Why are the changes needed? Move these tests beyond the MinIO image-availability workaround in #13112 while retaining their storage-authorization coverage. Related to #13154; automated image updates and vulnerability reporting are handled in a separate follow-up PR. ### Does this PR introduce _any_ user-facing change? No. This changes test infrastructure only. RustFS is a release candidate. These checks establish caller-plus-inline-policy enforcement, not AWS named-role trust or external-ID enforcement. The generated policy does not grant `AbortMultipartUpload`; the focused multipart test uses root credentials only to clean up unfinished uploads after failure. Further boundaries and commands are documented in [the fixture README](https://github.com/nevzheng/gravitino/blob/ffd21dd5ca8dedcca023731adb2aa5809889b566/integration-test-common/README-rustfs.md). ### How was this patch tested? On Corretto 17.0.19 and Docker 29.4/OrbStack with the pinned `linux/arm64` image: - Image-pin parsing: **2 passed**. - Signed STS policy and multipart probes: **5 passed**. - Spark/S3FileIO owner, modify, select-only, and active-role tests: **4 passed**. - Hadoop fileset catalog connection: **1 passed**. - Changed-module Spotless checks and `git diff --check` passed. Native Linux amd64 and deploy-mode execution remain for the existing backend CI matrix. This Mac's build-output redirection needed its documented compatibility switch because the existing catalog loader expects module-local `build/resources/main` paths. ```shell ./gradlew :catalogs:catalog-lakehouse-iceberg:jar \ :catalogs:catalog-fileset:jar :catalogs:hadoop-common:jar \ -DcentralBuildOutputs.disabled=true ./gradlew :integration-test-common:test \ --tests org.apache.gravitino.integration.test.container.TestRustFSContainer \ :catalogs:catalog-fileset:test \ --tests org.apache.gravitino.catalog.fileset.integration.test.FilesetS3TokenConnectionIT \ :iceberg:iceberg-rest-server:test \ --tests org.apache.gravitino.iceberg.integration.test.IcebergRESTRustFSTokenAuthorizationIT \ --tests org.apache.gravitino.iceberg.integration.test.RustFSS3TokenIT \ -PskipDockerTests=false -DcentralBuildOutputs.disabled=true ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
