guypayeur commented on issue #3742: URL: https://github.com/apache/polaris/issues/3742#issuecomment-4553996752
For anyone hitting this against MinIO (or any S3-compatible storage with limited STS), the working config combo is in Polaris's own integration test [`RestCatalogMinIOSpecialIT.java`](https://github.com/apache/polaris/blob/main/runtime/service/src/intTest/java/org/apache/polaris/service/it/RestCatalogMinIOSpecialIT.java) — but it isn't surfaced in the user-facing configuration reference, which made it take ~6 hours to find. Three pieces have to align: **1. Server-level Quarkus application properties** (NOT environment variables): ```properties polaris.storage.aws.access-key=<minio-root-user> polaris.storage.aws.secret-key=<minio-root-secret> polaris.features."SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION"=false ``` Counter-intuitive: `SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION` **must be `false`**. Setting it to `true` bypasses the storage-config path entirely, which means `pathStyleAccess` + `endpoint` never reach the S3 client, and the SDK falls back to virtual-hosted-style URLs (`bucket.endpoint`) → `UnknownHostException`. **2. Catalog `storageConfigInfo`** with all three together: ```json "storageConfigInfo": { "storageType": "S3", "allowedLocations": ["s3://my-bucket/"], "endpoint": "http://minio:9000";, "region": "us-east-1", "stsUnavailable": true, "pathStyleAccess": true } ``` **3. Catalog `properties` keys must be prefixed with `table-default.`**: ```json "properties": { "default-base-location": "s3://my-bucket/my-catalog", "table-default.s3.endpoint": "http://minio:9000";, "table-default.s3.access-key-id": "<minio-root-user>", "table-default.s3.secret-access-key": "<minio-root-secret>", "table-default.s3.path-style-access": "true", "table-default.s3.region": "us-east-1" } ``` The `table-default.` prefix is what makes the IT test work — it uses `TABLE_DEFAULT_PREFIX + AWS_KEY_ID.getPropertyName()` to construct these keys. Without the prefix, the response to the original reporter's symptom is exactly `"Credential vending was requested for table ... but no credentials are available"`. With all three, `create_table` → `append` → `scan` works end-to-end against MinIO. Verified on Polaris 1.5.0 + PyIceberg 0.9 + MinIO `2025-04-22`. Would be helpful to surface this pattern in the production-configuration docs under "Configuring AWS S3 Cloud Storage" — happy to draft a PR if a maintainer can confirm the recommended docs location. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
