dwsmith1983 opened a new pull request, #6887:
URL: https://github.com/apache/datafusion-comet/pull/6887
## Which issue does this PR close?
Closes #6884.
## Rationale for this change
Docker Hub no longer serves `minio/minio`, so every suite built on
`CometS3TestBase` aborts in `beforeAll` before a test runs. Once the image
pulls, two more gaps show up that only stayed hidden because these suites ran
on machines with an AWS region configured:
- the client `createBucketIfNotExists` builds has no region, so the AWS
SDK's region chain fails;
- the REST catalog tests vend `s3.region`, which iceberg-rust reads, but
Iceberg's JVM `S3FileIO` reads `client.region`, so the INSERT that writes the
table fails the same way.
## What changes are included in this PR?
- `CometS3TestBase` starts `cgr.dev/chainguard/minio:latest`, declared with
`asCompatibleSubstituteFor("minio/minio")` so `MinIOContainer` accepts it.
Chainguard's free images are only published as `latest`, and the old reference
was `minio/minio:latest` too.
- The bucket setup client sets `Region.US_EAST_1`, the region the suites
already pass to the catalogs.
- The REST catalog vended properties in `IcebergReadFromS3Suite` and
`CometS3CredentialBridgeSuite` add `client.region`, as
`CometIcebergWriteDetectionSuite` already does. `s3.region` stays for the
native scan.
## How are these changes tested?
No workflow runs these suites, so I ran them in a Linux container with no
AWS configuration. On `main`, `ParquetReadFromS3Suite` aborts with `pull access
denied for minio/minio`. With this change, `ParquetReadFromS3Suite`,
`IcebergReadFromS3Suite`, `CometS3CredentialBridgeSuite` and
`HadoopS3ACredentialProviderAdapterBridgeSuite` pass (27 tests), and the
`-Pspark-3.5 -Pstrict-warnings` test compile passes.
`REST catalog credential vending rejects wrong credentials` now gets the 403
it checks for. Without a region it failed on a region error instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
