blaginin commented on code in PR #13672:
URL: https://github.com/apache/datafusion/pull/13672#discussion_r1891645256
##########
.github/workflows/rust.yml:
##########
@@ -165,22 +165,33 @@ jobs:
name: cargo test datafusion-cli (amd64)
needs: [ linux-build-lib ]
runs-on: ubuntu-latest
- container:
- image: amd64/rust
+ env:
+ AWS_ENDPOINT: http://127.0.0.1:9000
+ AWS_ACCESS_KEY_ID: TEST-DataFusionLogin
+ AWS_SECRET_ACCESS_KEY: TEST-DataFusionPassword
+ TEST_STORAGE_INTEGRATION: 1
+ AWS_ALLOW_HTTP: true
+
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: Setup Rust toolchain
- uses: ./.github/actions/setup-builder
- with:
- rust-version: stable
- - name: Run tests (excluding doctests)
run: |
- cd datafusion-cli
- cargo test --lib --tests --bins --all-features
+ rustup toolchain install stable
+ rustup default stable
+ - name: Setup Minio - S3-compatible storage
+ working-directory: datafusion-cli
+ run:
+ echo "MINIO_CONTAINER=$(docker run -d -p 9000:9000 -e
MINIO_ROOT_USER=TEST-DataFusionLogin -e
MINIO_ROOT_PASSWORD=TEST-DataFusionPassword quay.io/minio/minio server /data)"
>> $GITHUB_ENV
+ - name: Run tests (excluding doctests, but with integration tests)
+ working-directory: datafusion-cli
+ run: cargo test --lib --tests --bins --all-features
Review Comment:
@alamb you sugessting copying `object_store` [approach for
testing](https://github.com/apache/arrow-rs/blob/9047d99f6bf87582532ee6ed0acb3f2d5f889f11/.github/workflows/object_store.yml#L91-L184).
In `object_store`, they use Localstack for S3 simulation. It works fine for
testing, but the problem is that [it doesn't actually validate the
credentials](https://github.com/localstack/localstack/blob/master/localstack-core/localstack/services/s3/provider.py#L840-L842).
In another part of `object_store`, [Minio is
used](https://github.com/apache/arrow-rs/blob/fc814bca6b743010bcde972e611c8ff8ea68c9f5/object_store/CONTRIBUTING.md?plain=1#L108),
and it *does* validate credentials. So, I think we should switch to using
Minio for testing here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
